Showing posts with label SQL Server 2019. Show all posts
Showing posts with label SQL Server 2019. Show all posts

Monday, January 31, 2022

Things you shouldn't really do in SQL Server: Disabling Join Types Globally

A while ago I presented a session which covered transformation rules that are used by the query optimiser to produce our execution plans. I'm not feeling in the mood for relational algebra this morning so instead I'll introduce a command that can cause mayhem on an instance SQL Server: DBCC RULEOFF.

DBCC RULEOFF is an undocumented command, that alone makes me want to use it but in order to cover one's backside please don't do any of the following in any environment apart from a disposable sandbox that only you use for weird and wonderful experiments in SQL Server, because we are going to break it...

Let's take a simple query with a join:

SELECT e.BusinessEntityID, e.JobTitle, p.FirstName
FROM HumanResources.Employee e
INNER JOIN Person.Person p
ON e.BusinessEntityID = p.BusinessEntityID;

If I look at the estimated execution plan I can see that the optimiser has selected a nested loop join:


Now I could use the QUERYRULEOFF hint to disable that join for this particular query and for this particular join type I'd need two different hints to disable nested loop operators for the join:

OPTION (QUERYRULEOFF JNtoNL, QUERYRULEOFF JNtoIdxLookup);

This will force the optimiser to use a different join type and the estimated plan this time shows a hash match join but for this article we're not looking at QUERYRULEOFF, we're looking at DBCC RULEOFF which will still do the same thing only it's not for one query, it's for the entire instance.

MWAHAHAHAHAHAHA

That's right, running DBCC RULEOFF('JNtoNL') will essentially switch off the nested loop join for every single query running on my instance, well, sort of. After running the command I can check what rules are off on my instance by first running DBCC TRACEON(3604) to redirect DBCC output to my messages tab and then run DBCC SHOWOFFRULES:

Rules that are off globally:

JNtoNL

That's just one of the rules though, so just as when I used the query hint with without nested loop joins I'll also run DBCC RULEOFF('JNtoIdxLookup') to disable the other transformation rule that the optimiser can use for a nested loop join and verify again with DBCC SHOWOFFRULES. Now that they're both disabled the estimated plan looks a bit different and is using a hash match operator instead.

A different logical join type will not be affected. If I change the INNER JOIN to a LEFT JOIN and check the execution plan I will see a nested loop operator again.

SELECT e.BusinessEntityID, e.JobTitle, p.FirstName
FROM HumanResources.Employee e
LEFT JOIN Person.Person p
ON e.BusinessEntityID = p.BusinessEntityID;

This is how transformation rules work, essentially they are a substitution for a logical operation (INNER JOIN, LEFT JOIN etc) for a physical operation, that is the operator we see in the execution plan. So far we've only disabled the following rules JNtoNL and JNtoIdxLookup and they only affect the INNER JOIN logical operations. 

For left joins we'll need to switch off the corresponding transformation rules (LOJNtoNL and LeftSideJNtoIdxLookup) and when done we can see the same effect on the estimated execution plan as before where the optimiser has used a hash match join.

Now whilst I can make a case of disabling rules at a query level to check execution plans using different join types (or you could use join hints) I can't really make the same case for disabling them at an instance level. But this post is titled "Things you shouldn't really do..." so let's do something completely reckless.

If I run the following query I can see every transformation rule that contains 'JN' as in JOIN and I've also included the command to disable that rule:

SELECT [name], 'DBCC RULEOFF (''' + [name] + ''');' AS [DontDoIt!!!]
FROM sys.dm_exec_query_transformation_stats
WHERE [name] LIKE '%JN%';

For no other reason but for widespread chaos I can now disable every transformation rule that substitute joins by running each all of the commands returned by my query. Now things are so bad that if I try to run my query I get the following error:

Msg 8624, Level 16, State 1, Line 5
Internal Query Processor Error: The query processor could not produce a query plan. For more information, contact Customer Support Services.

I think that means it's broken, at least if I want to join any tables anyway!

So please, please, please don't do this. There's literally no reason to do so on any SQL system of any kind. I will put everything back by running the following so I can generate the commands to switch on all rules once again and verify with DBCC SHOWOFFRULES that nothing has remained disabled. 

SELECT [name], 'DBCC RULEON (''' + [name] + ''');' AS [DoIt!!!]
FROM sys.dm_exec_query_transformation_stats
WHERE [name] LIKE '%JN%';

Another reason why I stay well away from relational algebra.

Friday, January 28, 2022

SQL 2019: Physical reads are counted two times for read-aheads in sys.dm_exec_query_stats


Microsoft recently released Cumulative Update 15 for SQL Server 2019. It contains a bunch of fixes and some improvements, I get a bit geeky with updates like this and love to have a look through the different fixes to see 

"Physical reads for read-ahead reads are counted incorrectly (two times) when you run queries. Therefore, the information in sys.query_store_runtime_stats and sys.dm_exec_query_stats shows incorrect values."

So if you're using these management views to look at performance metrics for your queries you're going to get incorrect results and you might be thinking the queries are doing way more work than what they're actually doing.

In order to test this in a before and after update type scenario I'm going to first force SQL Server to clean out it's buffer pool using DBCC DROPCLEANBUFFERS and the procedure cache with DBCC FREEPROCCACHE (btw, don't do this on anything else other than a sandbox environment). This will ensure the next time I run a query it will have no pages in memory and will have to retrieve the pages from disk, and I'm clearing the procedure cache as I'm going to be querying the sys.dm_exec_query_stats DMV.

Test query: 

SELECT * FROM Person.Person  

I've set SET STATISTICS IO ON so I can see after my query has ran the logical, and in this case more importantly the physical reads of my query and it shows 3 physical reads and 3866 read-ahead reads (which are still from disk):

Table 'Person'. Scan count 1, logical reads 3821, physical reads 3, page server reads 0, read-ahead reads 3866

SELECT [text] AS QueryText, last_physical_reads FROM sys.dm_exec_query_stats
CROSS APPLY sys.dm_exec_sql_text(sql_handle)
WHERE [text] = 'SELECT * FROM Person.Person'

And I get the following results showing the last physical reads of my query was 7779, which is wrong:


I've now updated one of my test instances to CU15 and will run the same queries as before, and this time we have a much more accurate value returned from the DMV:


Now I should add there's still a difference between the statistics output and the sys.dm_exec_query_stats DMV for physical reads and to be totally honest I don't know exactly why but I've the question using #sqlhelp on Twitter and will update when I find out!

Thursday, January 13, 2022

Getting locked out of SQL Server: Cannot open user default database. Login Failed.

I love playing around with test instances. Recently I've been doing a lot of preparation for some forthcoming blog posts and when trying to log into SSMS this morning I received the following error message:


Unfortunately as it's one of my test instances I'm the only person using it, which also means I'm the only person responsible for creating this problem but we don't have a blame culture around here so let's get on with fixing it and I'll have a lessons learned with myself later over a glass of something.

This error isn't to do with my login as such, it's still there with sysadmin role membership so I don't have to do anything too drastic like restarting SQL Server with the -m or -f startup parameters and recreate it. The error message is telling me that my logins default database cannot be opened, which is more than likely because I've deleted it.

To resolve the issue I just need to connect to a different database but thing is it's not immediately obvious how to do that (and I don't have another login available to simply change my default database). What I need to do is click the Options >> button on the bottom right of the Connect to Server dialog box:


If I then go to the Connection Properties tab I can then use the Connect to database option and browse the server for another database to connect to:


But there's a problem, in order to browse the server I need to connect to it and as I can't do that right now I'm going to run into the same problem again:


Which sucks...


Fortunately I don't have to browse the server and choose a database to connect to. I can just type into the connect to database box and put in a valid database name such as master like in the box below:


Now I've put in a valid database I can successfully connect to my instance in Management Studio and the first thing that I will do is change my logins default database to master with the following T-SQL and the issue won't re-occur:

ALTER LOGIN [DESKTOP-MFKSHR7\David] WITH DEFAULT_DATABASE = master;

Thursday, January 6, 2022

SQL Server on Ubuntu 20.04: System has not been booted with systemd as init system (PID 1). Can't operate.

I'm a Windows person. I know this because for two reasons; I've used Windows for a long, long time and am very comfortable with it but also when I venture to different worlds like Linux and encounter errors I literally have no clue what I'm looking at, at least with Windows I might have some sort of idea of where to start!

Earlier I was trying to install SQL Server on Ubuntu 20.04 on my Windows 10 laptop which is using Windows Subsystem for Linux 2.0 following this quickstart guide from Microsoft. 

I'm not going to go through the steps itself but the fun and games started when I got to this command: sudo /opt/mssql/bin/mssql-conf setup

Things originally went fine, and I got the following options for the edition of SQL Server I wanted to install, which I love because it saves having lots of installation media hanging around:


I chose the Developer edition, set my sa password then this happened:

ForceFlush is enabled for this instance.

ForceFlush feature is enabled for log durability.

System has not been booted with systemd as init system (PID 1). Can't operate.

Failed to connect to bus: Host is down

Attempting to start the Microsoft SQL Server service failed.

I'm going to cut a long story short because I'm not a Linux person and thankfully via this post on Github I found this excellent post which fully explained this issue (no support for systemd) and had a rather helpful resolution too which the author describes as a (dirty) trick!

That said, I still wasn't able to start SQL Server with sudo service mssql-server start ( as expected and instead used the following command:

sudo -u mssql /opt/mssql/bin/sqlservr -c -d/var/opt/mssql/data/master.mdf -l/var/opt/mssql/data/mastlog.ldf -e/var/opt/mssql/log/errorlog -x

Voila! SQL Server is now up and running and the solution has been saved to memory (well, the favourites section of my web browser).

Friday, November 8, 2019

SQL Server 2019, the vision is released.



As you probably know by now Microsoft announced the General Availability release of SQL Server 2019 earlier this week, here's the official announcement. For me it's the most significant release of SQL Server EVER; SQL 2005 was certainly groundbreaking and the 2016 version was amazing, sure, but for me SQL 2019 is the real game changer for Data Platforms.


There's a lot of material and blog posts out there detailing the new functionality and enhancements that come in SQL 2019 (and if you haven't already, you really need to start getting up to speed) but what I wanted to do in this post was look back at a post on the SQL Server blog from back in early 2014 by the then VP of the Data Platform Group at Microsoft, Quentin Clark.

The post is titled 'What Drives Microsoft's Data Platform Vision' and I've used this article quite often when I've been outlining how we should approach the concept of a data platform. It's a very interesting read and whilst it's very focused on the Azure offerings at the time we can see how this new release shows how the data platform vision outlined in this article shaped the future of SQL Server. I'll pull out a few sub-headings and points:

"Data types are diverse", "Examples include the rise of JSON, the embracing of Hadoop by enterprises" 

And (from the 2019 main page), we now have SQL Server fully embracing Hadoop:

"Manage your big data environment more easily with Big Data Clusters. They provide key elements of a data lake - Hadoop Distributed File System (HDFS), Apache Spark..."

Grabbing another snippet:

"Analytics usage is broadening", "Deep analytics and automated techniques, like machine learning, are being used more often"

OK so whilst Machine Learning Services were introduced in SQL Server 2017 the addition of Big Data Clusters coupled with R and Python brings a whole new level of analytical capabilities inside SQL Server.

The 2014 article also mentions the concept of "Modern Transaction Processing", to quote again "data services that modern applications need are broader now than traditional RDBMS". This statement for me sums up databases in general and now SQL Server has evolved from being traditional "place to store data" shall we say to being a true hub for all of your organisations enterprise data, regardless of its form - although PolyBase came along in 2016 (when it could connect to Hadoop) it's potential has grown massively in 2019 enabling external data connections to the likes of Oracle, Teradata and MongoDB.

The 2014 article is a great read and there's so many different points in it that have shaped this release of SQL Server and now that 2019 is released and here, Microsoft's vision for the Data Platform has become a reality. Bearing in mind that figures suggest the last release of SQL Server has a high adoption rate is an encouraging sign that organisations will also embrace the potential of SQL 2019 to fully realise, and extract real value from their own Data Platforms. 

Breaking up with SQL Server

I was inspired to write this after reading a post from Dave Mason regarding breaking up with Big Tech companies. Yet again I haven't wr...