mirror of
https://github.com/Microsoft/sql-server-samples.git
synced 2025-12-08 14:58:54 +00:00
Updating notebooks for SQL Server 2019
This commit is contained in:
Amit Banerjee
+80516
-80516
File diff suppressed because one or more lines are too long
+544
-544
File diff suppressed because it is too large
Load Diff
+461
-461
@@ -1,462 +1,462 @@
|
||||
{
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"name": "SQL",
|
||||
"display_name": "SQL",
|
||||
"language": "sql"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "sql",
|
||||
"version": ""
|
||||
}
|
||||
},
|
||||
"nbformat_minor": 2,
|
||||
"nbformat": 4,
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# SQL Server 2019 Standard Edition, Transparent Database Encryption and Azure Key Vault\r\n",
|
||||
"\r\n",
|
||||
"This notebook demonstrates the use of Azure Key Vault to enable TDE on a SQL Server 2019 Standard Edition database using EKM.\r\n",
|
||||
"\r\n",
|
||||
"**Pre-requisities** \r\n",
|
||||
"1. Install Python\r\n",
|
||||
"2. Install Azure CLI using the following command (You will need to ensure that the Python scripts folder is part of your PATH variable)\r\n",
|
||||
" <br> <code>pip install --user azure-cli</code>\r\n",
|
||||
"3. Log into Azure and create and Azure AD Service Principal\r\n",
|
||||
" <code><br> az login\r\n",
|
||||
" <br> az account set --subscription <-subscription id->\r\n",
|
||||
" <br> az ad sp create-for-rbac -n sqlaadtde --skip-assignment\r\n",
|
||||
" </code>\r\n",
|
||||
" <br> note the <b>appID</b> value which would be required later\r\n",
|
||||
" <code>\r\n",
|
||||
" <br>{\"appId\": \"<-guid->\",\r\n",
|
||||
" <br>\"displayName\": \"sqlaadtde\",\r\n",
|
||||
" <br>\"name\": \"http://sqlaadtde\",\r\n",
|
||||
" <br>\"password\": \"<-guid->\",\r\n",
|
||||
" <br>\"tenant\": \"<-guid->\"}\r\n",
|
||||
" </code>\r\n",
|
||||
"4. Create a new resource group and assign a newly created Azure Key Vault to the same resource group\r\n",
|
||||
" <code>\r\n",
|
||||
" <br> # Create a new resource group\r\n",
|
||||
" <br> az group create -n \"SQLTDEResourceGroup\" -l \"West US\"\r\n",
|
||||
" <br> # Register the Key Vault resource provider\r\n",
|
||||
" <br> az provider register -n Microsoft.KeyVault\r\n",
|
||||
" <br> az keyvault create --name \"SQLStandardKeyVault\" --resource-group \"SQLTDEResourceGroup\" --location \"West US\"\r\n",
|
||||
" </code>\r\n",
|
||||
"5. Register the Azure AD principl with AKV\r\n",
|
||||
" <code>\r\n",
|
||||
" <br> # Register the AAD principal\r\n",
|
||||
" <br> az keyvault set-policy --name \"SQLStandardKeyVault\" --spn <appID GUID from az ad create-for-rbac output> --key-permissions get list wrapKey unwrapKey --verbose\r\n",
|
||||
" </code>\r\n",
|
||||
"\r\n",
|
||||
"Details about the scenario is available [here](https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/setup-steps-for-extensible-key-management-using-the-azure-key-vault?view=sql-server-ver15#next-step)."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "f28683d2-1f27-42b3-8418-f6747ed878ec"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Configure SQL Server to use EKM\r\n",
|
||||
"Download and install the [SQL Server Connector](https://go.microsoft.com/fwlink/p/?LinkId=521700). (This should be done by the administrator of the SQL Server computer.) By default, the connector installs at <b>C:\\Program Files\\SQL Server Connector for Microsoft Azure Key Vault</b>. This location can be changed during setup. (If changed, adjust the scripts below.)\r\n",
|
||||
"<br>There is no interface for the Connector, but if it is installed successfully, the <b>Microsoft.AzureKeyVaultService.EKM.dll</b> is installed on the machine. This is the cryptographic EKM provider DLL that needs to be registered with SQL Server by using the CREATE CRYPTOGRAPHIC PROVIDER statement."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "c82cb49a-7e5c-419e-b109-da7d2f9defa0"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE master; \r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"sp_configure 'show advanced options', 1; \r\n",
|
||||
"GO \r\n",
|
||||
"RECONFIGURE WITH OVERRIDE; \r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"-- Enable EKM provider \r\n",
|
||||
"sp_configure 'EKM provider enabled', 1; \r\n",
|
||||
"GO \r\n",
|
||||
"RECONFIGURE WITH OVERRIDE; \r\n",
|
||||
"GO"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "4e7af918-31d1-40c2-a193-309672e1c0ef"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Configuration option 'show advanced options' changed from 1 to 1. Run the RECONFIGURE statement to install."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Configuration option 'EKM provider enabled' changed from 1 to 1. Run the RECONFIGURE statement to install."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.059"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 15
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Register (create) the SQL Server Connector as an EKM provider with SQL Server"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "27394b2a-b1da-497b-8c56-8a2fbd1af037"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"-- Create a cryptographic provider, using the SQL Server Connector\r\n",
|
||||
"-- which is an EKM provider for the Azure Key Vault. This example uses \r\n",
|
||||
"-- the name AzureKeyVault_EKM_Prov.\r\n",
|
||||
"\r\n",
|
||||
"CREATE CRYPTOGRAPHIC PROVIDER AzureKeyVault_EKM_Prov \r\n",
|
||||
"FROM FILE = 'C:\\Program Files\\SQL Server Connector for Microsoft Azure Key Vault\\Microsoft.AzureKeyVaultService.EKM.dll';\r\n",
|
||||
"GO \r\n",
|
||||
""
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "3f53701d-b27a-49a8-931e-e3319c1b6386"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.022"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 16
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Create a credential from your AAD Client ID and Secret that you can use to grant a SQL Server account access to your Azure key vault\r\n",
|
||||
"The IDENTITY here is the name of your Azure key vault.\r\n",
|
||||
"<br>The SECRET here is your AAD Client ID (with the hyphens removed) and your AAD Client Secret concatenanted together\r\n",
|
||||
"<br>You will need to create a \"New Client Secret\" for your Azure AD app registration i.e. *sqlaadtde*, which was created above. See steps [here](https://docs.microsoft.com/en-us/azure/healthcare-apis/register-confidential-azure-ad-client-app#application-secret)."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "e71869f8-0904-4167-8818-c7cb345915c0"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"CREATE CREDENTIAL Azure_EKM_TDE_cred\r\n",
|
||||
" WITH IDENTITY = 'SQLStandardKeyVault', -- for global Azure\r\n",
|
||||
" -- WITH IDENTITY = 'ContosoDevKeyVault.vault.usgovcloudapi.net', -- for Azure Government\r\n",
|
||||
" -- WITH IDENTITY = 'ContosoDevKeyVault.vault.azure.cn', -- for Azure China 21Vianet\r\n",
|
||||
" -- WITH IDENTITY = 'ContosoDevKeyVault.vault.microsoftazure.de', -- for Azure Germany \r\n",
|
||||
"\tSECRET = '<combination of AAD Client ID without hyphens and AAD Client Secret>'\r\n",
|
||||
" FOR CRYPTOGRAPHIC PROVIDER AzureKeyVault_EKM_Prov \r\n",
|
||||
"\r\n",
|
||||
""
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "600cf82f-0488-46da-a253-99146f20065b"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.012"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 25
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Create an Asymmetric Key using the AKV Key\r\n",
|
||||
"You will need to first create a Key in Azure Key Vault which can be used to create the Asymmetric Key in SQL Server\r\n",
|
||||
"<code>\r\n",
|
||||
"<br> # Create a software-protected key \r\n",
|
||||
"<br> az keyvault key create --vault-name \"SQLStandardKeyVault\" --name \"SQLTDEKey\" --protection software\r\n",
|
||||
"</code>"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "ef0ab374-182e-434b-b7d8-050b3a1c481b"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"CREATE ASYMMETRIC KEY dbAKV_Key \r\n",
|
||||
"FROM PROVIDER [AzureKeyVault_EKM_Prov] \r\n",
|
||||
"WITH PROVIDER_KEY_NAME = 'SQLTDEKey', -- This is the KEY that was created in the Azure Key Vault\r\n",
|
||||
"CREATION_DISPOSITION = OPEN_EXISTING;"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "923e5bc8-2751-4286-a0fe-fa37e1af59b0"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.145"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 27
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Now create a database and enable TDE on the database"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "c87fa2c2-1d3f-4dda-a2f9-d320af41486a"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE master; \r\n",
|
||||
"-- Create a SQL Server login associated with the asymmetric key \r\n",
|
||||
"-- for the Database engine to use when it loads a database \r\n",
|
||||
"-- encrypted by TDE. \r\n",
|
||||
"CREATE LOGIN TDE_Login \r\n",
|
||||
"FROM ASYMMETRIC KEY dbAKV_Key; \r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"-- Alter the TDE Login to add the credential for use by the \r\n",
|
||||
"-- Database Engine to access the key vault \r\n",
|
||||
"ALTER LOGIN TDE_Login \r\n",
|
||||
"ADD CREDENTIAL Azure_EKM_TDE_cred ; \r\n",
|
||||
"GO\r\n",
|
||||
"\r\n",
|
||||
"CREATE DATABASE dbAKVTDE;\r\n",
|
||||
"GO \r\n",
|
||||
"USE dbAKVTDE;\r\n",
|
||||
"GO\r\n",
|
||||
"\r\n",
|
||||
"CREATE DATABASE ENCRYPTION KEY \r\n",
|
||||
"WITH ALGORITHM = AES_256 \r\n",
|
||||
"ENCRYPTION BY SERVER ASYMMETRIC KEY dbAKV_Key; -- Use the key created above\r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"-- Alter the database to enable transparent data encryption. \r\n",
|
||||
"ALTER DATABASE dbAKVTDE \r\n",
|
||||
"SET ENCRYPTION ON; \r\n",
|
||||
"GO "
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "ee35f210-c356-4018-8994-057d16273b3a"
|
||||
},
|
||||
"outputs": [],
|
||||
"execution_count": 30
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Check if the database has been encrypted"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "6b8f4ad6-4998-4c0f-8fe5-f1a23d95055f"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE MASTER \r\n",
|
||||
"SELECT name,provider_type, algorithm_desc FROM sys.asymmetric_keys \r\n",
|
||||
"\r\n",
|
||||
"-- Check which databases are encrypted using TDE \r\n",
|
||||
"SELECT SERVERPROPERTY('Edition') as [Edition],d.name, dek.encryption_scan_state_desc, dek.encryptor_type \r\n",
|
||||
"FROM sys.dm_database_encryption_keys AS dek \r\n",
|
||||
"JOIN sys.databases AS d \r\n",
|
||||
" ON dek.database_id = d.database_id;"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "557fb1b5-7058-4888-986e-1ef2c03a1ab9"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "(1 row affected)"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "(3 rows affected)"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.048"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"metadata": {},
|
||||
"execution_count": 36,
|
||||
"data": {
|
||||
"application/vnd.dataresource+json": {
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "name"
|
||||
},
|
||||
{
|
||||
"name": "provider_type"
|
||||
},
|
||||
{
|
||||
"name": "algorithm_desc"
|
||||
}
|
||||
]
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"0": "dbAKV_Key",
|
||||
"1": "CRYPTOGRAPHIC PROVIDER",
|
||||
"2": "RSA_2048"
|
||||
}
|
||||
]
|
||||
},
|
||||
"text/html": "<table><tr><th>name</th><th>provider_type</th><th>algorithm_desc</th></tr><tr><td>dbAKV_Key</td><td>CRYPTOGRAPHIC PROVIDER</td><td>RSA_2048</td></tr></table>"
|
||||
}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"metadata": {},
|
||||
"execution_count": 36,
|
||||
"data": {
|
||||
"application/vnd.dataresource+json": {
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "Edition"
|
||||
},
|
||||
{
|
||||
"name": "name"
|
||||
},
|
||||
{
|
||||
"name": "encryption_scan_state_desc"
|
||||
},
|
||||
{
|
||||
"name": "encryptor_type"
|
||||
}
|
||||
]
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"0": "Standard Edition (64-bit)",
|
||||
"1": "tempdb",
|
||||
"2": "COMPLETE",
|
||||
"3": "ASYMMETRIC KEY"
|
||||
},
|
||||
{
|
||||
"0": "Standard Edition (64-bit)",
|
||||
"1": "dbTDE",
|
||||
"2": "COMPLETE",
|
||||
"3": "CERTIFICATE"
|
||||
},
|
||||
{
|
||||
"0": "Standard Edition (64-bit)",
|
||||
"1": "dbAKVTDE",
|
||||
"2": "COMPLETE",
|
||||
"3": "ASYMMETRIC KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"text/html": "<table><tr><th>Edition</th><th>name</th><th>encryption_scan_state_desc</th><th>encryptor_type</th></tr><tr><td>Standard Edition (64-bit)</td><td>tempdb</td><td>COMPLETE</td><td>ASYMMETRIC KEY</td></tr><tr><td>Standard Edition (64-bit)</td><td>dbTDE</td><td>COMPLETE</td><td>CERTIFICATE</td></tr><tr><td>Standard Edition (64-bit)</td><td>dbAKVTDE</td><td>COMPLETE</td><td>ASYMMETRIC KEY</td></tr></table>"
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 36
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# Best Practices\r\n",
|
||||
"To ensure quick key recovery and be able to access your data outside of Azure, we recommend the following best practices:\r\n",
|
||||
"<br> a. Create your encryption key locally on a local HSM device. (Make sure this is an asymmetric, RSA 2048 key so it's is supported by SQL Server.)\r\n",
|
||||
"<br> b. Import the encryption key to Azure Key Vault. See the steps in this [article](https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/setup-steps-for-extensible-key-management-using-the-azure-key-vault?view=sql-server-ver15#part-ii-create-a-key-vault-and-key) on how to do that.\r\n",
|
||||
"<br> c. Before using the key in Azure Key Vault for the first time, take an Azure Key Vault key backup. Learn more about the <b>Backup-AzureKeyVaultKey</b> command.\r\n",
|
||||
"Whenever any changes are made to the key (for example add ACLs, add tags, add key attributes), be sure to take another Azure Key Vault key backup.\r\n",
|
||||
"\r\n",
|
||||
"## Types of keys\r\n",
|
||||
"There are two types of keys you can generate in Azure Key Vault that will work with SQL Server. Both are asymmetric 2048-bit RSA keys.\r\n",
|
||||
"<br><b>Software-protected</b>: Processed in software and encrypted at rest. Operations on software-protected keys occur on Azure Virtual Machines. Recommended for keys not used in a production deployment.\r\n",
|
||||
"<br><b>HSM-protected</b>: Created and protected by a hardware security module (HSM) for additional security.\r\n",
|
||||
"\r\n",
|
||||
"For common troubleshooting and maintenance of the SQL Server Connector, please refer to [this article](https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/sql-server-connector-maintenance-troubleshooting?view=sql-server-ver15)."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "b74a3595-b2e9-4952-8e2a-c5876f740b70"
|
||||
}
|
||||
}
|
||||
]
|
||||
{
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"name": "SQL",
|
||||
"display_name": "SQL",
|
||||
"language": "sql"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "sql",
|
||||
"version": ""
|
||||
}
|
||||
},
|
||||
"nbformat_minor": 2,
|
||||
"nbformat": 4,
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# SQL Server 2019 Standard Edition, Transparent Database Encryption and Azure Key Vault\r\n",
|
||||
"\r\n",
|
||||
"This notebook demonstrates the use of Azure Key Vault to enable TDE on a SQL Server 2019 Standard Edition database using EKM.\r\n",
|
||||
"\r\n",
|
||||
"**Pre-requisities** \r\n",
|
||||
"1. Install Python\r\n",
|
||||
"2. Install Azure CLI using the following command (You will need to ensure that the Python scripts folder is part of your PATH variable)\r\n",
|
||||
" <br> <code>pip install --user azure-cli</code>\r\n",
|
||||
"3. Log into Azure and create and Azure AD Service Principal\r\n",
|
||||
" <code><br> az login\r\n",
|
||||
" <br> az account set --subscription <-subscription id->\r\n",
|
||||
" <br> az ad sp create-for-rbac -n sqlaadtde --skip-assignment\r\n",
|
||||
" </code>\r\n",
|
||||
" <br> note the <b>appID</b> value which would be required later\r\n",
|
||||
" <code>\r\n",
|
||||
" <br>{\"appId\": \"<-guid->\",\r\n",
|
||||
" <br>\"displayName\": \"sqlaadtde\",\r\n",
|
||||
" <br>\"name\": \"http://sqlaadtde\",\r\n",
|
||||
" <br>\"password\": \"<-guid->\",\r\n",
|
||||
" <br>\"tenant\": \"<-guid->\"}\r\n",
|
||||
" </code>\r\n",
|
||||
"4. Create a new resource group and assign a newly created Azure Key Vault to the same resource group\r\n",
|
||||
" <code>\r\n",
|
||||
" <br> # Create a new resource group\r\n",
|
||||
" <br> az group create -n \"SQLTDEResourceGroup\" -l \"West US\"\r\n",
|
||||
" <br> # Register the Key Vault resource provider\r\n",
|
||||
" <br> az provider register -n Microsoft.KeyVault\r\n",
|
||||
" <br> az keyvault create --name \"SQLStandardKeyVault\" --resource-group \"SQLTDEResourceGroup\" --location \"West US\"\r\n",
|
||||
" </code>\r\n",
|
||||
"5. Register the Azure AD principl with AKV\r\n",
|
||||
" <code>\r\n",
|
||||
" <br> # Register the AAD principal\r\n",
|
||||
" <br> az keyvault set-policy --name \"SQLStandardKeyVault\" --spn <appID GUID from az ad create-for-rbac output> --key-permissions get list wrapKey unwrapKey --verbose\r\n",
|
||||
" </code>\r\n",
|
||||
"\r\n",
|
||||
"Details about the scenario is available [here](https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/setup-steps-for-extensible-key-management-using-the-azure-key-vault?view=sql-server-ver15#next-step)."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "f28683d2-1f27-42b3-8418-f6747ed878ec"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Configure SQL Server to use EKM\r\n",
|
||||
"Download and install the [SQL Server Connector](https://go.microsoft.com/fwlink/p/?LinkId=521700). (This should be done by the administrator of the SQL Server computer.) By default, the connector installs at <b>C:\\Program Files\\SQL Server Connector for Microsoft Azure Key Vault</b>. This location can be changed during setup. (If changed, adjust the scripts below.)\r\n",
|
||||
"<br>There is no interface for the Connector, but if it is installed successfully, the <b>Microsoft.AzureKeyVaultService.EKM.dll</b> is installed on the machine. This is the cryptographic EKM provider DLL that needs to be registered with SQL Server by using the CREATE CRYPTOGRAPHIC PROVIDER statement."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "c82cb49a-7e5c-419e-b109-da7d2f9defa0"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE master; \r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"sp_configure 'show advanced options', 1; \r\n",
|
||||
"GO \r\n",
|
||||
"RECONFIGURE WITH OVERRIDE; \r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"-- Enable EKM provider \r\n",
|
||||
"sp_configure 'EKM provider enabled', 1; \r\n",
|
||||
"GO \r\n",
|
||||
"RECONFIGURE WITH OVERRIDE; \r\n",
|
||||
"GO"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "4e7af918-31d1-40c2-a193-309672e1c0ef"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Configuration option 'show advanced options' changed from 1 to 1. Run the RECONFIGURE statement to install."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Configuration option 'EKM provider enabled' changed from 1 to 1. Run the RECONFIGURE statement to install."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.059"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 15
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Register (create) the SQL Server Connector as an EKM provider with SQL Server"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "27394b2a-b1da-497b-8c56-8a2fbd1af037"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"-- Create a cryptographic provider, using the SQL Server Connector\r\n",
|
||||
"-- which is an EKM provider for the Azure Key Vault. This example uses \r\n",
|
||||
"-- the name AzureKeyVault_EKM_Prov.\r\n",
|
||||
"\r\n",
|
||||
"CREATE CRYPTOGRAPHIC PROVIDER AzureKeyVault_EKM_Prov \r\n",
|
||||
"FROM FILE = 'C:\\Program Files\\SQL Server Connector for Microsoft Azure Key Vault\\Microsoft.AzureKeyVaultService.EKM.dll';\r\n",
|
||||
"GO \r\n",
|
||||
""
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "3f53701d-b27a-49a8-931e-e3319c1b6386"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.022"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 16
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Create a credential from your AAD Client ID and Secret that you can use to grant a SQL Server account access to your Azure key vault\r\n",
|
||||
"The IDENTITY here is the name of your Azure key vault.\r\n",
|
||||
"<br>The SECRET here is your AAD Client ID (with the hyphens removed) and your AAD Client Secret concatenanted together\r\n",
|
||||
"<br>You will need to create a \"New Client Secret\" for your Azure AD app registration i.e. *sqlaadtde*, which was created above. See steps [here](https://docs.microsoft.com/en-us/azure/healthcare-apis/register-confidential-azure-ad-client-app#application-secret)."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "e71869f8-0904-4167-8818-c7cb345915c0"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"CREATE CREDENTIAL Azure_EKM_TDE_cred\r\n",
|
||||
" WITH IDENTITY = 'SQLStandardKeyVault', -- for global Azure\r\n",
|
||||
" -- WITH IDENTITY = 'ContosoDevKeyVault.vault.usgovcloudapi.net', -- for Azure Government\r\n",
|
||||
" -- WITH IDENTITY = 'ContosoDevKeyVault.vault.azure.cn', -- for Azure China 21Vianet\r\n",
|
||||
" -- WITH IDENTITY = 'ContosoDevKeyVault.vault.microsoftazure.de', -- for Azure Germany \r\n",
|
||||
"\tSECRET = '<combination of AAD Client ID without hyphens and AAD Client Secret>'\r\n",
|
||||
" FOR CRYPTOGRAPHIC PROVIDER AzureKeyVault_EKM_Prov \r\n",
|
||||
"\r\n",
|
||||
""
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "600cf82f-0488-46da-a253-99146f20065b"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.012"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 25
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Create an Asymmetric Key using the AKV Key\r\n",
|
||||
"You will need to first create a Key in Azure Key Vault which can be used to create the Asymmetric Key in SQL Server\r\n",
|
||||
"<code>\r\n",
|
||||
"<br> # Create a software-protected key \r\n",
|
||||
"<br> az keyvault key create --vault-name \"SQLStandardKeyVault\" --name \"SQLTDEKey\" --protection software\r\n",
|
||||
"</code>"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "ef0ab374-182e-434b-b7d8-050b3a1c481b"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"CREATE ASYMMETRIC KEY dbAKV_Key \r\n",
|
||||
"FROM PROVIDER [AzureKeyVault_EKM_Prov] \r\n",
|
||||
"WITH PROVIDER_KEY_NAME = 'SQLTDEKey', -- This is the KEY that was created in the Azure Key Vault\r\n",
|
||||
"CREATION_DISPOSITION = OPEN_EXISTING;"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "923e5bc8-2751-4286-a0fe-fa37e1af59b0"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.145"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 27
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Now create a database and enable TDE on the database"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "c87fa2c2-1d3f-4dda-a2f9-d320af41486a"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE master; \r\n",
|
||||
"-- Create a SQL Server login associated with the asymmetric key \r\n",
|
||||
"-- for the Database engine to use when it loads a database \r\n",
|
||||
"-- encrypted by TDE. \r\n",
|
||||
"CREATE LOGIN TDE_Login \r\n",
|
||||
"FROM ASYMMETRIC KEY dbAKV_Key; \r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"-- Alter the TDE Login to add the credential for use by the \r\n",
|
||||
"-- Database Engine to access the key vault \r\n",
|
||||
"ALTER LOGIN TDE_Login \r\n",
|
||||
"ADD CREDENTIAL Azure_EKM_TDE_cred ; \r\n",
|
||||
"GO\r\n",
|
||||
"\r\n",
|
||||
"CREATE DATABASE dbAKVTDE;\r\n",
|
||||
"GO \r\n",
|
||||
"USE dbAKVTDE;\r\n",
|
||||
"GO\r\n",
|
||||
"\r\n",
|
||||
"CREATE DATABASE ENCRYPTION KEY \r\n",
|
||||
"WITH ALGORITHM = AES_256 \r\n",
|
||||
"ENCRYPTION BY SERVER ASYMMETRIC KEY dbAKV_Key; -- Use the key created above\r\n",
|
||||
"GO \r\n",
|
||||
"\r\n",
|
||||
"-- Alter the database to enable transparent data encryption. \r\n",
|
||||
"ALTER DATABASE dbAKVTDE \r\n",
|
||||
"SET ENCRYPTION ON; \r\n",
|
||||
"GO "
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "ee35f210-c356-4018-8994-057d16273b3a"
|
||||
},
|
||||
"outputs": [],
|
||||
"execution_count": 30
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Check if the database has been encrypted"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "6b8f4ad6-4998-4c0f-8fe5-f1a23d95055f"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE MASTER \r\n",
|
||||
"SELECT name,provider_type, algorithm_desc FROM sys.asymmetric_keys \r\n",
|
||||
"\r\n",
|
||||
"-- Check which databases are encrypted using TDE \r\n",
|
||||
"SELECT SERVERPROPERTY('Edition') as [Edition],d.name, dek.encryption_scan_state_desc, dek.encryptor_type \r\n",
|
||||
"FROM sys.dm_database_encryption_keys AS dek \r\n",
|
||||
"JOIN sys.databases AS d \r\n",
|
||||
" ON dek.database_id = d.database_id;"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "557fb1b5-7058-4888-986e-1ef2c03a1ab9"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "(1 row affected)"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "(3 rows affected)"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.048"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"metadata": {},
|
||||
"execution_count": 36,
|
||||
"data": {
|
||||
"application/vnd.dataresource+json": {
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "name"
|
||||
},
|
||||
{
|
||||
"name": "provider_type"
|
||||
},
|
||||
{
|
||||
"name": "algorithm_desc"
|
||||
}
|
||||
]
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"0": "dbAKV_Key",
|
||||
"1": "CRYPTOGRAPHIC PROVIDER",
|
||||
"2": "RSA_2048"
|
||||
}
|
||||
]
|
||||
},
|
||||
"text/html": "<table><tr><th>name</th><th>provider_type</th><th>algorithm_desc</th></tr><tr><td>dbAKV_Key</td><td>CRYPTOGRAPHIC PROVIDER</td><td>RSA_2048</td></tr></table>"
|
||||
}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"metadata": {},
|
||||
"execution_count": 36,
|
||||
"data": {
|
||||
"application/vnd.dataresource+json": {
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "Edition"
|
||||
},
|
||||
{
|
||||
"name": "name"
|
||||
},
|
||||
{
|
||||
"name": "encryption_scan_state_desc"
|
||||
},
|
||||
{
|
||||
"name": "encryptor_type"
|
||||
}
|
||||
]
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"0": "Standard Edition (64-bit)",
|
||||
"1": "tempdb",
|
||||
"2": "COMPLETE",
|
||||
"3": "ASYMMETRIC KEY"
|
||||
},
|
||||
{
|
||||
"0": "Standard Edition (64-bit)",
|
||||
"1": "dbTDE",
|
||||
"2": "COMPLETE",
|
||||
"3": "CERTIFICATE"
|
||||
},
|
||||
{
|
||||
"0": "Standard Edition (64-bit)",
|
||||
"1": "dbAKVTDE",
|
||||
"2": "COMPLETE",
|
||||
"3": "ASYMMETRIC KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"text/html": "<table><tr><th>Edition</th><th>name</th><th>encryption_scan_state_desc</th><th>encryptor_type</th></tr><tr><td>Standard Edition (64-bit)</td><td>tempdb</td><td>COMPLETE</td><td>ASYMMETRIC KEY</td></tr><tr><td>Standard Edition (64-bit)</td><td>dbTDE</td><td>COMPLETE</td><td>CERTIFICATE</td></tr><tr><td>Standard Edition (64-bit)</td><td>dbAKVTDE</td><td>COMPLETE</td><td>ASYMMETRIC KEY</td></tr></table>"
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 36
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# Best Practices\r\n",
|
||||
"To ensure quick key recovery and be able to access your data outside of Azure, we recommend the following best practices:\r\n",
|
||||
"<br> a. Create your encryption key locally on a local HSM device. (Make sure this is an asymmetric, RSA 2048 key so it's is supported by SQL Server.)\r\n",
|
||||
"<br> b. Import the encryption key to Azure Key Vault. See the steps in this [article](https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/setup-steps-for-extensible-key-management-using-the-azure-key-vault?view=sql-server-ver15#part-ii-create-a-key-vault-and-key) on how to do that.\r\n",
|
||||
"<br> c. Before using the key in Azure Key Vault for the first time, take an Azure Key Vault key backup. Learn more about the <b>Backup-AzureKeyVaultKey</b> command.\r\n",
|
||||
"Whenever any changes are made to the key (for example add ACLs, add tags, add key attributes), be sure to take another Azure Key Vault key backup.\r\n",
|
||||
"\r\n",
|
||||
"## Types of keys\r\n",
|
||||
"There are two types of keys you can generate in Azure Key Vault that will work with SQL Server. Both are asymmetric 2048-bit RSA keys.\r\n",
|
||||
"<br><b>Software-protected</b>: Processed in software and encrypted at rest. Operations on software-protected keys occur on Azure Virtual Machines. Recommended for keys not used in a production deployment.\r\n",
|
||||
"<br><b>HSM-protected</b>: Created and protected by a hardware security module (HSM) for additional security.\r\n",
|
||||
"\r\n",
|
||||
"For common troubleshooting and maintenance of the SQL Server Connector, please refer to [this article](https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/sql-server-connector-maintenance-troubleshooting?view=sql-server-ver15)."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "b74a3595-b2e9-4952-8e2a-c5876f740b70"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
+490
@@ -0,0 +1,490 @@
|
||||
{
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"name": "SQL",
|
||||
"display_name": "SQL",
|
||||
"language": "sql"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "sql",
|
||||
"version": ""
|
||||
}
|
||||
},
|
||||
"nbformat_minor": 2,
|
||||
"nbformat": 4,
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# Exploring Memory-Optmized TempDB Metadata\r\n",
|
||||
"\r\n",
|
||||
"TempDB metadata contention has historically been a bottleneck to scalability for many workloads running on SQL Server. SQL Server 2019 introduces a new feature that is part of the [In-Memory Database](https://docs.microsoft.com/sql/relational-databases/in-memory-database) feature family, memory-optimized tempdb metadata, which effectively removes this bottleneck and unlocks a new level of scalability for tempdb-heavy workloads. In SQL Server 2019, the system tables involved in managing temp table metadata can be moved into latch-free non-durable memory-optimized tables.\r\n",
|
||||
"\r\n",
|
||||
"To learn more about tempdb metadata contention, along with other types of tempdb contention, check out the blog article [TEMPDB - Files and Trace Flags and Updates, Oh My!](https://techcommunity.microsoft.com/t5/SQL-Server/TEMPDB-Files-and-Trace-Flags-and-Updates-Oh-My/ba-p/385937). Keep reading to explore the new memory-optimized tempdb metadata feature."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "0e65e08d-0c31-4214-9380-f13be28dd5e8"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Configure the AdventureWorks sample database\r\n",
|
||||
"Follow these steps to configure your environment in preparation for the demo. Alternatively, you can use the pre-configured container using the instructions in the Python companion notebook. Keep in mind that whether you use the container or your own server, you will need at least 4 cores to generate TempDB metadata contention.\r\n",
|
||||
"\r\n",
|
||||
"1. Ensure you have the latest version of SQL Server 2019 installed. You will need Evaluation, Enterprise, or Developer Edition in order to execute this demo.\r\n",
|
||||
"2. Download the [AdventureWorks2017.bak](https://github.com/Microsoft/sql-server-samples/releases/download/adventureworks/AdventureWorks2017.bak) sample database backup from GitHub. For your convenience, a copy of this backup has been included in the demo folder.\r\n",
|
||||
"3. Restore the database as `AdventureWorks`.\r\n",
|
||||
"> NOTE\r\n",
|
||||
"> <br> You will need to change the paths in the following example to match your server file paths."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "f3dc9c3f-a060-457a-afac-5befa4146462"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE [master]\r\n",
|
||||
"RESTORE DATABASE [AdventureWorks] FROM DISK = N'C:\\Program Files\\Microsoft SQL Server\\MSSQL15.SQL2019\\MSSQL\\Backup\\AdventureWorks2017.bak' \r\n",
|
||||
"WITH FILE = 1, \r\n",
|
||||
"\t\tMOVE N'AdventureWorks2017' TO N'C:\\Program Files\\Microsoft SQL Server\\MSSQL15.SQL2019\\MSSQL\\DATA\\AdventureWorks.mdf', \r\n",
|
||||
"\t\tMOVE N'AdventureWorks2017_log' TO N'C:\\Program Files\\Microsoft SQL Server\\MSSQL15.SQL2019\\MSSQL\\DATA\\AdventureWorks_log.ldf', NOUNLOAD\r\n",
|
||||
"GO"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "14617686-43fb-42b1-afe3-9e1ca9a2d4e3"
|
||||
},
|
||||
"outputs": [],
|
||||
"execution_count": 0
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"4. Run the 01b-Create_EmployeeBirthdayList.sql script to create the workload procedure."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "6913e3cb-c3e6-48ed-ba3d-4178a1bd69dc"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE AdventureWorks\r\n",
|
||||
"GO\r\n",
|
||||
"\r\n",
|
||||
"CREATE OR ALTER PROCEDURE usp_EmployeeBirthdayList @month int AS\r\n",
|
||||
"BEGIN\r\n",
|
||||
"\r\n",
|
||||
"\tIF OBJECT_ID('tempdb..#Birthdays') IS NOT NULL DROP TABLE #Birthdays;\r\n",
|
||||
"\r\n",
|
||||
"\tCREATE TABLE #Birthdays (BusinessEntityID int NOT NULL PRIMARY KEY);\r\n",
|
||||
"\r\n",
|
||||
"\tINSERT #Birthdays (BusinessEntityID)\r\n",
|
||||
"\tSELECT BusinessEntityID\r\n",
|
||||
"\tFROM HumanResources.Employee \r\n",
|
||||
"\tWHERE MONTH(BirthDate) = @month\r\n",
|
||||
"\r\n",
|
||||
"\tSELECT p.FirstName, p.LastName, a.AddressLine1, a.AddressLine2, a.City, sp.StateProvinceCode, a.PostalCode\r\n",
|
||||
"\tFROM #Birthdays b\r\n",
|
||||
"\tINNER JOIN Person.Person p ON b.BusinessEntityID = p.BusinessEntityID\r\n",
|
||||
"\tINNER JOIN Person.BusinessEntityAddress bea ON p.BusinessEntityID = bea.BusinessEntityID\r\n",
|
||||
"\tINNER JOIN Person.Address a ON bea.AddressID = a.AddressID\r\n",
|
||||
"\tINNER JOIN Person.StateProvince sp ON a.StateProvinceID = sp.StateProvinceID\r\n",
|
||||
"\tINNER JOIN Person.AddressType at ON at.AddressTypeID = bea.AddressTypeID\r\n",
|
||||
"\tWHERE at.Name = N'Home'\r\n",
|
||||
"\r\n",
|
||||
"END;"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "e1a478af-f345-43f8-a74a-c795715dfe40"
|
||||
},
|
||||
"outputs": [],
|
||||
"execution_count": 0
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Run the demo scenario to generate the workload\r\n",
|
||||
"\r\n",
|
||||
"Use the included ostress.exe tool to generate a multi-threaded load against your server. You will need a minimum of 4 cores to generate the contention. When configuring the ostress command, generally a 1:4 ratio of cores to concurrent threads works best to simulate the scenario and demonstrate the improvement. For example, this demo was tested with 4 cores and 16 concurrent threads. If you have 8 cores, start with 32 concurrent threads. You will also need to configure the number of iterations to allow the scenario to run long enough to observe the server. For 4 cores and 16 threads, 120 iterations should take around 30 seconds without Memory-Optimized TempDB Metadata enabled, 20 seconds with it enabled. Use the `-r` parameter to increase this number if you would like the script to run longer. Switch to the Python notebook to run the command, or you can open a Command Prompt window and execute the following command:\r\n",
|
||||
"\r\n",
|
||||
"`ostress.exe -Slocalhost,1455 -Usa -PP@ssw0rd! -dAdventureWorks -Q\"EXEC dbo.usp_EmployeeBirthdayList 4\" -mstress -quiet -n20 -r120 | FINDSTR \"QEXEC Starting Creating elapsed\"`"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "a500d78a-4e93-4795-8bd5-1b2d01b50968"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Monitor your workload for page contention\r\n",
|
||||
"\r\n",
|
||||
"You can use the following script to view all the sessions that are waiting for page-related wait types and get information about the objects that the pages belong to."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "ab3146e1-652f-41a2-83f3-21847f81ca2c"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"USE master\r\n",
|
||||
"GO\r\n",
|
||||
"\r\n",
|
||||
"SELECT \r\n",
|
||||
"er.session_id, er.wait_type, er.wait_resource, \r\n",
|
||||
"OBJECT_NAME(page_info.[object_id],page_info.database_id) as [object_name],\r\n",
|
||||
"er.blocking_session_id,er.command, \r\n",
|
||||
" SUBSTRING(st.text, (er.statement_start_offset/2)+1, \r\n",
|
||||
" ((CASE er.statement_end_offset \r\n",
|
||||
" WHEN -1 THEN DATALENGTH(st.text) \r\n",
|
||||
" ELSE er.statement_end_offset \r\n",
|
||||
" END - er.statement_start_offset)/2) + 1) AS statement_text,\r\n",
|
||||
"page_info.database_id,page_info.[file_id], page_info.page_id, page_info.[object_id], \r\n",
|
||||
"page_info.index_id, page_info.page_type_desc\r\n",
|
||||
"FROM sys.dm_exec_requests AS er\r\n",
|
||||
"CROSS APPLY sys.dm_exec_sql_text(er.sql_handle) AS st \r\n",
|
||||
"CROSS APPLY sys.fn_PageResCracker (er.page_resource) AS r \r\n",
|
||||
"CROSS APPLY sys.dm_db_page_info(r.[db_id], r.[file_id], r.page_id, 'DETAILED') AS page_info\r\n",
|
||||
"WHERE er.wait_type like '%page%'\r\n",
|
||||
"GO"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "526098c3-5404-4735-b112-b5e6e2d4906d"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "(3 rows affected)"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.267"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"metadata": {},
|
||||
"execution_count": 11,
|
||||
"data": {
|
||||
"application/vnd.dataresource+json": {
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "session_id"
|
||||
},
|
||||
{
|
||||
"name": "wait_type"
|
||||
},
|
||||
{
|
||||
"name": "wait_resource"
|
||||
},
|
||||
{
|
||||
"name": "object_name"
|
||||
},
|
||||
{
|
||||
"name": "blocking_session_id"
|
||||
},
|
||||
{
|
||||
"name": "command"
|
||||
},
|
||||
{
|
||||
"name": "statement_text"
|
||||
},
|
||||
{
|
||||
"name": "database_id"
|
||||
},
|
||||
{
|
||||
"name": "file_id"
|
||||
},
|
||||
{
|
||||
"name": "page_id"
|
||||
},
|
||||
{
|
||||
"name": "object_id"
|
||||
},
|
||||
{
|
||||
"name": "index_id"
|
||||
},
|
||||
{
|
||||
"name": "page_type_desc"
|
||||
}
|
||||
]
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"0": "70",
|
||||
"1": "PAGELATCH_EX",
|
||||
"2": "2:1:118",
|
||||
"3": "sysschobjs",
|
||||
"4": "71",
|
||||
"5": "EXECUTE",
|
||||
"6": "EXEC dbo.usp_EmployeeBirthdayList 4",
|
||||
"7": "2",
|
||||
"8": "1",
|
||||
"9": "118",
|
||||
"10": "34",
|
||||
"11": "2",
|
||||
"12": "INDEX_PAGE"
|
||||
},
|
||||
{
|
||||
"0": "78",
|
||||
"1": "PAGELATCH_EX",
|
||||
"2": "2:1:118",
|
||||
"3": "sysschobjs",
|
||||
"4": "71",
|
||||
"5": "EXECUTE",
|
||||
"6": "EXEC dbo.usp_EmployeeBirthdayList 4",
|
||||
"7": "2",
|
||||
"8": "1",
|
||||
"9": "118",
|
||||
"10": "34",
|
||||
"11": "2",
|
||||
"12": "INDEX_PAGE"
|
||||
},
|
||||
{
|
||||
"0": "79",
|
||||
"1": "PAGELATCH_EX",
|
||||
"2": "2:1:118",
|
||||
"3": "sysschobjs",
|
||||
"4": "72",
|
||||
"5": "CREATE TABLE",
|
||||
"6": "CREATE TABLE #Birthdays (BusinessEntityID int NOT NULL PRIMARY KEY)",
|
||||
"7": "2",
|
||||
"8": "1",
|
||||
"9": "118",
|
||||
"10": "34",
|
||||
"11": "2",
|
||||
"12": "INDEX_PAGE"
|
||||
}
|
||||
]
|
||||
},
|
||||
"text/html": "<table><tr><th>session_id</th><th>wait_type</th><th>wait_resource</th><th>object_name</th><th>blocking_session_id</th><th>command</th><th>statement_text</th><th>database_id</th><th>file_id</th><th>page_id</th><th>object_id</th><th>index_id</th><th>page_type_desc</th></tr><tr><td>70</td><td>PAGELATCH_EX</td><td>2:1:118</td><td>sysschobjs</td><td>71</td><td>EXECUTE</td><td>EXEC dbo.usp_EmployeeBirthdayList 4</td><td>2</td><td>1</td><td>118</td><td>34</td><td>2</td><td>INDEX_PAGE</td></tr><tr><td>78</td><td>PAGELATCH_EX</td><td>2:1:118</td><td>sysschobjs</td><td>71</td><td>EXECUTE</td><td>EXEC dbo.usp_EmployeeBirthdayList 4</td><td>2</td><td>1</td><td>118</td><td>34</td><td>2</td><td>INDEX_PAGE</td></tr><tr><td>79</td><td>PAGELATCH_EX</td><td>2:1:118</td><td>sysschobjs</td><td>72</td><td>CREATE TABLE</td><td>CREATE TABLE #Birthdays (BusinessEntityID int NOT NULL PRIMARY KEY)</td><td>2</td><td>1</td><td>118</td><td>34</td><td>2</td><td>INDEX_PAGE</td></tr></table>"
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 11
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Enable and Disable Memory-Optimized TempDB Metadata\r\n",
|
||||
"\r\n",
|
||||
"The following script will enable Memory-Optimized TempDB Metadata. **Note that this change requires a server restart to take effect**."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "ceda1d60-f135-4438-9587-8f8e27dea6ea"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"ALTER SERVER CONFIGURATION SET MEMORY_OPTIMIZED TEMPDB_METADATA=ON;\n",
|
||||
"GO"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "2d95de38-e28a-4642-be3c-c4668bcfa504"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.057"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 12
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Once the command is complete, restart the SQL Server service, then run the following command to verify that Memory-Optimized TempDB Metadata has been enabled."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "12ff967e-f2c3-4e60-880e-49bd4d06163e"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"SELECT SERVERPROPERTY('IsTempDBMetadataMemoryOptimized') AS IsTempDBMetadataMemoryOptimized; \r\n",
|
||||
"GO"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "e62081d9-fe1d-4a0a-a367-1f5892b5451f"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "(1 row affected)"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.101"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"metadata": {},
|
||||
"execution_count": 17,
|
||||
"data": {
|
||||
"application/vnd.dataresource+json": {
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "IsTempDBMetadataMemoryOptimized"
|
||||
}
|
||||
]
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"0": "0"
|
||||
}
|
||||
]
|
||||
},
|
||||
"text/html": "<table><tr><th>IsTempDBMetadataMemoryOptimized</th></tr><tr><td>0</td></tr></table>"
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 17
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"The system stored procedure `sp_configure` can also be used. If the `config_value` does not equal the `run_value`, it means that the configuration has been changed, but the server must be restarted in order for it to take effect."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "4d0f3cc3-665c-4e0f-b0b3-a5bdd043c509"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"EXEC sp_configure 'show advanced options', 1\r\n",
|
||||
"RECONFIGURE\r\n",
|
||||
"\r\n",
|
||||
"EXEC sp_configure 'tempdb metadata memory-optimized'"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "43bf54dc-86da-48a3-a581-955e228c7f6a"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Configuration option 'show advanced options' changed from 1 to 1. Run the RECONFIGURE statement to install."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.242"
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"metadata": {},
|
||||
"execution_count": 18,
|
||||
"data": {
|
||||
"application/vnd.dataresource+json": {
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "name"
|
||||
},
|
||||
{
|
||||
"name": "minimum"
|
||||
},
|
||||
{
|
||||
"name": "maximum"
|
||||
},
|
||||
{
|
||||
"name": "config_value"
|
||||
},
|
||||
{
|
||||
"name": "run_value"
|
||||
}
|
||||
]
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"0": "tempdb metadata memory-optimized",
|
||||
"1": "0",
|
||||
"2": "1",
|
||||
"3": "0",
|
||||
"4": "0"
|
||||
}
|
||||
]
|
||||
},
|
||||
"text/html": "<table><tr><th>name</th><th>minimum</th><th>maximum</th><th>config_value</th><th>run_value</th></tr><tr><td>tempdb metadata memory-optimized</td><td>0</td><td>1</td><td>0</td><td>0</td></tr></table>"
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 18
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Simlarly, you can use the following script to disable Memory-Optimized TempDB Metadata. This also requires a server restart."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "aeb7e67e-a436-42d6-9e4f-b6994dc1cfbe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"ALTER SERVER CONFIGURATION SET MEMORY_OPTIMIZED TEMPDB_METADATA=OFF;\r\n",
|
||||
"GO"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "5af4903e-61f9-4bb2-9342-3624894619dd"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Commands completed successfully."
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/html": "Total execution time: 00:00:00.037"
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"execution_count": 16
|
||||
}
|
||||
]
|
||||
}
|
||||
+263
@@ -0,0 +1,263 @@
|
||||
{
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"name": "python3",
|
||||
"display_name": "Python 3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.6.5",
|
||||
"mimetype": "text/x-python",
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"pygments_lexer": "ipython3",
|
||||
"nbconvert_exporter": "python",
|
||||
"file_extension": ".py"
|
||||
}
|
||||
},
|
||||
"nbformat_minor": 2,
|
||||
"nbformat": 4,
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# Exploring Memory-Optmized TempDB Metadata\r\n",
|
||||
"\r\n",
|
||||
"TempDB metadata contention has historically been a bottleneck to scalability for many workloads running on SQL Server. SQL Server 2019 introduces a new feature that is part of the [In-Memory Database](https://docs.microsoft.com/sql/relational-databases/in-memory-database) feature family, memory-optimized tempdb metadata, which effectively removes this bottleneck and unlocks a new level of scalability for tempdb-heavy workloads. In SQL Server 2019, the system tables involved in managing temp table metadata can be moved into latch-free non-durable memory-optimized tables.\r\n",
|
||||
"\r\n",
|
||||
"To learn more about tempdb metadata contention, along with other types of tempdb contention, check out the blog article [TEMPDB - Files and Trace Flags and Updates, Oh My!](https://techcommunity.microsoft.com/t5/SQL-Server/TEMPDB-Files-and-Trace-Flags-and-Updates-Oh-My/ba-p/385937). Keep reading to explore the new memory-optimized tempdb metadata feature."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "491417cb-b92b-43bc-b87b-7e67bcae5589"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Configure your environment\r\n",
|
||||
"\r\n",
|
||||
"Contention in tempdb happens when a large number of concurrent threads are attemping to create, modify or drop temp tables. In order to simulate this situation, you'll need to have a SQL Server instance that has multiple cores (4 or more is recommended), and a way to simulate multiple concurrent threads. For this example, we'll be using the ostress.exe tool to generate multiple concurrent threads. If you have an existing multi-core SQL Server instance, you can follow the T-SQL instructions to set up the demo, otherwise you can try the docker container steps instead. \r\n",
|
||||
"\r\n",
|
||||
"First, download the demo files to your local computer.\r\n",
|
||||
"\r\n",
|
||||
"### Docker Container Setup\r\n",
|
||||
"Note that the docker commands may take some time to execute, but you will not see progress here until they are complete.\r\n",
|
||||
"\r\n",
|
||||
"1. Make sure you have your docker environment configured, more information [here](https://docs.docker.com/get-started/). \r\n",
|
||||
"> NOTE\r\n",
|
||||
"> <br>If you are using Docker Desktop for Windows or Mac, the default configuration will limit your containers to 2 cores, regardless of the number of cores on your computer. Be sure to configure docker to allow at least 4 cores and 4GB of RAM for this demo to run properly. To do this, right click on the Docker Desktop icon in the status bar and choose Settings -> Advanced.\r\n",
|
||||
"2. Pull the demo container with the following command:"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "c740199e-107b-484a-9994-6883680db75e"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"! docker pull bluefooted/sql2019tempdbdemo"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "1c435674-3a29-43db-af14-3bbaeb69cba8"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"text": "Using default tag: latest\nlatest: Pulling from bluefooted/sql2019tempdbdemo\nDigest: sha256:035a1bda5539bfe68ad1b2f032a6e389cea91a0cd880e75b83ef186c46b2e34f\nStatus: Image is up to date for bluefooted/sql2019tempdbdemo:latest\ndocker.io/bluefooted/sql2019tempdbdemo:latest\n",
|
||||
"output_type": "stream"
|
||||
}
|
||||
],
|
||||
"execution_count": 3
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"3. Start the demo container with the following command:"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "94a1d4c5-806b-4823-bbcc-527a46dcac53"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"! docker run -e \"ACCEPT_EULA=Y\" -e \"SA_PASSWORD=P@ssw0rd!\" -p 1455:1433 --name sql2019tempdbdemo -d bluefooted/sql2019tempdbdemo"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "50d697be-9130-4bf8-8071-a670fce06a0c"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"text": "2a3dbfff94ce2bd277778e36ac268b4495afb77f1b6c5417478b10a9c545cca6\n",
|
||||
"output_type": "stream"
|
||||
}
|
||||
],
|
||||
"execution_count": 2
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"> NOTE:\r\n",
|
||||
"> <br> If you see the following error, you may already have run the docker run command with this image:\r\n",
|
||||
"<br> <br> *docker: Error response from daemon: Conflict. The container name \"/sql2019tempdbdemo\" is already in use by container \"3f662e0fd9b8cbdc1013e874722e066aa8e81ec3a07423fc3ab95cb75e640af9\". You have to remove (or rename) that container to be able to reuse that name.\r\n",
|
||||
"See 'docker run --help'.*\r\n",
|
||||
"\r\n",
|
||||
"If you see this message, you can start the container instead with the following command:"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "b713d8ad-99df-4a32-a65e-ab0779575f3b"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"! docker start sql2019tempdbdemo"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "e0d6ed1d-c6bc-4f08-a041-b6b70ea2054e"
|
||||
},
|
||||
"outputs": [],
|
||||
"execution_count": 0
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"4. Connect to the demo SQL Server instance using Azure Data Studio or SQL Server Management Studio using the following information:\r\n",
|
||||
" \r\n",
|
||||
" **Server Name**: localhost,1455<br>\r\n",
|
||||
" **Username**: sa<br>\r\n",
|
||||
" **Password**: P@ssw0rd!"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "895b3f6a-1ba1-4480-9b06-9cfb2b3c246d"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"### Existing SQL Server Instance Setup (skip if you are using the demo container)\r\n",
|
||||
"\r\n",
|
||||
"If you already have a SQL Server instance with a minimum of 4 cores, you can download and restore the AdventureWorks database and use the scripts in this repo to configure the database. Follow the steps in the T-SQL notebook to complete this setup.\r\n",
|
||||
""
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "0302cf7c-5ee1-426c-b1a5-17fb492a0a8a"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Detecting TempDB Metadata Contention\r\n",
|
||||
"\r\n",
|
||||
"The first thing to figure out before you turn on this new feature is whether or not you are experiencing TempDB metadata contention. The main symptom of this contention is a number of sessions in `Suspended` state with a wait type of `PAGELATCH_xx` and a wait resource of a page that hosts a TempDB system table, such as `2:1:118`. In order to know whether or not the page is part of a TempDB system table, you can use the new dynamic management function `sys.dm_db_page_info()` in SQL Server 2019 or later, or the older `DBCC PAGE` command in older versions. For this demo, let's focus on `sys.dm_db_page_info()`. Note that this command will take some time to complete, you'll want to proceed to the next step before it completes.\r\n",
|
||||
"\r\n",
|
||||
"First, start the workload using the `ostress.exe` tool that is included in the downloads. Note that if you are not using the demo container, you will need to change the server name and login information."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "7ac2df51-c609-45fc-9e97-996c7dc6b505"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"! ostress.exe -Slocalhost,1455 -Usa -PP@ssw0rd! -dAdventureWorks -Q\"EXEC dbo.usp_EmployeeBirthdayList 4\" -mstress -quiet -n16 -r120 | FINDSTR \"QEXEC Starting Creating elapsed\""
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "b738ca65-ec24-4762-a773-d37674b41884"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"text": "10/08/19 15:39:54.739 [0x00007A74] -QEXEC dbo.usp_EmployeeBirthdayList 4\n10/08/19 15:39:54.779 [0x00007A74] Starting query execution...\n10/08/19 15:39:54.783 [0x00007A74] Creating 16 thread(s) to process queries\n10/08/19 15:40:30.158 [0x00007A74] OSTRESS exiting normally, elapsed time: 00:00:35.419\n",
|
||||
"output_type": "stream"
|
||||
}
|
||||
],
|
||||
"execution_count": 11
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"While the above script is running, switch over to the T-SQL notebook and run the script to monitor your workload for page contention. You should see several sessions with a `wait_type` of `PAGELATCH_EX` or `PAGELATCH_SH`, often with an `object_name` of `sysschobjs`.\r\n",
|
||||
"\r\n",
|
||||
"> NOTE\r\n",
|
||||
"> <br>If this query does not return any results, make sure the command above is still running. If it is not running, start it and try the query again. If you still do not see any sessions waiting, you may need to increase the number of CPUs available to your server, and/or increase the number of concurrent threads by increasing the `-n` parameter in the command. This demo was tested with 4 cores and 16 concurrent sessions, which should yield the expected results. If you would like more time to examine the contention, you can increase the `-r` parameter, which will increase the number of iterations."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "a3421322-4a07-4348-88f0-2e870368291b"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Improve performance with Memory-Optimized TempDB Metadata\r\n",
|
||||
"\r\n",
|
||||
"Now that you have observed TempDB metadata contention, let's see how SQL Server 2019 addresses this contention. Switch over to the T-SQL notebook to review and run the script to enable Memory-Optimized TempDB Metadata.\r\n",
|
||||
"\r\n",
|
||||
"Once you have run the T-SQL command, you will need to restart the service. If you are using the demo container, you can do so with the following command:"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "aa10efd1-a1f1-4fbc-9e20-5ab94929d9ff"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"! docker restart sql2019tempdbdemo"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "65567cd7-bdda-4501-b04a-3e7c3579252c"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"text": "sql2019tempdbdemo\n",
|
||||
"output_type": "stream"
|
||||
}
|
||||
],
|
||||
"execution_count": 10
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Once the server is restarted, you can use queries in the T-SQL notebook to verify that the feature has been enabled.\r\n",
|
||||
"\r\n",
|
||||
"> NOTE\r\n",
|
||||
"> <br> It's a good idea to run a few T-SQL queries after the restart to make sure the server is up and running before you attempt the scenario again.\r\n",
|
||||
"\r\n",
|
||||
"Now that we have enabled Memory-Optimized TempDB Metadata, let's try running the workload again:"
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "9624ed6d-06b4-49b2-bb5e-5a0c2b6b457c"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"! ostress.exe -Slocalhost,1455 -Usa -PP@ssw0rd! -dAdventureWorks -Q\"EXEC dbo.usp_EmployeeBirthdayList 4\" -mstress -quiet -n16 -r120 | FINDSTR \"QEXEC Starting Creating elapsed\""
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "58c2f587-a9a5-4d6c-8cd3-93e555f08dee"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"text": "10/08/19 15:38:27.261 [0x00009AA0] -QEXEC dbo.usp_EmployeeBirthdayList 4\n10/08/19 15:38:27.316 [0x00009AA0] Starting query execution...\n10/08/19 15:38:27.321 [0x00009AA0] Creating 16 thread(s) to process queries\n10/08/19 15:38:54.045 [0x00009AA0] OSTRESS exiting normally, elapsed time: 00:00:26.784\n",
|
||||
"output_type": "stream"
|
||||
}
|
||||
],
|
||||
"execution_count": 9
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"While this is running, switch over to the T-SQL notebook to run the monitoring script again. This time, you should not see any sessions waiting on `PAGELATCH_EX` or `PAGELATCH_SH`. Also, the script should complete faster than before the change was made."
|
||||
],
|
||||
"metadata": {
|
||||
"azdata_cell_guid": "da79d28b-80d6-4644-98e5-23f250ec9c19"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user