mirror of
https://github.com/Microsoft/sql-server-samples.git
synced 2025-12-08 14:58:54 +00:00
Added aks sample script
This commit is contained in:
Mihaela Blendea
@@ -1,53 +1,12 @@
|
||||
# Creating a Kubernetes cluster for SQL Server 2019 big data cluster
|
||||
SQL Server 2019 big data cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes cluster using different methods.
|
||||
|
||||
# Creating a Kubernetes cluster for SQL Server 2019 big data cluster
|
||||
|
||||
SQL Server 2019 big data cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes clusters using different environments.
|
||||
|
||||
## Create a Kubernetes cluster using Kubeadm on Ubuntu 16.04 LTS or 18.04 LTS
|
||||
In this example, we will deploy Kubernetes over multiple Linux machines (physical or virtualized) using kubeadm utility. These instructions have been tested primarily with Ubuntu 16.04 LTS version. If you are using Ubuntu 18.04 LTS then install of docker.io may fail with message below depending on your configuration.
|
||||
|
||||
Package docker.io is not available, but is referred to by another package.
|
||||
This may mean that the package is missing, has been obsoleted, or
|
||||
is only available from another source
|
||||
|
||||
E: Package 'docker.io' has no installation candidate
|
||||
|
||||
To install docker, you can follow the steps below:
|
||||
Use the scripts in the **kubeadm** folder to deploy Kubernetes over multiple Linux machines (physical or virtualized) using `kubeadm` utility.
|
||||
|
||||
#!/usr/bin/env bash
|
||||
|
||||
sudo apt update
|
||||
|
||||
sudo apt --yes install \
|
||||
software-properties-common \
|
||||
apt-transport-https \
|
||||
ca-certificates \
|
||||
curl
|
||||
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
|
||||
|
||||
sudo add-apt-repository \
|
||||
"deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
|
||||
|
||||
sudo apt update
|
||||
|
||||
sudo apt --yes install docker-ce
|
||||
|
||||
sudo usermod --append --groups docker $USER
|
||||
## Deploy a SQL Server big data cluster on Azure Kubernetes Service (AKS)
|
||||
|
||||
|
||||
### Pre-requisites
|
||||
1. Multiple Linux machines or virtual machines. Recommended configuration is 8 CPUs, 32 GB memory each and at least 100 GB storage for each machine. Minimum number of machines required is three machines
|
||||
1. Designate one machine as the Kubernetes master
|
||||
1. Rest of the machines will be used as the Kubernetes agents
|
||||
|
||||
#### Useful resources
|
||||
[Deploy SQL Server 2019 big data cluster on Kubernetes](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
|
||||
|
||||
[Creating a cluster using kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
|
||||
[Troubleshooting kubeadm](https://kubernetes.io/docs/setup/independent/troubleshooting-kubeadm/)
|
||||
|
||||
#### Instructions
|
||||
1. Execute [kubeadm/ubuntu-setup-k8s-prereqs.sh](kubeadm/ubuntu-setup-k8s-prereqs.sh/) script on each machine
|
||||
1. Execute [kubeadm/ubuntu-setup-k8s-master.sh](kubeadm/ubuntu-setup-k8s-master.sh/) script on the machine designated as Kubernetes master
|
||||
1. After successful initialization of the Kubernetes master, follow the kubeadm join commands output by the script on each agent machine
|
||||
1. Now, you can deploy SQL Server 2019 big data cluster using instructions [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
|
||||
Using the sample Python script in **aks** folder, you will deploy a Kubernetes cluster in Azure using AKS and a SQL Server big data cluster using on top of it.
|
||||
@@ -0,0 +1,47 @@
|
||||
|
||||
# Deploy a SQL Server big data cluster on Azure Kubernetes Service (AKS)
|
||||
|
||||
Using this sample Python script, you will deploy a Kubernetes cluster in Azure using AKS and a SQL Server big data cluster using this AKS cluster as its environment. The script can be run from any client OS.
|
||||
|
||||
|
||||
## Pre-requisites
|
||||
|
||||
1. Running the script will require: [python minimum version 3.0](https://www.python.org/downloads)
|
||||
1. Ensure you have installed `mssqlctl` CLI and its prerequisites:
|
||||
- Install [pip3](https://pip.pypa.io/en/stable/installing/).
|
||||
- Install/update requests package. Run the command below using elevated priviledges (sudo or admin cmd window):
|
||||
```
|
||||
python -m pip install requests
|
||||
python -m pip install requests --upgrade
|
||||
```
|
||||
- Install mssqlctl CLI latest version using . Run the command below using elevated priviledges (sudo or admin cmd window):
|
||||
```
|
||||
pip3 install --index-url https://private-repo.microsoft.com/python/ctp-2.0 mssqlctl
|
||||
```
|
||||
1. Login into your Azure account. Run this command:
|
||||
```
|
||||
az login
|
||||
```
|
||||
|
||||
## Instructions
|
||||
|
||||
Run the script using:
|
||||
```
|
||||
python deploy-sql-big-data-aks.py
|
||||
```
|
||||
|
||||
>[!Note]
|
||||
>
|
||||
>If you have both pyhon3 and pyhon2 on your client machine and in the path, you will have to run the command using pyhon3:
|
||||
>```
|
||||
>python3 deploy-sql-big-data-aks.py
|
||||
>```
|
||||
|
||||
|
||||
When prompted, provide your input for Azure subscription ID, Azure resource group to create the resources in, and Docker credentials. Optionally, you can also provide your input for below configurations or use the defaults provided:
|
||||
- azure_region
|
||||
- vm_size - we recommend to use a VM size to accommodate your workload. Default `Standard_DS3_v2` is minimum for running just basic validations
|
||||
- aks_node_count - this is the number of the worker nodes for the AKS cluster, excluding master node
|
||||
- cluster_name - this value is used for both AKS cluster and SQL big data cluster created on top of AKS. Note that the name of the SQL big data cluster is going to be a Kubernetes namespace
|
||||
- password - same value is going to be used for all accounts that require user password input: SQL Server master instance SA account, controller user and Knox user
|
||||
- controller_username - this is the username for the cluster admin account
|
||||
@@ -0,0 +1,94 @@
|
||||
#
|
||||
# Prerequisites:
|
||||
#
|
||||
# Azure CLI (https://docs.microsoft.com/en-us/cli/azure/install-azure-cli), python3 (https://www.python.org/downloads), mssqlctl CLI (pip3 install --index-url https://private-repo.microsoft.com/python/ctp-2.0 mssqlctl )
|
||||
#
|
||||
# Run `az login` at least once BEFORE running this script
|
||||
#
|
||||
|
||||
from subprocess import check_output, CalledProcessError, STDOUT, Popen, PIPE
|
||||
import os
|
||||
|
||||
def executeCmd (cmd):
|
||||
if os.name=="nt":
|
||||
process = Popen(cmd.split(),stdin=PIPE, shell=True)
|
||||
else:
|
||||
process = Popen(cmd.split(),stdin=PIPE)
|
||||
stdout, stderr = process.communicate()
|
||||
if (stderr is not None):
|
||||
raise Exception(stderr)
|
||||
|
||||
#
|
||||
# MUST INPUT THESE VALUES!!!!!
|
||||
#
|
||||
SUBSCRIPTION_ID = input("Provide your Azure subscription ID:")
|
||||
GROUP_NAME = input("Provide Azure resource group name to be created:")
|
||||
DOCKER_USERNAME = input("Provide your Docker username:")
|
||||
DOCKER_PASSWORD = input("Provide your Docker password:")
|
||||
|
||||
#
|
||||
# Optionally change these configuration settings
|
||||
#
|
||||
AZURE_REGION=input("Provide Azure region - Press ENTER for using `westus`:") or "westus"
|
||||
VM_SIZE=input("Provide VM size for the AKS cluster - Press ENTER for using `Standard_DS3_v2`:") or "Standard_DS3_v2"
|
||||
AKS_NODE_COUNT=input("Provide number of worker nodes for AKS cluster - Press ENTER for using `2`:") or "2"
|
||||
#This is both Kubernetes cluster name and SQL Big Data cluster name
|
||||
CLUSTER_NAME=input("Provide name of AKS cluster and SQL big data cluster - Press ENTER for using `sqlbigdata`:") or "sqlbigdata"
|
||||
#This password will be use for Controller user, Knox user and SQL Server Master SA accounts
|
||||
PASSWORD=input("Provide password to be used for Controller user, Knox user and SQL Server Master SA accounts - Press ENTER for using `MySQLBigData2019`:") or "MySQLBigData2019"
|
||||
CONTROLLER_USERNAME=input("Provide username to be used for Controller user - Press ENTER for using `admin`:") or "admin"
|
||||
|
||||
#
|
||||
DOCKER_REGISTRY="private-repo.microsoft.com"
|
||||
DOCKER_REPOSITORY="mssql-private-preview"
|
||||
DOCKER_IMAGE_TAG="latest"
|
||||
|
||||
print ('Setting environment variables')
|
||||
os.environ['MSSQL_SA_PASSWORD'] = PASSWORD
|
||||
os.environ['CONTROLLER_USERNAME'] = CONTROLLER_USERNAME
|
||||
os.environ['CONTROLLER_PASSWORD'] = PASSWORD
|
||||
os.environ['KNOX_PASSWORD'] = PASSWORD
|
||||
os.environ['DOCKER_REGISTRY'] = DOCKER_REGISTRY
|
||||
os.environ['DOCKER_REPOSITORY'] = DOCKER_REPOSITORY
|
||||
os.environ['DOCKER_USERNAME']=DOCKER_USERNAME
|
||||
os.environ['DOCKER_PASSWORD']=DOCKER_PASSWORD
|
||||
os.environ['DOCKER_EMAIL']=DOCKER_USERNAME
|
||||
os.environ['DOCKER_IMAGE_TAG']=DOCKER_IMAGE_TAG
|
||||
os.environ['DOCKER_IMAGE_POLICY']="IfNotPresent"
|
||||
os.environ['DOCKER_PRIVATE_REGISTRY']="1"
|
||||
os.environ['CLUSTER_PLATFORM']="aks"
|
||||
os.environ['ACCEPT_EULA']="Y"
|
||||
|
||||
print ("Set azure context to subcription: "+SUBSCRIPTION_ID)
|
||||
command = "az account set -s "+ SUBSCRIPTION_ID
|
||||
executeCmd (command)
|
||||
|
||||
print ("Creating azure resource group: "+GROUP_NAME)
|
||||
command="az group create --name "+GROUP_NAME+" --location "+AZURE_REGION
|
||||
executeCmd (command)
|
||||
|
||||
print("Creating AKS cluster: "+CLUSTER_NAME)
|
||||
command = "az aks create --name "+CLUSTER_NAME+" --resource-group "+GROUP_NAME+" --generate-ssh-keys --node-vm-size "+VM_SIZE+" --node-count "+AKS_NODE_COUNT+" --kubernetes-version 1.10.7"
|
||||
executeCmd (command)
|
||||
|
||||
command = "az aks get-credentials --name "+CLUSTER_NAME+" --resource-group "+GROUP_NAME+" --admin"
|
||||
executeCmd (command)
|
||||
|
||||
print("Creating SQL Big Data cluster:" +CLUSTER_NAME)
|
||||
command="mssqlctl create cluster "+CLUSTER_NAME
|
||||
executeCmd (command)
|
||||
|
||||
print("")
|
||||
print("SQL Server big data cluster connection endpoints: ")
|
||||
print("SQL Server master instance:")
|
||||
command="kubectl get service service-master-pool-lb -o=custom-columns=""IP:.status.loadBalancer.ingress[0].ip,PORT:.spec.ports[0].port"" -n "+CLUSTER_NAME
|
||||
executeCmd(command)
|
||||
print("")
|
||||
print("HDFS/KNOX:")
|
||||
command="kubectl get service service-security-lb -o=custom-columns=""IP:status.loadBalancer.ingress[0].ip,PORT:.spec.ports[0].port"" -n "+CLUSTER_NAME
|
||||
executeCmd(command)
|
||||
print("")
|
||||
print("Cluster administration portal (https://<ip>:<port>):")
|
||||
command="kubectl get service service-proxy-lb -o=custom-columns=""IP:status.loadBalancer.ingress[0].ip,PORT:.spec.ports[0].port"" -n "+CLUSTER_NAME
|
||||
executeCmd(command)
|
||||
print("")
|
||||
@@ -0,0 +1,56 @@
|
||||
|
||||
|
||||
# Create a Kubernetes cluster using Kubeadm on Ubuntu 16.04 LTS or 18.04 LTS
|
||||
|
||||
In this example, we will deploy Kubernetes over multiple Linux machines (physical or virtualized) using kubeadm utility. These instructions have been tested primarily with Ubuntu 16.04 LTS version. If you are using Ubuntu 18.04 LTS then install of docker.io may fail with message below depending on your configuration.
|
||||
|
||||
Package docker.io is not available, but is referred to by another package.
|
||||
This may mean that the package is missing, has been obsoleted, or
|
||||
is only available from another source
|
||||
|
||||
E: Package 'docker.io' has no installation candidate
|
||||
|
||||
To install docker, you can follow the steps below:
|
||||
|
||||
#!/usr/bin/env bash
|
||||
|
||||
sudo apt update
|
||||
|
||||
sudo apt --yes install \
|
||||
software-properties-common \
|
||||
apt-transport-https \
|
||||
ca-certificates \
|
||||
curl
|
||||
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
|
||||
|
||||
sudo add-apt-repository \
|
||||
"deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
|
||||
|
||||
sudo apt update
|
||||
|
||||
sudo apt --yes install docker-ce
|
||||
|
||||
sudo usermod --append --groups docker $USER
|
||||
|
||||
|
||||
## Pre-requisites
|
||||
|
||||
1. Multiple Linux machines or virtual machines. Recommended configuration is 8 CPUs, 32 GB memory each and at least 100 GB storage for each machine. Minimum number of machines required is three machines
|
||||
1. Designate one machine as the Kubernetes master
|
||||
1. Rest of the machines will be used as the Kubernetes agents
|
||||
|
||||
### Useful resources
|
||||
|
||||
[Deploy SQL Server 2019 big data cluster on Kubernetes](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
|
||||
|
||||
[Creating a cluster using kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
|
||||
|
||||
[Troubleshooting kubeadm](https://kubernetes.io/docs/setup/independent/troubleshooting-kubeadm/)
|
||||
|
||||
### Instructions
|
||||
|
||||
1. Execute [kubeadm/ubuntu-setup-k8s-prereqs.sh](kubeadm/ubuntu-setup-k8s-prereqs.sh/) script on each machine
|
||||
1. Execute [kubeadm/ubuntu-setup-k8s-master.sh](kubeadm/ubuntu-setup-k8s-master.sh/) script on the machine designated as Kubernetes master
|
||||
1. After successful initialization of the Kubernetes master, follow the kubeadm join commands output by the script on each agent machine
|
||||
1. Now, you can deploy SQL Server 2019 big data cluster using instructions [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
|
||||
Reference in New Issue
Block a user