github/sql-server-samples
1
0
Fork 0
mirror of https://github.com/Microsoft/sql-server-samples.git synced 2025-12-08 14:58:54 +00:00
Code Issues Releases Wiki Activity

Merge pull request #3 from ananto-msft/tina-private-preview

Browse Source 
Editorial of the readme.md files.
This commit is contained in:
ananto-msft
2019-11-19 21:03:01 -08:00
committed by GitHub
parent acbfde6a2a 6914f8111c
commit 1cb2a4dd35
5 changed files with 14 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files
samples/features/azure-arc-data-controller/README.md
+2 -57
View File
@@ -1,59 +1,4 @@
# Azure Arc Data Controller clusters
Installation instructions for SQL Server 2019 big data clusters can be found [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sql-server-ver15).
# Azure Arc Data Controller cluster
## Samples Setup
**Before you begin**, load the sample data into your big data cluster. For instructions, see [Load sample data into a SQL Server 2019 big data cluster](https://docs.microsoft.com/en-us/sql/big-data-cluster/tutorial-load-sample-data).
## Executing the sample scripts
The scripts should be executed in a specific order to test the various features. Execute the scripts from each folder in below order:
1. __[spark/data-loading/transform-csv-files.ipynb](spark/data-loading/transform-csv-files.ipynb)__
1. __[data-virtualization/generic-odbc](data-virtualization/generic-odbc)__
1. __[data-virtualization/hadoop](data-virtualization/hadoop)__
1. __[data-virtualization/storage-pool](data-virtualization/storage-pool)__
1. __[data-virtualization/oracle](data-virtualization/oracle)__
1. __[data-pool](data-pool/)__
1. __[machine-learning/sql/r](machine-learning/sql/r)__
1. __[machine-learning/sql/python](machine-learning/sql/python)__
## __[data-pool](data-pool/)__
SQL Server 2019 big data cluster contains a data pool which consists of many SQL Server instances to store data & query in a scale-out manner.
### Data ingestion using Spark
The sample script [data-pool/data-ingestion-spark.sql](data-pool/data-ingestion-spark.sql) shows how to perform data ingestion from Spark into data pool table(s).
### Data ingestion using sql
The sample script [data-pool/data-ingestion-sql.sql](data-pool/data-ingestion-sql.sql) shows how to perform data ingestion from T-SQL into data pool table(s).
## __[data-virtualization](data-virtualization/)__
SQL Server 2019 or SQL Server 2019 big data cluster can use PolyBase external tables to connect to other data sources.
### External table over Generic ODBC data source
The [data-virtualization/generic-odbc](data-virtualization/generic-odbc) folder contains samples that demonstrate how to query data in MySQL & PostgreSQL using external tables and generic ODBC data source. The generic ODBC data soruce can be used only in SQL Server 2019 on Windows.
### External table over Hadoop
The [data-virtualization/hadoop](data-virtualization/hadoop) folder contains samples that demonstrate how to query data in HDFS using external tables. This demonstrates the functionality available from SQL Server 2016 using the HADOOP data source.
### External table over Oracle
The [data-virtualization/oracle](data-virtualization/oracle) folder contains samples that demonstrate how to query data in Oracle using external tables.
### External table over Storage Pool
SQL Server 2019 big data cluster contains a storage pool consisting of HDFS, Spark and SQL Server instances. The [data-virtualization/storage-pool](data-virtualization/storage-pool) folder contains samples that demonstrate how to query data in HDFS inside SQL Server 2019 big data cluster.
## __[deployment](deployment/)__
The [deployment](deployment) folder contains the scripts for deploying a Kubernetes cluster for SQL Server 2019 big data cluster.
## __[machine-learning](machine-learning/)__
SQL Server 2016 added support executing R scripts from T-SQL. SQL Server 2017 added support for executing Python scripts from T-SQL. SQL Server 2019 adds support for executing Java code from T-SQL. SQL Server 2019 big data cluster adds support for executing Spark code inside the big data cluster.
### SQL Server Machine Learning Services
The [machine-learning\sql](machine-learning\sql) folder contains the sample SQL scripts that show how to invoke R, Python, and Java code from T-SQL.
### Spark Machine Learning
The [machine-learning\spark](machine-learning\spark) folder contains the Spark samples.
Follow the instrutions here: https://raw.githubusercontent.com/ananto-msft/sql-server-samples/master/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/README.md
samples/features/azure-arc-data-controller/deployment/README.md
+3 -11
View File
@@ -1,16 +1,8 @@
# Creating a Kubernetes cluster for SQL Server 2019 big data cluster
# Creating a Kubernetes cluster for Azure Arc Data Controller cluster
SQL Server 2019 big data cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes clusters using different environments.
Azure Arc Data Controller cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes clusters using different environments.
## __[Deploy a Kubernetes cluster using kubeadm](kubeadm/)__
Use the scripts in the **kubeadm** folder to deploy a Kubernetes cluster over one or more Linux machines (physical or virtualized) using `kubeadm` utility.
## __[Deploy a SQL Server big data cluster on Azure Kubernetes Service (AKS)](aks/)__
Using the sample Python script in **aks** folder, you will deploy a Kubernetes cluster in Azure using AKS and a SQL Server big data cluster using on top of it.
## __[Push SQL Server big data cluster images to your own private Docker repository](offline/)__
Using the sample Python script in **offline** folder, you will push the necessary images required for the deployment to your own repository.
Use the scripts in the **kubeadm** folder to deploy a Kubernetes cluster over one or more Linux machines (physical or virtualized) using `kubeadm` utility.
samples/features/azure-arc-data-controller/deployment/kubeadm/README.md
+1 -5
View File
@@ -7,8 +7,4 @@ This folder contains scripts that provide a template for deploying a Kubernetes
## __[ubuntu-single-node-vm](ubuntu-single-node-vm/)__
This folder contains a sample script that can be used to create a single-node Kubernetes cluster on a Linux machine and deploy SQL Server big data cluster.
## __[ubuntu-single-node-vm-ad](ubuntu-single-node-vm-ad/)__
This folder contains a sample script that can be used to create a single-node Kubernetes cluster on a Linux machine and deploy SQL Server big data cluster with Active Directory integration.
This folder contains a sample script that can be used to create a single-node Kubernetes cluster on a Linux machine and deploy Azure Arc Data Controller cluster.
samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/README.md
+8 -14
View File
@@ -1,7 +1,7 @@
# Deploy a SQL Server big data cluster on single node Kubernetes cluster (kubeadm)
# Deploy a Azure Arc Data Controller on single node Kubernetes cluster (kubeadm)
Using this sample bash script, you will deploy a single node Kubernetes cluster using kubeadm and a SQL Server big data cluster on top of it. The script must be run from the VM you are planning to use for your kubeadm deployment.
Using this sample bash script, you will deploy a single node Kubernetes cluster using kubeadm and a Azure Arc Data Controller on top of it. The script must be run from the VM you are planning to use for your kubeadm deployment.
## Pre-requisites
@@ -20,34 +20,28 @@ sudo systemctl reboot
1. Use checkpoint or snapshot capability in your hyper visor so that you can rollback the virtual machine to a clean state.
## Instructions to deploy SQL Server big data cluster
## Instructions to deploy Azure Arc Data Controller
1. Download the script on the VM you are planning to use for the deployment
``` bash
curl --output setup-bdc.sh https://raw.githubusercontent.com/microsoft/sql-server-samples/master/samples/features/sql-big-data-cluster/deployment/kubeadm/ubuntu-single-node-vm/setup-bdc.sh
curl --output setup-controller.sh https://raw.githubusercontent.com/ananto-msft/sql-server-samples/master/samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu-single-node-vm/setup-controller.sh
```
2. Make the script executable
``` bash
chmod +x setup-bdc.sh
chmod +x setup-controller.sh
```
3. Run the script (make sure you are running with sudo)
``` bash
sudo ./setup-bdc.sh
sudo ./setup-controller.sh
```
4. Refresh alias setup for azdata
``` bash
source ~/.bashrc
```
When prompted, provide your input for the password that will be used for all external endpoints: controller, SQL Server master and gateway. The password should be sufficiently complex based on existing rules for SQL Server password. The controller username is defaulted to *admin*.
When prompted, provide your input for the password that will be used for all external endpoints: controller, SQL Server master and gateway. The password should be sufficiently complex based on existing rules for SQL Server password. The controller username is defaulted to *controlleradmin*.
## Cleanup
1. The [cleanup-bdc.sh](cleanup-bdc.sh/) script is provided as convenience to reset the environment in case of errors. However, we recommend that you use a virtual machine for testing purposes and use the snapshot capability in your hyper-visor to rollback the virtual machine to a clean state.
1. The [cleanup-controller.sh](cleanup-controller.sh/) script is provided as convenience to reset the environment in case of errors. However, we recommend that you use a virtual machine for testing purposes and use the snapshot capability in your hyper-visor to rollback the virtual machine to a clean state.
samples/features/azure-arc-data-controller/deployment/kubeadm/ubuntu/README.md
-6
View File
@@ -12,8 +12,6 @@ In this example, we will deploy Kubernetes over multiple Linux machines (physica
### Useful resources
[Deploy SQL Server 2019 big data cluster on Kubernetes](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
[Creating a cluster using kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
[Troubleshooting kubeadm](https://kubernetes.io/docs/setup/independent/troubleshooting-kubeadm/)
@@ -25,7 +23,3 @@ In this example, we will deploy Kubernetes over multiple Linux machines (physica
1. After successful initialization of the Kubernetes master, follow the kubeadm join commands output by the setup script on each agent machine
1. Execute [setup-volumes-agent.sh](setup-volumes-agent.sh/) script on each agent machine to create volumes for local storage
1. Execute ***kubectl apply -f local-storage-provisioner.yaml*** against the Kubernetes cluster to create the local storage provisioner. This will create a Storage Class named "local-storage".
1. Now, you can deploy the SQL Server 2019 big data cluster following instructions [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions).
Simply type in "local-storage" twice (once for data, once for logs) when facing the following prompt by azdata :
`Kubernetes Storage Class - Config Path: spec.storage.data.className - Description: This indicates the name of the Kubernetes Storage Class to use. You must pre-provision the storage class and the persistent volumes or you can use a built in storage class if the platform you are deploying provides this capability. - Please provide a value:`