This is part 4 of the “BDC series.” You can read part 1 here, part 2 here, and part 3 here. This blog post will go into the available monitoring tools available to monitor the health of your Big Data Cluster.
There are many ways to view the health of your Big Data Cluster. As of CTP 3.0, there are kubectl commands, mssqlctl commands as well as dashboards. For the sake of this series, I will focus on the dashboards. I will blog about some of the useful kubectl and mssqlctl commands in later posts.
This is part 3 of the “BDC series.” You can read part 1 here and part 2 here. This blog post will go into creating the Big Data Cluster on top of the Azure Kubernetes Service (AKS) cluster we created in Part 2.
Before I get started I want to say that there are many ways to deploy a Big Data Cluster. There is a “Default configuration” way and a “Custom configuration” way. You can read more about the custom config way here. I will be posting blogs on the other ways to deploy a BDC but for the sake of this series I will be deploying the BDC via the default way. The BDC team at Microsoft is constantly revamping and tweaking the BDC deployment process in order to make it more streamline and easier.
As of the date of this post, Big Data Clusters is only available as a limited public preview through the SQL Server 2019 Early Adoption Program. You need to sign up here to get access.
So far, Microsoft does not have a simple way to create a Big Data Cluster. It’s a bit cumbersome of a process and the learning curve is a bit steep. However, Microsoft is currently working on making it easier to deploy a Big Data Cluster via Notebook in Azure Data Studio and eventually some type of “deployment wizard.” But for now, the only option is to do it the long way.
Recently I was in the process of creating a AKS cluster and encountered an error:
Operation failed with status: ‘Bad Request’. Details: Provisioning of resource(s) for container service myfirstcluster in resource group sqlresourcegroup failed. Message: Operation results in exceeding quota limits of standardDSv2Family Cores. Maximum allowed: 20, Current in use: 0, Additional requested: 32. Please read more about quota increase at https://aka.ms/ProdportalCRP/?#create/Microsoft.Support/Parameters/
As of the date of this post, Big Data Clusters is only available as a limited public preview through the SQL Server 2019 Early Adoption Program. You need to sign up here to get access. Also, once you get access (which could take a few weeks), there is a great workshop created by the wonderful folks at Microsoft (Buck Woody, Bob Ward, et al) which is located here. This is a massive workshop that will take days, if not weeks, to complete. The learning curve is steep for the average Windows DBA, but overall a great learning experience. I am about done with the workshop and will be posting my progress soon. If you’d like to stay updated, without doing the heavy work, feel free to register for my newsletter. I will email out blog posts of my journey down the wonderful road of BDCs.
When I think of SQL Server 2019, I think of BIG DATA CLUSTERS. I remember first hearing the term and immediately thinking, “eh, what does that have to do with a Windows SQL DBA?” But the more I thought about it, the more I fell in love with it. And here’s why:
First of all, the entire market is shifting more and more towards the cloud. Whether you personally believe it or not, or like it or not, it is happening. In 3-5 years from now, majority of jobs on the market will require some level of cloud knowledge (Azure, AWS, etc.)
Second, over the past few version of SQL Server (2016+) it’s apparent that Microsoft is pushing SQL Server down the path of becoming “OS agnostic.” By that I mean it will no longer matter what the underlying OS is. Windows? Linux? Who cares? SQL Server will run on it all!
It was an absolute honor to host this month’s TSQL Tuesday. I decided on doing the “Dear 20 year old self” as a way for us to reflect on life. It seemed like this topic hit home with a lot of people. I enjoyed reading each one of the posts.
If you don’t find your post in this Round Up, please email me your link and I will update this post!
Yesterday was my 41st birthday. Twenty years ago, I remember my best friend asking me, “Where do you see yourself when you’re 40?” My reply was something like, “I can’t see myself as a 40 year old.” For some weird reason my mind went blank at 40. It wasn’t like I thought I’d be dead by 40, but I remember thinking of 30 or 35, but not 40. Maybe because 40 was twice my age and just too “far into the future” to think about?! But in a “blink of an eye” here I am twenty-one years later. Funny enough, now I can see myself as an 80 year old. Weird.
Installing and deploying a Kubernetes cluster on-prem can be a pain in the arse. Especially if you are new to Kubernetes. That’s where a cloud provider like Microsoft’s Azure comes in handy. Instead of having to go through the arduous task of installing, setting up, configuring and deploying Kubernetes clusters, you can just use Microsoft’s AKS, or Azure Kubernetes Service, to quickly deploy clusters. That way you can focus on your organization’s mission critical issues, rather than worring about ongoing operations and maintenance of your Kubernetes cluster.
In 2017, Microsoft introduced “SQL Server on Linux.” In 2019, you can configure Availability Groups to run on Kubernetes cluster. Another very interesting feature in SQL Serve 2019 is called Big Data Clusters (read the MS white paper here). The more I read about these new features the more I realize how *important* Kubernetes is becoming.
As a SQL Server professional, I find it extremely exciting when new features come out. For example, when Microsoft launched SQL Server 2017, you could install it on Linux. SQL Server 2019 supports availability groups on containers in a Kubernetes cluster. Also in SQL Server 2019, there is the new Big Data Clusters feature, and guess what it uses for container orchestration? You guessed it, Kubernetes.
The average SQL Server DBA might not have much experience with setting up HA/DR solution utilizing Availability Groups, let alone installing it on Linux or figuring out the ins and outs of containers and Kubernetes. But for those who like to push themselves by learning new things and securing their future, this blog post is a review of a book by my friend Nigel Poulton (b | t), titled, “The Kubernetes Book.” Continue reading “Book Review – “The Kubernetes Book” by Nigel Poulton”
Containers, Containers, Containers. It’s one of those things that apparently has been around “forever” but I just recently started to hear about it. As a Windows SQL DBA, things like Containers seems absolutely foreign to me.
I decided to start a series of blogs where I interview key people in the SQL Server community. Instead of me asking technical questions, I plan on asking about their outlook on the future, books they read (non-fiction and/or technical), and their overall thoughts on where technology (mainly SQL Server) is headed. You can find more interviews here.
Mohammad: Do you think people who dismiss the cloud as a “fad” or just don’t take it serious enough to learn about it (i.e. Azure, AWS, etc), will be in a tough spot to find a job 5 years from now?
Melody: Will people who do not embrace the cloud be in a tough spot to find a job in 5 years. I don’t have a crystal ball, but, I would be surprised if they would be happy with their decision. 😊 Our industry is constantly changing and if we do not change with it we will be unhappy. That is not a cloud vs non cloud issue but just a general, we as professionals should try to continue to learn to ensure we can stay relevant.