I don’t recall how I came across this Kubernetes IDE called Lens, but all I know is it’s cool as hec! It connects to a Kubernetes cluster (using the kube config file) and gives you an in depth view of all the different Kubernetes objects, their associated yaml files, health/metrics, etc. In this blog post I will show you how we can look into a Big Data Cluster’s Kubernetes infrastructure using Lens.
I remember when I first started deploying Big Data Clusters, they were on Azure Kubernetes Service utilizing the $200 credit for first time sign ups. By the time I got around to figuring out how to deploy the BDC, not only was my $200 credit gone, but I started to incur cost out of pocket.
If only there was a feature that would allow me to stop the VMs in AKS whenever I wasn’t using them. Well, I’m excited to share that Microsoft AKS (Azure Kubernetes Service) came out with a neat feature (currently in preview at the time of the publishing of this post) that allows you to stop and start your AKS cluster by running a simple command. Of course I had to try it out on BDCs and to my surprise it worked. Well, sort of. Let me explain…
Are you a data professional and curious about Kubernetes but not quite sure what type of opportunities are available? Maybe you’re hesitant because you think Kubernetes is a “fad”? Or perhaps you’re just starting out in IT and don’t know what path to take?
Whether you are new to IT, or a seasoned IT professional, pondering these questions can be exhausting. In this blog post I will go over the importance of learning Kubernetes and how it can massively level up your career!
About 2 years ago I started coming across a lot of online chatter on “containers” and “Kubernetes”. This was back in 2018 and around that time I had no interest of learning about it because it had no direct connection to SQL Server and my daily job duties as a DBA. Up until that point I had been working with SQL Server for about ten years. So like most people, “containers and Kubernetes” went in one ear and out the other.
That all changed with the hype, and eventual release, of SQL Server 2019. In SQL Server 2019 comes a feature called “Big Data Clusters”. This new feature in SQL Server really intrigued me because it was something completely different. I started to hear those terms again (containers and Kubernetes) because those are technologies behind Big Data Clusters. Over the next year, I heavily blogged, spoke, and created video content on Big Data Clusters. As a result of my deep passion and promotion of the product, I was awarded Microsoft MVP. My journey didn’t stop there as I have a natural “thirst for knowledge” and had to learn more about the underlying technology that makes Big Data Clusters feasible in the first place: Kubernetes.
So I started to study for the Certified Kubernetes Administrator (CKA) exam.
In my previous post, I talked about deploying a Big Data Cluster on a single node Kubernetes cluster. That’s cool and all but what if you’re a business or organization that cannot have your data on the cloud for whatever reason? Is there a way to deploy a Big Data Cluster on-premise? Absolutely! I’ll walk you through setting that up this blog post. I will walk you through deploying a 3-node Kubernetes cluster, then deploying a Big Data Cluster on top of that.
This is part 4 of the “BDC series.” You can read part 1 here, part 2 here, and part 3 here. This blog post will go into the available monitoring tools available to monitor the health of your Big Data Cluster. If you’d like to stay updated, without doing the heavy work, feel free to register for my newsletter. I will email out blog posts of my journey down the wonderful road of BDCs.
[Updated for CTP 3.2] – There are kubectl commands and azdata commands to check the health of your cluster but I will focus on the Kubernetes Dashboard for this series. I will blog about some of the useful kubectl and mssqlctl commands in later posts.
This is part 3 of the “BDC series.” You can read part 1 here and part 2 here. This blog post will go into creating the Big Data Cluster on top of the Azure Kubernetes Service (AKS) cluster we created in Part 2. If you’d like to stay updated, without doing the heavy work, feel free to register for my newsletter. I will email out blog posts of my journey down the wonderful road of BDCs.
Before I get started I want to say that there are many ways to deploy a Big Data Cluster. There is a “Default configuration” way and a “Custom configuration” way. You can read more about the custom config way here. I will be posting blogs on the other ways to deploy a BDC but for the sake of this series I will be deploying the BDC via the default way. The BDC team at Microsoft is constantly revamping and tweaking the BDC deployment process in order to make it more streamline and easier.
This is part 2 of the “BDC series.” You can read part 1 here. This blog post will go into creating the Azure Kubernetes Service (AKS) cluster. If you’d like to stay updated, without doing the heavy work, feel free to register for my newsletter. I will email out blog posts of my journey down the wonderful road of BDCs.
If you’d like to stay updated, without doing the heavy work, feel free to register for my newsletter. I will email out blog posts of my journey down the wonderful road of BDCs.
So far, Microsoft does not have a simple way to create a Big Data Cluster. It’s a bit cumbersome of a process and the learning curve is a bit steep. However, Microsoft is currently working on making it easier to deploy a Big Data Cluster via Notebook in Azure Data Studio and eventually some type of “deployment wizard.” But for now, the only option is to do it the long way.
Installing and deploying a Kubernetes cluster on-prem can be a pain in the arse. Especially if you are new to Kubernetes. That’s where a cloud provider like Microsoft’s Azure comes in handy. Instead of having to go through the arduous task of installing, setting up, configuring and deploying Kubernetes clusters, you can just use Microsoft’s AKS, or Azure Kubernetes Service, to quickly deploy clusters. That way you can focus on your organization’s mission critical issues, rather than worring about ongoing operations and maintenance of your Kubernetes cluster.
In 2017, Microsoft introduced “SQL Server on Linux.” In 2019, you can configure Availability Groups to run on Kubernetes cluster. Another very interesting feature in SQL Serve 2019 is called Big Data Clusters (read the MS white paper here). The more I read about these new features the more I realize how *important* Kubernetes is becoming.
As a SQL Server professional, I find it extremely exciting when new features come out. For example, when Microsoft launched SQL Server 2017, you could install it on Linux. SQL Server 2019 supports availability groups on containers in a Kubernetes cluster. Also in SQL Server 2019, there is the new Big Data Clusters feature, and guess what it uses for container orchestration? You guessed it, Kubernetes.
The average SQL Server DBA might not have much experience with setting up HA/DR solution utilizing Availability Groups, let alone installing it on Linux or figuring out the ins and outs of containers and Kubernetes. But for those who like to push themselves by learning new things and securing their future, this blog post is a review of a book by my friend Nigel Poulton (b | t), titled, “The Kubernetes Book.” Continue reading “Book Review – “The Kubernetes Book” by Nigel Poulton”