DevOps with Kubernetes – Zero to Hero Guide

This is an exhaustive guide that should take you from zero to hero with Kubernetes.

Objectives

• Learn Devops as a practice
• Understand, deploy, and use Kubernetes
• Get started with containerization and run those containers on Kubernetes
• Deploy Kubernetes locally, on-prem, and on AWS
• Run stateless and stateful applications on Kubernetes
• Administer Kubernetes
• Package and deploy applications using Helm

Is this for me?

For starters, did you know Devops professionals are raking in the dough? On average in the US you can make between $100-147k/year! If you are looking to increase your income - these skills are easy to acquire.

Also if you work in the web development world and would like to take a few steps towards being a Devops engineer or learning how cloud computing works, this article should cover most of the main DevOps basics you'll come into contact with. In this article I will be teaching what I know about Kubernetes, Helm, and much more.

I'm writing this more for myself as notes to refer to later in life but if you are interested in enhancing your Devops level of knowledge, you've come to the right place. Even if you aren't in a Devops role, you could strongly benefit from the content and guides contained within.

I've structured the content to be a straightforward guide to go from zero Devops knowledge to being able to apply for most Devops positions in no time. I hope that it is easy to consume with demo's built in incase you are the type of learner that has to get their hands dirty.

I hope you enjoy the content as much as I did writing and learning it - it's been a journey and I hope this guide saves you some of the headaches and frustrations I've ran into in the past through trial by fire.

Table of contents

• Introduction to Kubernetes

• Kubernetes Demo

• Kubernetes Deeper Dive

• Advanced Topics

• Packaging

• Commands Quick Reference

• Index

Introduction to Kubernetes

What is Kubernetes?

Kubernetes is an open-source orchestration system for Docker containers originally created by Google.

What does it do though?

• Kubernetes let's you create and schedule containers on a cluster of machines
• Kubernetes will manage the state of these containers
  • Containers will automatically be regenerated upon termination
  • You can start a container on specific nodes or move a container from one node to another
• Instead of just running a few docker containers on one host manually, kubernetes is a platform that will manage all of your containers for you
• Kubernetes clusters can start with 1 node and scale up to thousands of nodes
• It will manage "long running" web applications and services
• You can run lots of containers on one machine and manage them
• Alternatives:
  • Docker Swarm
  • Mesos
• It is highly modular, open source, rich community, founded by Google.

Architectural Overview of how Kubernetes works

In the diagram above you can see kubectl is communicating with the REST interface after authentication.

Every time you send new resources or objects to the REST interface using kubectl, they are going to be saved/fetched in/from the "etcd" directory.

"etcd" stands for /etc directory where most global configuration files live across machines. It's basically just a distributes data store typically consisting of 3-5 nodes.

This REST API is called the "kube-api server" and if you were paying attention when we used kops to create resources on AWS, we used api.kubernetes..

We also on the left have a scheduler that communicates with the REST interface and acts as a queue to schedule pods that have not started yet and will become pluggable so you can use any scheduler you want.

There is also a Controller Manager (also know as the "rc" or "replication controller") - it consists of the node controller which discovers new nodes and then manages those nodes or you have the Replication Controller which is manage the replicas of pods. If you have those defined.

All of those components communicate with the REST API. for one purpose - to communicate with the kubelet

The kubelets are found on the nodes and takes instructions from the REST api to perform.

Whats a node?

If you'd like to skip ahead until we get to this topic, you can but I thought it was important to link so you had some context: Node Architecture. Long story short, a node contains pods and pods contain, well, containers. Don't trip too much, we haven't discussed most of these terms just yet.

What is Containerization?

Imagine a ship with containers on it. Every single container has everything it needs inside and it cannot affect other containers on the ship. This is kind of a lose analogy to how your code will run inside a container which can easily be transported from one environment to another and have everything it needs.

What is a container?

Let's take a look at what virtualization looks like versus containerization and how it looks on the public cloud. Let's look at some diagrams of how servers are generally assembled today.

Virtualization:

Containerization:

Public Cloud

Note: the main difference is the host machine is separated from the virtual machine via a "Hypervisor"

Where can you run Kubernetes?

• On-premise
• Public Cloud
• Hybrid: Public & Private

Devops Methodology to Know and Love

If you are going to be working in Devops, you'll need to understand and benefit from a deeper dive into some of the 12 factors methodology which any engineer or developer could benefit from.

The Twelve Factors for stateless applications

from the Heroku platform

1. Codebase - One codebase tracked in revision control, many deploys
2. Dependencies - Explicitly declare and isolate dependencies
3. Config - Store config in the environment
4. Backing services - Treat backing services as attached resources
5. Build, release, run - Strictly separate build and run stages
6. Processes - Execute the app as one or more stateless processes
7. Port binding - Export services via port binding
8. Concurrency - Scale out via the process model
9. Disposability - Maximize robustness with fast startup and graceful shutdown
10. Dev/prod parity - Keep development, staging, and production as similar as possible
11. Logs - Treat logs as event streams
12. Admin processes - Run admin/management tasks as one-off processes

If you want to do a deeper dive, I highly recommend reading the root document and dig into each of the factors more for a holistic view of Devops as a practice. You can find those here: The Twelve Factors.

As a Devops engineer, if you are good at what you do, you should be automating everything you do. Oftentimes you will hear people say "automate yourself out of a job" - in reality,that isn't likely considering how challenging it can be to learn and master combined with the complexities of your individual application.

Getting Accounts Setup Before We Begin

Before we get too much further and then you run into roadblocks, I recommend setting up a few accounts so you can get the most out of this course:

• Get a domain if you don't already have one at dot.tk
• Docker Hub OR Preferably Gitlab
• AWS Free tier account -- this can take up to 24 hours for verification, so do it early.
• [Cloudflare] is a free Nameserver management tool that is free to use for our purposes and boasts instant domain propagation.

Install K8s Locally

Kubernetes should really be able to run anywhere but there are more integrations for cloud providers like AWS, AZURE, or GCP for things like volumes, loadBalancers, and the like that you cannot run locally - in any case, if you are looking to learn but don't want to spend money on an AWS account - you can run most everything here locally using minkube.

I recommend downloading and installing Docker Desktop before starting.

Kubernetes is complicated and intended to be ran on linux clusters on the cloud. Getting it to run locally can be challenging, I'm providing instructions on how to install on Mac Big Sur, if you are on another OS, consider getting a mac 🤪 but in all seriousness, the process is very similar but you will need to leverage google for other OS.

Minikube (pronounced "Mini Cube") Intro

MiniKube is a cut down version of kubernetes designed to be ran locally using the linux kernel. You can download it and get started with installation here but I'll try to make it as easy as possible in a single document.

Installing Minikube

If you have followed any of my articles on software installation, you probably know I'm a big fan of installing it with Brew.

We are going to install minikube and kubectl (pronounced "Cube Cuddle" or "Cube control") also known as "Kubernetes CLI".

1. If you don't have brew installed, head over to brew.sh and follow it's instructions. Then in terminal, run the following command (source)

   brew install minikube && brew install kubernetes-cli

2. Verify it's installed by running:

   which minikube

3. Start your local cluster:

   minikube start

4. Interact with the cluster using kubectl

   kubectl get po -A

5. To create your first "Hello World K8S, run:

   kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.4

   Followed by:

   kubectl expose deployment hello-minikube --type=NodePort --port=8080 && minikube service hello-minikube --url

6. Run minikube service hello-minikube and it will open a browser to the URL provided in terminal.

7. Terminate the deployment by running minikube stop

Cluster Setup

So far we have ran kubernetes locally but if you want to run a production cluster, you will need different tools. Minikube is great for local deployments but not for real clusters

For that we will use Kops or Kubeadm which are tools used to spin up a production cluster (you don't need both).

KOPS stands for Kubernetes Operations - KOPS was the preferred solution for AWS but recently AWS EKS (Hosted Kubernetes) is available and the preferred option. Kubeadm is an alternative approach and supported by AWS. Also important note: KOPS only works on Mac/Linux.

• Allows you to do production grade Kubernetes installations, upgrades, and management and we will use this tool to start a kubernetes cluster on AWS.

Install KOPS Demo

Deploy to AWS using Kubernetes

KOPS allows you to manage AWS resources - spin up a new ec2, load balancer and more. We will use KOPS to create our Nodes which later we will use to create pods on individual nodes then a load balancer to send traffic to the appropriate node's pod that is running our application. If this seems confusing, that's understandable. The more you follow this guide, the less confusing it will be.

1. Install KOPS using brew: brew install kops (source)

2. Log into AWS account > services > IAM > Users and create a new user with programmatic access > permissions

3. Click "Next: Permissions" and select the tab "Attach existing policies directly" then select Administrator Access > next > next > create user.

4. Copy your Access and secret keys and store them someplace safe.

5. Log into AWS from command line with: aws configure and you will be prompted for the access and secret keys you just saved then ask you to set defaults if you like.

   🚨Don't forget to change the <SOMERANDOMSTRING> bit In the next step

6. You can do the following step in AWS CLI using aws s3 mb s3://kops-state-<SOMERANDOMSTRING> or from your web console like this:

   • In the AWS console go to "services" > "S3" (S3 is storage like a google cloud drive if you aren't at all familiar)
   • Create a new bucket and name it Kops-state-<SOMERANDOMSTRING>
     

     🚨 You can delete the bucket using aws s3 rb s3://kops-state-<SOMERANDOMSTRING>

7. Create a new domain at dot.tk If you haven't already.

   🚨 This step is not free-tier eligible and may cost $0.50/month but allows you to stay in the AWS ecosystem.

8. In the AWS console go to "services" > route 53.

   Route53 is amazon's nameserver service that allows you to manage DNS records and route browser traffic to a webserver.

9. Create a new hosted zone.

10. Add your domain and complete the creation of the new hosted zone.

11. Copy the nameservers.

12. Log into your domain registrar and create new nameserver records for your domain that match the route53 dashboard for your hosted zone.

    For this step I logged into cloudflare and added NS (Nameserver) records to a subdomain Which will be used as a placeholder thoughout this demo.

13. Check to make sure your domain's nameservers are pointed to AWS appropriately by going here: https://www.whatsmydns.net/?utm_source=whatsmydns.com&utm_medium=redirect#NS/ (obviously your domain name will be different)

14. Let's set up our Kubernetes cluster on our AWS account. In this step we are finally going to use KOPS. It's going to plan out a master and 2 nodes.

    🚨 don't forget to change "<kubernetes.codingwithdrew.com>" below and "<RANDOMSTRING>" in the commands below where applicable.

    kops create cluster --name= --state=s3://kops-state- --zones=us-east-1a --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=

    🚨 You can delete those clusters and name space using:

    kops delete cluster --name= --state=s3://kops-state-  --yes

15. You will want to locate your SSH public key for example, it's typically located at:

    ~/.ssh/id_rsa.pub 

16. Make sure your ssh key has the proper permissions:

    chmod 400 ~/.ssh/id_rsa.pub

17. Now add that SSH key to your statefile so that new ec2 instances created will get access to it: kops create secret --name <kubernetes.codingwithdrew.com> --state=s3://kops-state-<RANDOMSTRING> sshpublickey admin -i ~/.ssh/id_rsa.pub

18. If you'd are satisfied with the launch details

19. Push your changes to AWS:

    kops update cluster --name <kubernetes.codingwithdrew.com> --state=s3://kops-state-<RANDOMSTRING> --yes --admin

20. Verify your cluster is running and all services are validated using this fancy command:

    kops validate cluster --wait 10m --state=s3://kops-state-

21. SSH into your EC2 using:

    ssh -i ~/.ssh/id_rsa ubuntu@api.<kubernetes.codingwithdrew.com>

    💡 Since we are using route53 we don't have to log into our AWS account and find the IP address, we can just use the default api.yourdomain.com that is created in order to SSH in. If you'd like to enable to do this, you can read more into external dns here.

22. You can get relevant information about the client-server status by using the following command: kubectl config view and you can update or set kubernetes context accordingly with the following command: kubectl config use-context <kubernetes.codingwithdrew.com> and if you run into troubles you can always run kubectl config --help for mor necessary information.

23. Let's see if the nodes are up, run: kubectl get node

    You should see something like this:

    NAME                            STATUS   ROLES    AGE   VERSION
    ip-172-20-57-164.ec2.internal   Ready    node     10m   v1.19.7
    ip-172-20-59-39.ec2.internal    Ready    node     10m   v1.19.7
    ip-172-20-61-190.ec2.internal   Ready    master   13m   v1.19.7

    What you are looking at is three running EC2 instances (1 master, 2 nodes).

    Now that we are done with the demo we can delete everything we have done so far but don't brain dump this process, we will be come back to it over and over. The way these commands are structured is the same across many CLI services such as helm cli, terraform, and oc-cli - so it's very beneficial to get used to.

24. Delete the cluster we created

    exit && kops delete cluster --name= --state=s3://kops-state-  --yes

What is Docker?

Docker is the most popular container software which leverages the Docker Runtime called "Docker Engine" to make and run Docker images.

There is also Docker Hub which is a public docker image repository where you can build, store, and fetch docker images online.

Benefits:

• It can run on any environment, unchanged
• It's shipped in isolation meaning it has all the dependencies bundled up. No more "it works on my machine, but not in production"
• Docker allows development teams to be able to ship code to production faster.
• Docker leverages linux containers for operating system-level isolation so everything is contained inside without access to the operating system.

Building Containers

To build a container we are going to use Docker Engine

• If you want you can go hard mode and Download it here

• Easy mode:

  brew install --cask docker && open /Applications/Docker.app

Dockerfile

Dockerizing a simple node.js application only needs a few files. Let's break that down but first let's add an extension to our vscode for dockerfile support

• If you already have a Dev directory on your computer, skip this step:

  • mkdir -p ~/Dev/k8s && cd ~/Dev/k8s && code .

  • Add necessary permissions:

    sudo groupadd docker && sudo usermod -aG docker $USER

  • You may need to log out an log back in for this to take effect, possibly even reboot.

  • Verify you can run Docker:

    docker run hello-world

Create A Dockerfile

• Create a Dockerfile using touch Dockerfile.

• To Run node, we need to include in that file some boilerplate (ignore the #| bit):

  1| FROM node:15.9.0
  2| WORKDIR /app
  3| ADD . /app
  4| RUN npm install
  5| EXPOSE 3000
  6| CMD npm start

• Line 1. We are saying our image will be based on this official image

• Line 2. We are going to create a working directory on our linux server called /app

• Line 3. The files in this current directory are going to be ./app. We pass 2 arguments, the files you want to add and where you want to add them. In this case . or "all files in our project" at /app
  -line 4. We are going to have the image run npm install which will install node and all it's dependencies for the package.

• Line 5. We are going to open up port 3000 or "expose" the port for incoming connections.

• Line 6. Lastly we are going to run a command from our package.json file npm start which will run our application.

Setup your Javascript Bits

• In our k8s directory we need to create an package.json file. This file tells line 4 in the command above what packages are necessary to install. The easiest way is to run: npm init -y

• Now install express using npm i express (npm i is short for npm install).

• Let's add a line to our package.json and delete the package-lock.json file. with rm package-lock.json then navigate to package.json in visual studio code and add a line after your scripts

  "engines": {
      "node": "^15.9.0"
  },

• Now edit your scripts:

  "scripts": {
      "start": "node index.js"
  },

We will also need to create an index.js file - this tutorial isn't intended to cover javascript in detail so I'll glazing over it.

• Add index.js file using touch index.js.

• It's contents will read:

  const express = require('express');
  const app = express();
  const portNum = 3000
  const message='Hello World'
  
  app.get('/', function(req, res){
      res.send(message);
      }
  );
  
  let server = app.listen(portNum, function(){
      let port = server.address().port;
      console.log('Demo app is now listening on http://%s:%d', port);
  });

  💡 The %d here allows you to play a number and similarly, %s allows you to play a string from the arguments passed into console.log(function, stringOrDigit).

• Save all files

Building and Running your first Container Image

• To Build this project with docker using our docker file we can use docker build . (the . indicates "All files" in linux).

• You should see a final line without errors that reads somethign like: => => writing image sha256:e5fafeaa9d272b51494d587367ce77df2e6d4dd85 0.0s

Copy that sha256 value.

• We can run this container now using docker run -p 3000:3000 -it e5fafeaa9d272b51494d587367ce77df2e6d4dd85. The -p is the port that you expose on the server and you connect it with the local port 3000. The i flag allows it to be interactive so when you close the command the conainter will also stop running, you can omit this flag if you like. The t "tag" flag runs the image specified afterwards, in this case the sha256:e5fafeaa9d272b51494d587367ce77df2e6d4dd85.

• If you did everything correctly, you should see "Demo app is now listening on http://::3000."

• Now go to http://localhost:3000 in your browser:

• The server should be running in your docker desktop application:

• Press ^+ c to close out of that terminal command and kill the running docker container. (remember that it only stops because we passed the "interactive" -i flag in the docker run command).

Deploy a Docker Image to a Container Registry

So far we have been a bit all over the place with mikube, kops, aws, docker desktop etc, but we aren't really in a place yet where we can fully appreciate how these pieces fit together. In this section we will cover how to set up your Dockerfile so it can be hosted online and consumed by kubernetes.

To be able to use docker with kubernetes it first has to be in a registry like Docker Hub but I like Gitlab so my instructions will be to use it.

You can build and deploy any application you want using docker and kubernetes if you take into account a few limitations:

• You should only run one process in one container - Don't try to create one giant docker image for your monolithic application, instead split it up as necessary
• All the data in the container is ephemeral (temporary and not preserved). All changes made to the container before it is terminated will be lost unless your preserve data using volumes (we will get there).

• If you are not already logged in, you need to authenticate to the Container Registry by using your GitLab username and password. If you have Two-Factor Authentication enabled, use a Personal Access Token instead of a password. Use: docker login registry.gitlab.com to login.

• To submit a package you build to the Gitlab container registry, you'll need to first build your project then push it to Gitlab with the following 2 commands (obviously yourname and project name will be different):

  $ docker build -t registry.gitlab.com/drewkarriker/drew-learns-Devops .
  
  $ docker push registry.gitlab.com/drewkarriker/drew-learns-Devops

Deploy the Demo Application to a New Kubernetes Cluster

Before we can launch a container based on the image we created, we first need to create a pod definition.

Wait, what's a pod?

• A pod describes an application running on Kubernetes.
• Pods live inside nodes & Nodes host many pods.
• A pod can contain one or more tightly coupled containers that make up your application allowing your application to communicate with other containers using their port numbers.

Ok, so now I know what a Pod is but how do I define it?

• Create a the pod definition file touch pod-hello-world.yaml

• Edit that file to read like this:

  apiVersion: v1
  kind: Pod
  metadata:
      name: drewlearnsk8s
      labels:
      app: helloworld
  spec:
      containers:
      - name: k8s-demo
      image: drewlearns/k8s-demo
      ports:
      - name: nodejs-port
          containerPort: 3000

This is probably a good time to direct you to the section at the end of this article Kubernetes Commands to learn more about the individual commands you will be using.

Getting familiar with Docker commands

The 9 main commands you should know are:

Here is a list of other Useful commands

Demo kubectl with Docker

• Turn on Docker Desktop
• Run minikube delete && minikube start && minikube status to kick off minikube locally.
• Make sure you are in the directory for your project that you created the pod definition in. Example: cd ~/Dev/k8s/
• Run kubectl get pod - you likely won't have one but perhaps still have the hello-world example pod running. In any case, you should see a table in terminal that looks something like this:

• Run kubectl create -f pod-hello-world.yaml to kick it off and then we can run kubectl get pod again to see it updated on the table above.

• Now we can have kubernetes describe our pod using kubectl describe pod drewlearnsk8s

• We should see output like below. Take some time to review the important and useful data contained such as events, volumes, conditions, and containers...

      Name:         drewlearnsk8s
      Namespace:    default
      Priority:     0
      Node:         minikube/192.168.49.2
      Start Time:   Sat, 20 Feb 2021 20:50:26 -0500
      Labels:       app=helloworld
      Annotations:  
      Status:       Running
      IP:           172.17.0.4
      IPs:
      IP:  172.17.0.4
      Containers:
      k8s-demo:
          Container ID:   docker://048fe16d604d2e140df532e27374ff514da9d6c8f4c5542904a601df0f5e9ab4
          Image:          drewlearns/k8s-demo
          Image ID:       docker-pullable://drewlearns/k8s-demo
          Port:           3000/TCP
          Host Port:      0/TCP
          State:          Running
          Started:      Sat, 20 Feb 2021 20:50:28 -0500
          Ready:          True
          Restart Count:  0
          Environment:    
          Mounts:
          /var/run/secrets/kubernetes.io/serviceaccount from default-token-7mjkg (ro)
      Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
      Volumes:
      default-token-7mjkg:
          Type:        Secret (a volume populated by a Secret)
          SecretName:  default-token-7mjkg
          Optional:    false
      QoS Class:       BestEffort
      Node-Selectors:  
      Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                      node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
      Events:
      Type    Reason     Age   From               Message
      ----    ------     ----  ----               -------
      Normal  Scheduled  22m   default-scheduler  Successfully assigned default/drewlearnsk8s to minikube
      Normal  Pulling    22m   kubelet            Pulling image "drewlearns/k8s-demo"
      Normal  Pulled     22m   kubelet            Successfully pulled image "drewlearns/k8s-demo" in 498.3679ms
      Normal  Created    22m   kubelet            Created container k8s-demo
      Normal  Started    22m   kubelet            Started container k8s-demo

Port forwarding

The easiest way to get to this pod is to "port forward"

Run the following command: kubectl port-forward drewlearnsk8s 8081:3000 meaning we can listen on https://localhost:8081

Alternatively, you can just use kubectl expose pod drewlearnsk8s --type=NodePort --name drewlearnsk8s-service

• If you are operating locally using minikube, you can use minikube service drewlearnsk8s-service to view what the address is (it will also automatically open up a browser to the address).

💡 You can also pass --url in that command to have it output the URL but it's already in a neat table for ya in terminal like you see below.

    |-------------|-------------------------|---------------|--------------------------|
    | NAMESPACE   | NAME                    | TARGET PORT   | URL                      |
    | ----------- | ----------------------- | ------------- | ------------------------ |
    | default     | drewlearnsk8s-service   |               | http://127.0.0.1:61091   |
    | ----------- | ----------------------- | ------------- | ------------------------ |
        http://127.0.0.1:61091

• If we run kubectl get service we get all of our services listed out in a slightly less neat table but useful none the less so that we can see the IP addresses for our clusters. Bear in mind, the URL for minikube is for local usage, where as the IP below are IPs within our cluster

  NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
  drewlearnsk8s-service   NodePort    10.108.45.137           3000:30681/TCP   9m18s
  hello-minikube          NodePort    10.101.69.104           8080:32351/TCP   84m
  kubernetes              ClusterIP   10.96.0.1               443/TCP          84m

• If you'd like to open this service in a shell and run commands - which could be especially useful for troubleshooting, run kubectl run -i --tty busybox --image=busybox --restart=Never -- sh which will open up a shell terminal pod within the cluster called "busybox" which is a light weight SSH client.

  • Bear in mind curl doesn't exist so you can use telnet instead. After that shell starts run: telnet 127.0.0.1 61091 then in the prompt after type GET / and you should get a "curl" like output.

Set Up an External AWS Load balancer

You will need to be able to access your application from outside of the cluster, currently you only see the 10.xx.xx.xx internal IPs which are not externally accessible. For that we will need an external loadbalancer. Loadbalancers will route traffic to the correct pod in kubernetes - it's very similar to a Firewall (I think it technically is a Firewall application).

There are other solutions you can employ for other cloud providers that don't have a load balancer solutions:

• haproxy
• Nginx
• Expose ports directly

In the first application we created a Pod Definition in our pod-hello-world.yaml file where we exposed port 3000.
We also created a service which we will need to create a Service Definition for.

ELB Demo Time

1. Create a new file touch hello-world-service.yaml then edit that file to look like this:

   apiVersion: v1
   kind: Service
   metadata:
   name: drewlearnsk8s-service
   spec: 
   ports:
   - port: 80
       targetPort: nodejs-port
       protocol: TCP
   selector:
       app: drewlearnsk8s
   type: LoadBalancer

2. You will need to use your Gitlab account and push your work - I'm not going into git too deeply in this article, but after you have committed and pushed your files to gitlab (You will need a personal access token to access it from your ec2 ssh).

3. Log into your AWS account if you aren't already using aws configure

4. You can skip this step if you didn't turn them off from the KOPS section.

   🚨 don't forget to change "<kubernetes.codingwithdrew.com>" below and "<RANDOMSTRING>" in the commands below where applicable.

   kops create cluster --name= --state=s3://kops-state- --zones=us-east-1a --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=

   1. Now we want to create a pod on AWS using kubectl create -f pod-hello-world.yaml then for the services kubectl create -f hello-world-service.yaml.
   2. Push that configuration to AWS using kops update cluster --name <kubernetes.codingwithdrew.com> --state=s3://kops-state-<RANDOMSTRING> --yes --admin
   3. ssh -i ~/.ssh/id_rsa ubuntu@api.<kubernetes.codingwithdrew.com>

5. Now that you are SSH'd into your AWS cluster, we can start running kubectl commands to push our pod and service.

6. Clone your repository, it should look something like this: git clone https://<GITLABUSERNAME>:<ACCESSTOKEN>@gitlab.com/DrewKarriker/drew-learns-Devops.git - this is how we are going to get our files onto our AWS server.

7. Spin up your pod on your cluster using kubectl like so: kubectl create -f <YOURGITLABPROJECTNAME>/pod-hello-world.yaml you should see an output like this: "pod/drewlearnsk8s created"

8. Now do the same thing for your service kubectl create -f <YOURGITLABPROJECTNAME>/hello-world-service.yaml, now it can take some time for that ELB to spin up (think like 30 minutes in some cases).

9. To get the status check: kubectl get service, You should see something like this:

   NAME	TYPE	CLUSTER-IP	EXTERNAL-IP	PORT(S)	AGE
   hello-world-service	LoadBalancer	100.64.226.45	a4af716f7ff49434bbf6eee487fef7bb-811765999.us-east-1.elb.amazonaws.com	80:30887/TCP	14m
   kubernetes	ClusterIP	100.64.0.1		443/TCP	59m

   If it's been longer than 30 minutes - It's likely that your ELB (Elastic Load balancer) won't spin up since you've likely never created one before. If that happens run kubectl delete service hello-world-service to delete your service then aws iam create-service-linked-role --aws-service-name "elasticloadbalancing.amazonaws.com" which will create the necessary roles alternatively you can create an ELB then delete it after the role is created automatically. Then try adding your service again.

   💡 You can pass "kubectl get" commands the -w "watch" command to update the terminal while you are watching it instead of having to run it continuously

10. As things are now, you haven't got a domain that works with the load balancer. You'll want to update your route53 by adding a new DNS A record - name it helloworld and set it as an alias and select the dualstack option in the drop down after you select your AZ.

11. After updating your DNS, you should be able to access the http:// version of your domain and see "hello world" print out on your terminal.

Kubernetes Introduction Summary

We created a container image locally then uploaded to Gitlab's container registry. We then used KOPS to create our AWS resources then used kubectl to deploy our pod and services (load balancer). We learned a ton of commands and now have a load balancer contacting our containers located on our pods within the three nodes.

Kubernetes Deeper Dive

Node Architecture

• The load balancer forwards traffic to our iptables on individual nodes
• kube-proxy works with iptables to feed it information about the nodes. Whenever a new node is created, kube-proxy will notify and update the iptables accordingly.
• Kublet is responsible for launching the pods. It connects to the master-node for the necessary information.
• Docker engine is responsible for the container image on the node.
• In green you can see there are pods, each one has a container, some with 1 some with more. You can have multiple containers in a single pod. those containers can easily communicate with each other within the same pod. Pods within a cluster across pods can communicate over the network with each other but use service discovery instead of port numbers like they would within the same pod.

Scaling Pods

If your application is stateless (ephemeral) then you can horizontally scale it. If there isn't a state, it doesn't write to any local files or keep local sessions. All traditional databases are stateful (persistent) - they have files that can't be split over multiple instances.

Currently, the app we created is stateless. We can horizontally scale or "add more resources". Scaling in kubernetes can be done using the "Replication Controller" which will ensure a specified number of pod replicas that you want to have running at all times. This helps with reliability because if a pod errors or gets terminated, it will automatically be replaced.

If there are only 4 running but your replication controller demands there is 5, a new one will spin up. This is very similar to autoscaling groups in AWS.

To enable a replication controller we just create a new yaml file to read like this:

    apiVersion: v1
    kind: ReplicationController
    metadata:
    name: drewlearnsk8s-controller
    labels:
        app: helloworld
    spec:
        replicas: 2
        selector:
            app: helloworld
        template:
            metadata:
            labels:
                app: helloworld
            spec:
                containers:
                - name: drewlearnsk8s
                image: drewlearns/k8s
                ports:
                - containerPort: 3000 

After running that file using kubectl create -f <filename> we should be able to run commands like kubectl scale --replicas=1 rc/drewlearnsk8s-controller and scale it down to 1 replica instead of 2 like our yaml file dictates.

Deployments in kubernetes focuses on replication sets. Replication Set is a next gen Replication Controller. It supports a new selector that can do selection based on filtering according to a set of rules which is great for automation of environments, for example you could set your replication based on "dev" or "qa" whereas the Replication Controller is only able to be set to evaluate this == that. The Deployment object uses the Replica set. Let's talk about deployments next!

Deployments

Deployments are how we want to get our application on our servers. We wouldn't want to use just the replication controller or replication set for deploying applications because it would get cumbersome.

So what is a deployment?

A deployment is a declaration in Kubernetes that allows you to do app deployments and updates. When using the deployment object, you define the state of your application and it will then make sure the clusters match your desired state.

With a deployment object you can:

• Create a deployment
• update a deployment
• Do rolling updates (zero downtime deployments)
• Roll back to a previous version
• Pause and resume deployments

This is an example of a deployment:

    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
        name: helloworld-deployment
    spec:
        replicas: 3
        template
            metadata:
                labels:
                    app: helloworld
            spec:
                containers:
                - name: drewlearnsk8s
                image: drewlearns/kubernets-demo
                ports:
                - name: nodejs-port
                  containerPort: 3000

Just like before we use the "create" command to deploy and this gets repetitive but you'll then view the pods to understand what happened.

• To deploy this, run kubectl create -f <Filepath>

• Run kubectl get deployments, and we should be able to see the number of replica sets for each deployment.

• We can use kubectl get pods --show-labels and see there are 3 pods that match with the deployments and also see what application we are running.

• We can also update our deployment and then roll it out like this:

  • We can then update the image that is deployed in our registry kubectl rollout status example/example-deployment
  • Expose the deployment using kubectl expose deployment example-deployment --type=NodePort, this created a service for us.
  • You can run kubectl describe service example=deployment to view the IP addresses.
  • We can then update our image using kubectl set image deployment/example-deployment example=example:2
  • Check your image should be updated and rolled out (deployed the update to new pods and terminate the old pods) using kubectl rollout status deployment/example-deployment
  • View the terminating pods and new pods using kubectl get pods -w

• We can also view the deployment history by running kubectl rollout history deployment/example-deployment

  💡 By default, it will only show you the most recent 2 deployments. you can edit that using kubectl edit deployment/example-deployment and change the limit by adding a line like below. It will open vim, remember to save and close you press :wq.

      ...
      spec:
          replicas:3
          revisionHistoryLimit: 100
      ...

• We can rollback a deployment very easily in kubectl as well using kubectl rollout undo deployment/example-deployment

Services

A service is a logical bridge between the temporary pod lifespan and other services or end-users.

When using a replication controller, you terminate and create pods continuously during scaling operations and also deployments, image version, or pod termination and new pods are created.

You could describe them as dynamic (meaning they may get terminated).

For this reason, pods should never be accessed directly but always through a service which is "intelligent".

When using kubectl expose - we create a service for our pod so it can be accessed externally.

Creating a service will create a new endpoint that your pods can connect to. There are three service endpoints to know:

1. ClusterIP - this is a virtual IP address that is only reachable from inside the cluster. External connections will be refused and this is the default endpoint that will be created.

2. NodePort - This is a single port open across each node that is externally accessible.

   💡 Ports can only run between ports 30000-32767 but you can change this default behavior by adding --service-node-port-range= argument to your kube-apiserver in your init scripts.

3. LoadBalancer - On AWS, this is called "Elastic Load Balancer" or "ELB" for short. It is a LoadBalancer created by a cloud provider will handle external traffic requests and send that traffic to every node on the NodePort.

   These options only allow you to create virtual IPs or ports and there is a possibility to use DNS names for external access using ExternalName for service auto discovery but requires a DNS add-on enabled.

4. ** Ingress and Ingress Controller

Demo - Create a service of type NodePort

1. Turn on minikube - minikube start and verify it's running with kubectl get node then use kubectl create -f <filename_for_our_hello_world application>
2. Let's get some more details about the deployment using kubectl describe pod <your pod name>

3. Let's create a new service using touch hello-world-nodeport-service.yaml then edit the contents of that file to look like this:

   apiVersion: v1
   kind: Service
   metadata:
       name: hello-world-service
   spec:
       ports:
       - port: 31001
       nodePort: 31001
       targetPort: nodejs-port
       protocol: TCP
   selector:
       app: helloworld
   type: NodePort

   💡 Note the ports are specified in this case, and it will assign them automatically if not specified. We also set the node port and then connect it to "nodejs-port" created on the helloworld pod definition we created earlier since this will be a service on that pod.

Labels

Labels are key/value pairs that objects can use and also match up with AWS and other cloud provider's architecture use to tag resources.

You can label your objects such as pods or nodes following your organization's naming conventions.

Labels are not unique and can be added to multiple resources, multiple labels can be added to a single object. Example:

After labels are attached to an object you can use filters to narrow down results - this is called label selectors when you can use to match expressions for labels.

What is the benefit?

This will allow you to run a specific pod only on a node labeled for "environment" == "development"

How do I label objects after I created it?

Easy-peasy: kubectl label <nodes> <node1> <key>=<value> for example: kubectl label nodes node2 environment=prod

How do make a resource only use a specific label?

The easiest way is to add them in your pod definitions under your spec in a nodeSelector like this:

    ...
    spec:
        containers:
        - name: some name
        image: example/name
        ports:
        - containerPort: 3000
        nodeSelector:
            environment: production

Heath checks (Liveliness Probe)

If your application malfunctions, the pod/container may still be running which is problematic for obvious reasons.

To detect and resolve problems with our pods on our kubernetes cluster we can run health checks by either running a command in the container periodically or using HTTP checks where you ping an endpoint.

If they indicate failures - the containers will be restarted.

Production applications behind a load balancer should always have health checks implemented to ensure availability and resiliency of your application - a core architecture design principle for AWS by the way.

If your containers always terminates when something goes wrong, then a liveliness probe is not necessary and you'd probably want to use a readiness probe.

How does a health check look in Yaml?

• Create a new file: touch example-healthcheck.yaml and then edit it.

• We can create a health check container under the "livenessProbe" definition:

  apiVersion: v1
  kind: Pod
  metadata:
  name: example-healthcheck.codingwithdrew.com
  labels:
      app: helloworld
  spec:
      containers:
      - name: drewlearnsk8s
          image:  drewlearnsk8s
          ports:
          - name: nodejs-port
          containerPort: 3000
          livenessProbe:
              httpGet:
                  path: /
                  port: 3000
              initialDelaySeconds: 15
              timeoutSeconds: 30

Cool, how do I use that health check though?

We have done it a bunch now, but run kubectl create -f example-healthcheck.yaml and it should run that service.

Readiness Probe

These are similar to the healthchecks we discussed in the previous section except instead of checking for failures after a pod is running, it's checking to see if a container is ready to serve requests.

This readiness probe will make sure that at startup, the pod will only receive traffic when the test is successful.

How does a readiness probe look in Yaml?

• Create a new file: touch example-readinessprobe.yaml and then edit it.

• We can create a readiness probe container under the "readinessProbe" definition:

  apiVersion: v1
  kind: Deployment
  metadata:
  name: example-readiness.codingwithdrew.com
  labels:
      app: helloworld
  spec:
      containers:
      - name: drewlearnsk8s
          image:  drewlearnsk8s
          ports:
          - name: nodejs-port
              containerPort: 3000
          livenessProbe:
              httpGet:
                  path: /
                  port: nodjs-port
              initialDelaySeconds: 15
              timeoutSeconds: 30
          readinessProbe:
              httpGET:
                  path: /
                  port: nodejs-port
              initialDelaySeconds: 15
              timeoutSeconds: 30

This may seem repetitive but that's a good thing when learning, run kubectl create -f example-readinessprobe.yaml and it should run that readiness probe deployment.

Pod State and LifeCycle

Pods and Containers can have different statuses and states. To understand their life cycle we first have to understand their status/state:

Pod Statuses

💡 You can get the pod conditions using kubectl describe pod <PODNAME>

Pod Conditions

Container State

You can see the container state using -n kube-system -o yaml which will show you the containers status

Container status can be "Running", "Terminated", or "Waiting".

Pod LifeCycle

1. Init container

You can launch a new container, that is separate from the main container that will execute some commands.

This can be interesting if you have volumes for example. To do some work on those volumes before the main container starts to set permissions, create some directories, etc before the main container starts. The main container will only start once init container has started.

2. Post Start Hook

The post start hook starts at the same time as the main container (if you define it in your configuration file.)

When you launch a pod, there are settings you can establish in your yaml file for post start hook and pre stop hooks within your pod spec (specification) to exited commands at certain points in the pod's lifecycle.

3. Readiness and Liveliness Probes.

There is an initial delay, which makes since because you wouldn't want to start checking the health of a container immediately upon creation because it won't be ready.

More information about these probes can be found in the readiness probe and the health check sections.

Secrets

Secrets is a way for kubernetes to share credentials, keys, passwords, and similar data to the pods.

A secret can also be an SSH key or an SSL certificate.

Kubernetes uses this "Secrets" mechanism to provide credentials to access the internal API.

This is only one way to provide secrets and is native to kubernetes, there are other methods for example, using external vault services.

Secrets can be used in the following ways:

• Use secrets to set environment variables
• Used as a file in a pod - this uses volumes to be mounted in a container. You can have files in this volume - for example, you could use dotenv files or your application can just read this volume.
• Use an external image to "pull" secrets from a private image registry

To generate secrets using files using these commands:

    echo -n "root" > /username.txt
    echo -n "password" > /password.txt
    kubectl create secret generic db-user-pass-from-file=/username.txt --from-file=/password.txt

Another Example:

    kubectl create secret generic ssl-certificate --from-file=ssh-privatekey=~/.ssh/id_rsa --ssl-cert-=ssl-cert-mysshcert.crt

To create secrets using Yaml definitions:

• Create a new file touch secrets-db-secret.yaml and edit it to look like this:

  apiVersion: v1
  kind: Secret
  metadata: 
      name: db-secret
  type: Opaque
  data:
      password: 
      username: 

• You can create the Base64 strings with the following command, just copy the command's output into your yaml file above:

  echo -n ""|base64
  echo -n ""|base64

After creating the yaml file, you can create it with that good 'ole kubectl create -f secrets-db-secret.yaml

Once the secrets are created then you can use them.

Using Secrets:

If you use environment variables, you need to create a pod that exposes the secrets as an environment variable in a yaml file:

    [...]
    env:
        - name: SECRET_USERNAME
        valueFrom:
            secretKeyRef:
                name: db-secret
                key: username
        - name: SECRET_PASSWORD
        [...]

Alternatively, you can provide the secrets in a file using volume mounts in your pod definition yaml.

    [...]
    volumeMounts:
    - name: credvolume
        mountPath: /etc/creds
        readOnly: true
    volumes:
    - name: credvolume
        secret:
            secretName: db-secrets 
    [...]

In the example above, the credentials will be saved in a file called /etc/creds/db-secrets/username & /etc/creds/db-secrets/password

Web UI

Kubernetes has a built in web user interface that you can use instead of kubectl. It's great for visual overview of running applications on your cluster, creating/modifying resources and workloads (think kubectl create and delete). It can also retrieve information on the state of resources (like kubectl describe pod).

You can typically access it by going to your master-node's URL /ui so for example: https://<kubernetes-master>/ui. You will not be able to access if it is not enabled on your deployment type.

You can install it manually using kubectl create -f "https://rawgit.com/kubernetes/dashbaord/master/src/deploy/kubernetes-dashboard.yaml" You may be prompted for a password which you can get from: kubectl config view.

In minikube you can use minikube dashboard

Advanced Topics

Different "Kinds" you can provision

ConfigMap

Configuration Maps are configuration parameters that are not secrets - the input is key-value pairs which can be read by the app using the similar methods as secrets:

• Environment Variables
• Container commandline arguments in the pod configuration
• Using volumes

It can also contain full configuration files. It can then be mounted using volumes where the application expects it's config file. This way you can "inject" configuration settings into containers without changing the container itself.

How do I generate configmaps using files

Run the following example command:

    cat < app.properties
    prod=a
    database=mysql
    environment=production
    EOF

    Kubectl create configmap app-config --from-file=app.properties

How do I use a configmap?

You can create a pod that exposes the config map using a volume.

        [...]
        volumeMounts:
        - name: config-volume
            mountPath: /etc/config 
        volumes:
        - name: config-volume
        configMap:
            name: app-config
        [...]

>💡 The config values will be stored in files: etc/config/driver & /etc/config/param/with/hierarchy

• Create that pod using `kubectl create -f

Ingress & Ingress Controller

Ingress is a solution that allows inbound connections to your cluster and is an alternative to external load balancers and nodePorts.

It allows you to easily expose services that need to be accessible from outside of the cluster and you can run your own controller within the kubernetes cluster (basically your own load balancer).

There are default ingress controllers available from kubernetes or you can write your own.

On public cloud providers you can use your ingress controller to reduce the cost of your load balancers by using 1 ELB that captures all external traffic and then sends it to the ingress controller as opposed to many ELBs and it could be configured to route different traffic to all your applications based on HTTP rules (host and prefixes) - bear in mind this only works on web applications leveraging http protocols.

External DNS

Route53 will automatically create the necessary DNS records - for every hostname that you use in ingress, it'll create a new record to send traffic to your load balancer automagically.

Alternatively you can use Google CloudDNS, Cloudflare, AzureDNS, Digital Ocean, and many others.

Volumes

Running Applications with state

Volumes in kubernetes allows you to store data outside of a container. Without this feature, all data on the container itself will be lost upon termination meaning it's stateless/ephemeral.

To make stateful containers - you will want to use external services like a database (MySQL), caching server, AWS S3, etc. In any case, your application needs read and write access to files in the local filesystem that need to be persistent in time.

Persistent volumes in kubernetes allows you to attache a volume to a container that will exist even when the container stops.

Volumes can be attached using different volume plugins based on your provider:

In the example below, imagine you have 2 nodes and node 1 fails/terminates. The "myapp" pod will be rescheduled on another node and the volume can be attached to node 2 allowing it to keep it's state.

Volume Provisioning

Kubernetes plugins have the ability to provision storage for you - the aws plugin specifically can provision storage by creating volumes before attaching them to a node using the "StorageClass" object.

To use auto provisioned persistent volume claims using the aws-ebs provisioner you can create a StorageClass yaml to look something like this:

    kind: StorageClass
    apiVersion: storage.k8s.io
    metadata:
        name: standard
    provisioner: kubernetes.io/aws-ebs
    parameters:
        type: gp2
        zone: us-east-1a

💡 gp2 is "general purpose - SSD"

Next, you will want to create your persistent volume claim.

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
        name: myclaim
        annotations:
            volume.beta.kubernetes.io/storage-class: "standard"
    spec:
        accessModes:
            - ReadWriteOnce
        resources:
            requests:
                storage: 8Gi 

Pod Presets

Presets can inject information into pods at runtime like secrets, configmaps, volumes, and environment vairables.

You can create 1 preset object which will inject a resource into all matching pds.

When injecting environment variables an d VolumeMounts, the pod preset will apply the change to all containers within the pod.

You can use more than one PodPreset, they'll all be applied to 0 or more matching pods. If there is a conflict, the PodPreset will not be applied to the pod. If no pods currently match but are later launched - they will be applied at that time.

StatefulSets

Statefulsets allow stateful applications to have stable storage with volumes based on their ordinal number (example podname-#).

This feature enables the use of Stateful applications that need stable pod hostname (instead of podname-<randomstring>).

Your pod name will have a sticky identity using an index (podname-0, podname-1, etc for example) and when a pod gets rescheduled, it'll keep that identity.

Deleteing and/or scaling a StatefulSet down will not delete the volumes associated with the StatefulSet (preserving data).

💡 A statefulSet will allow your stateful app to use DNS to find it's peers. ElasticSearch clusters use DNS to find other members of their cluster.

If you don't use StatefulSet, you would get a dynamic hostname which would be impossible to use in your configuration files since the name can change without notice.

A StatefulSet will also allow your stateful app to order the startup and teardown instead of randomly terminating a pod, you will know which one will get the axe.

• When scaling up it goes from 0 to n-1 (n = replication factor).
• When scaling down, the first pod to go will be the one with the highest podname-#.
  • This is useful if you first need to drain the data from a node before it can be shut down.

Daemon Sets

Daemon Sets ensure that every single node in the kubernetes cluster runs the same pod resource meaning if you want to ensure that a certain pod is running on every single kubernetes node, you can and when a node is added to the cluster, a new pod will be started automatically (same thing happens when the node is removed, the pod won't be rescheduled on another node).

Typical use cases:

• Logging aggregators
• Monitoring
• Load Balancers
• Reverse Proxies
• API gateways
• Running a daemon that only needs one instance per physical instance

Resource Usage Monitoring

Heapster enables Container Cluster Monitoring and Performance Analysis and exports cluster metrics via REST endpoints.

It's providing a monitoring platform for kubernetes and is a prerequisite if you want to do pod auto-scaling in kubernetes.

You can use different backends with Heapster such as CloudWatch, Google Cloud Monitoring/Logging, kafka or influxDB.

It can be shown on graphs (visualizations) using grafana. The kubernetes dashboard can also show these visualizations.

All these technologies can be started in Pods. You can find the premade yaml files on Heapster's Git repo

• After downloading the repo above you can deploy using kubectl create -f <directory-with-yaml-files

Autoscaling (Horizontal Pod AutoScaling)

Kubernetes has the possibility to automatically scale pods based on metrics such as cpu load, queries per second, or average request latency.

To enable this you have to start the cluster with the env var ENABLE_CUSTOM_METRICS set to "true".

Autoscaling will periodically query the utilization for the targeted pods and will do so by default every 30 seconds. Autoscaling will use heapster (the monitoring tool mentioned in the previous section)

Heapster must be installed and running before autoscaling will work.

With AWS you may want to use autoscaling groups for this instead of managing it with kubernetes directly.

You can set up autoscaling using yaml like this:

    apiVersion: autoscaling
    kind: HorizontalPodAutoscaler
    metadata:
        name: "hp-example-autoscaler"
    spec:
        scaleTargetRef:
            apiVersion: extensions
            kind: deployment
            name: hpa-example
        minReplicas: 1
        maxReplicas: 10
        targetCPUUtilizationPercentage: 50

Node Affinity and InterPod (Anti-)Affinity

Node Affinity

In a previous section I discussed how to make resources only use a specific lablel. Affinity/Anti-Affinity feature allows you to do more complex scheduling than the nodeSelector and also works on Pods.

The language is more expressive and you can create rules that are not hard requirements but rather preferred rules. This means that the scheduler will still be able to create your pod even if the rules cannot be met. You can even create rules that take other pod labels into account. An example of this would be to make sure 2 different pods will never be on the same node.

Kubernetes can do node affinity and pod affinity/anti-affinity. Node afinity is very similar to the nodeSelector whereas pod affinity/anti-affinity allows you to create rules on how pods should be scheduled taking into account other running pods.

Affinity/anti-affinity mechanism is only relevant during scheduling so once a pod is running, it'll need to be recreated to apply the rules again.

Affitinity VS anti Affinity

You can leverage weighting to produce "preferences". The higher the weighting in the preferredDuringSchedulingIgnoreDuringExecution, the more preference is given to a rule. When scheduling, kubernetes will "score" every node by summarizing the weightings per node and the node that has the highest score is where the pod will be scheduled on. Let's look at an example:

    spec:
        affinity:
            nodeAffinity:
                requiredDuringSchedulingIgnoredDuringExecution:
                    nodeSelectorTerms:
                    - matchExpressions:
                        - key: env
                            operator: In
                            values:
                            -   dev
                    preferredDuringSchedulingIgnoreDuringExecution:
                    - weight: 1
                        preference:
                            matchExpressions:
                            - keyL team
                                operator: In
                                values:
                                - engineering-project1
            containers:
            [...]

There are also built-in node labels:

• In addition to the labels that you can add yourself to nodes, there are pre-populated labels that you can use such as kubernetes.io/hostname

InterPod (Anti-)Affinity

This mechanism allows you to influence scheduling based on the labels of other pods that are already running on the cluster - these pods belong to a namespace, so your affinity rules will apply to a specific namespace (If none provided it defaults to the pod's namespace).

Similar to node affinity you have 2 types of pod affinity/anti-affinity:

1. requiredDuringSchedulingIgnoredDuringExecution - creates a rule that
   must

   be met for the pod to be scheduled.

2. preferredDuringSchedulingIgnoredDuringExecution - creates a rule that the preferred type is a "soft" type and rules may be met.

💡 When writing pod affinity rules, you can use In and NotIn as operators but bear in mind, interpod affinity and anti-affinity utilizes a lot of processing and may not be desirable in large clusters.

A good use case for pod affinity is co-located pods - you may want it so that 1 pod is always co-located on the same node as another pod. Example: Your application uses redis for cache and you want to have the application and the redis cache pod on the same node.

What nodes would these examples create the new pod on?

When writing pod affinity/anti-affinity rules - you need to specify a topology domain, called topologyKey in the rules.

The topologyKey refers to a node label and if the affinity rule matches, the new pod will only be scheduled on nodes that have the same topologyKey value as the current running pod.

Taints and Tolerations

Tolerations is the opposite of node affinity and it allows a node to repel a set of pods. Taints mark a node, tolerations are applied to a pod to influce the scheduling of those pods.

A use case is to make sure that when you create a new pod, thare not scheduled on the master.

You can add a new taint to a node using kubectl taint nodes <NODE_NAME> key=value.NoSchedule.

This will make sure that no pods will be scheduled on as long as they don't have a matching tolerations.

    [...]
    tolerations:
    - key: "key"
        operator: "Equal"
        value: "value"
        effect: "NoSchedule"
    [...]

This tolerations above would allow the taint we created with kubectl above to run on it since it matches on the key and value.

Just like affinity taints can also be a preference rather than a requirement using NoSchedule (a hard requirement) or PreferNoSchedule (k8s will try to avoid placing a pod that doesn't have a matching tolerance but is not a hard requirement).

If the taint is applied while there are already running pods, these will not be evicted, unless you use NoExecute: evict on pods with non-matching tolerations. You can also specify how long the pod can run on a tainted node before eviction within the NoExecute specifications using something like tolerationSeconds: 10000. If you do not specify this tolerationSeconds value, the toleration will match and the pod will keep running on the node. So in that snippet example, the pod would be evicted after 10000 seconds.

Example use cases are:

• The existing node taints for master nodes (default)
• Taint nodes that are dedicated for a team or a user
• If you have a few nodes with specific hardware (GPUs for example), you can taint them to avoid running non-specific applications on those nodes.
• Taint nodes by condition which will automatically taint nodes that have problems allowing you to add tolerations to time the eviction of pods from nodes. Pretty neat stuff.

You can enable alpha features by passing the --feature-gates to the k8s controller manager, or in kops.

    [...]
    spec:
        kubelet
            featureGates:
                TaintNodesByCondition: "true"
    [...]

Taint Operators

You can use the following operators:

Equal: providing a key and value.
Exists: only providing a key, checking only whether a key exists.

Taint Keys

You can set the following pre-built taint key values:

• node.kubernetes.io/not-ready - Node is not ready.
• node.kubernetes.io/unreachable - Node is unreachable from the node controller.
• node.kubernetes.io/out-of-disk - Node is out of disk space.
• node.kubernetes.io/memory-pressure - Node has memory pressure.
• node.kubernetes.io/disk-pressure - Node has disk pressure.
• node.kubernetes.io/network-unavailble - The network isn't responding on the Node
• node.kubernetes.io/unschedulable - Node is unscheduable.
• node.kubernetes.io/master - Master Node is unavailable to schedule pods

Customer Resource Definitions

A custom resource that you might add to your cluster, it's not available on every cluster and described in you yaml file.

Custom resource definitions allow you to extend the kubernetes API - these resources are the endpoints in the kubernetes API that store collections of API objects.

As an administrator, you can dynamically Custom Resource Definitions to add extra functionality to your cluster.

An example:

You have a built-in Deployment resource that you can use to deploy applications. In your yaml files, you describe the object using the "Deployment" type and then you create the object on the cluster using kubectl.

Custom Resource Operators

An operator is a method of packaging, deploying and managing a Kubernetes Application. --[source](http://coreos.com/operators). This is especially beneficial because it hides the complexities of cloud infrastructure from the end user.

The benefit of custom resource operators is that it puts operational knowledge in an application which brings the developers and operations teams closer to the experience of a managed cloud service rather than having to know all the specifics of an application deployed with kubernetes.

After a custom resource operator is deployed, it can be managed using custom resource definitions (types that extend the kubernetes API.) and provides a great way to t deploy stateful services.

Any third party can create operators that you can start using - prometheus for example is a great use case I intend to write notes up on soon.

An operator contains a bunch of management logic that an administrator may crave rather than having to implement everything individually. For example, if you wanted to create a PostgreSQL server, you could do it as an operator and it'll allow you to also create backups, auto scaling, replicas, and even a failover.

If you'd like to try a PostgreSQL operator, [here is a link](https://gitlab.com/DrewKarriker/postgrsql-operator) to some starter files: with readme.md instructions included.

---

Quotas and Limits

When a Kubernetes cluster is used by multiple people or teams, resource management become more important and you want to be able to manage the resources you give to a person or team.

You certainly don't want anyone to hog all the resources (memory or CPU) of the cluster.

You can divide your cluster into Namespaces and then enable resource quotas on it using ResourceQuota and ObjectQuota.

Each Container can specify **Request Capacity** and **Capacity limits**.

Request Capcity is a hard explicit request for resources. The scheduler can then use the request capacity to make decisions on what Node create a Pod on. This can be seen as a "Minimum amount of resources" the pod needs to be successful.

Lastly, a Resource Limit is a limit imposed on the **container** as a hard cap for storage, memory, CPU, etc...

If a capacity quota has been specified by the administrator, then each pod needs to specify capacity quota during creation. An example:

**CPU resource request of 200m**

200m == 200millicpu == 200 millicores
200m = 0.2 which is 20% of a CPU core of the running node. If there are 2 CPUs on the node, it's still 20% of a single core.

💡 You can also limit it and set memory quotas which are defined by MiB or GiB

The administrator can can specify default request values for pods that don't specify any values for capacity and the same applies for limit quotas.

💡 If a resource is requested more than the allowed capacity, the server API will give an error 403 FORBIDDING and kubectl will show an error.

Resource limits

You can create resource quota yaml like this:

	apiVersion: v1
	kind: ResourceQuota
	metadata:
		name: compute-resources
		namespace: <NAME-OF-NAMESPACE>
	spec:
		hard:
			requests.cpu: *1*
			limits.cpu: *2*
			limits.memory: 2Gi

Object Limits

You can also create object limits like this:

	apiVersion: v1
	kind: ResourceQuota
	metadata:
		name: object-counts
		namespace: <NAME-OF-NAMESPACE>
	spec:
		hard:
			configmaps: *10*
			persistentvolumeclaims: *4*
			replicationcontrollers: *20*
			secrets: *10*
			services: *10*
			services.loadbalancers: *2*

Namespaces

Namespaces are virtual clusters within your physical cluster and allow you to logically separate your project's clusters.

The intention of namespaces is to help segregate multiple teams and projects within your kubernetes clusters so that you can allocate appropriate resources and group nodes appropriately. This is especially important with enterprise companies.

The names of resources need to be unique within a namespace but not across namespaces. That is to say you can have a deployment called "hello_world" in multiple different namespaces but not twice in the same namespace.

💡 The standard namespace is called "default" and that's where all resources are launched by default.

💡 There is also a default namespace for kubernetes specific resources called "kube-system"

You can then divide resources on a per namespace basis. More on that in the Resource Quota section.

You can see a list of namespaces using kubectl get namespaces and also create them with kubectl create namespace <NAME-OF-NAMESPACE>

You can also set default namespace to launch resources in by running:

	export CONTEXT=$(kubectl config view | awk '/current-context/ {print $2}')
	kubectl config set-context $CONTEXT --namespace=<NAME-OF-NAMESPACE>`

User Management

There are two types of users you can create Normal Users & Service Users.

1. Normal User

Used to access the user externally through kubectl and is not managed using objects.

There are various athentication stratagies for normal users:

 - Client Certificates
 - Bearer Tokens
 - Authentication Proxy
 - HTTP Basic Authentication
 - OpenID
 - Webhook: sends authorization request to an external REST interface

Independant of the authentication mechanism, normal users have the following attributes:

• A username
• A UserID (UID)
• Groups
• Extra fields to store misc information

After a normal user authenticates, it will have access to everything. This isn't necessarily good so to limit access you need to configure the authorization using one of many options:

• AlwaysAllow / AlwaysDeny
• ABAC: attribute based access control - access rights controlled by polices that combine attributes. (Not granular)
• RBAC (Role Based Access Control)
• Webhook (Authorization by a remote service)
• Nodes: a special purpose authorization mode that authorizes API requests made by kubelets

2. Service user

Specific to a namespace, service users are managed by objects in kubernetes and are automatically created by the API or manually using objects. It's used to authenticate within the cluster (from a kubelet) and are managed like secrets.

Service users will leverage Service Account Tokens which are stored as credentials using [Secrets](#secrets)

Those secrets are also mounted in pds to allow communication between the services.

> 💡 Any API call not authenticated is considered as an anonymous user.

RBAC

RBAC (Role Based Access Control) uses rbac.authorization.k8s.io API group and allows admins to dynamically configure permissions through the API and is enabled by default. Roles are added to users. If you provision a new cluser using Kops or kubeadmn, it'll be set by default to RBAC.

After authentication, authorization controls what the user can do and what they do not have access to. These access controls are implemented on the API level (kube-apiserver).

When an API request comes in (such as kubectl get nodes), it will be checked to see whether you have access to execute this command.

For more details visit http://kubernetes.io/docs/admin/authorization

To enable an authorization mode, you need to pass --authorization-mode=RBAC to the API server at startup.

You can add RBAC resources with kubectl to grant permissions, first describe them then apply them to the cluster.

You can create roles limited to a namespace or you can crete roles where the access applies to all namespaces:

• Role - single namespace
• ClusterRole - applies to entire cluster
• RoleBinding - single namespace
• ClusterRoleBinding - You guessed it, applies to the entire cluster.

Example yaml for creating a new Role (ClusterRole looks almost identical):

	kind: Role
	apiVersion: rbac.authorization.k8s.io/v1
	metadata:
		namespace: default
		name: pod-reader
	rules:
	- apiGroups: [""]
		resources: ["pods", "secrets"]
		verbs: ["get", "watch", "list"]

After you have created a new role, then you can create a RoleBinding to assign users the new role. The process for ClusterRoleBinding is the same but kind is different. Take a look:

	kind: RoleBinding
	apiVersion: rbac.authorization.k8s.io/v1
	metadata:
		name: read-pods
		namespace: default
	subjects:
	- kind: User
		name: Drew
		apiGroup: rbac.authorization.k8s.io
	roleRef:
		kind: Role
		name: pod-reader
		apiGroup: rbac.authorization.k8s.io

Networking

So far we have covered container to container communication through local host and the port number, pod to service using NodePort and DNS, and also external to service communication through a load balancer and nodeport.

In kubernetes, the pod should always be routable with pod to pod communications and it assumes that pods are able to commincate across nodes.

Every pod will have it's own IP address.

On AWS: Kubernet networking (kops defualt) - every pod can get an IP that is routable using AWS VPC (virtual private network).

The kubernetes master allocates a 254 IP addresses to each node using a /24 cidr subnet to each node.

There is also a 50 node limit on a single AWS cluster due to a limit of 50 entries possible on the VPC (though this can be raised to 100, it's not advisable).

Node Maintenance

In our Diagram from the architecture section you can see the Node Controller is responsible for managing the node objects. It assigns IP space to the node when a new Node is launched and it keeps the node list up to date with the available machines. Meanwhile, it's also monitoring the health of nodes and if a node gets unhealthy, it gets deleted and any pods running on the unhealthy node will get rescheduled.

When adding a new node, the kubelet will attempt to register itself, this is called self-registration and is the default behavior. It allows you to easily add more nodes without making API changes yourself.

A new node object is automatically created with:

• The metadata - The default value will be the name: IP or Hostname
• Labels - the default value will be the Availability zone

When you want to gracefully decommision a node, you can do a "drain" which removes all it's resources before you terminate it. You can do so with kubectl drain <nodename> --grace-period=600. If the pod isn't managed by a controller, you can use kubectl drain <nodename> --force.

High Availability

When I was in the Navy, everything came in sets of 3+ for high availability. This was because in the event of catastrophy, our crew would have two backups for everything that was running. The saying was "3 is 2, 2 is 1, and 1 your dead in the water".

If you are going to run a cluster in production, you are going to want to have all your master services in a high availability setup which will look something like this:

The intention is to ensure your nodes are scaled horizontally with all of it's containers duplicated to avoid the dead in the water scenario.

TLS on ELB

You can set up cloud specific features like TLS termination on AWS ELB (Elastic Load Balancers) that you create in kubernetes using services of type LoadBalancer.

You can do this using annotations like so:

	apiVersion: 1
	kind: Service
	metadata:
		name: example-service
		annotations:
			service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xx-xxxx-x:xxxxxxxxxx:xxxxxxx/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
			service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http

You can provision certificates on AWS certificate Manager.

1. Request a plublic certificate for the domain in our hosted zone.
2. Select DNS validation and create the certificate.
3. It will ask you to create a CNAME and there is a blue button that read "Create record on Route 53" which will update your DNS records without having to do it manually.
4. Get your ARN:
5. Use our elb-demo Yaml and deploy it using kubectl create -f <filename>
6. We can then get the hostname using kubectl get services -o wide copy that hostname and head back to AWS route53 and create a new A record aliased to your ELB hostname.
7. curl your new https address and see if "hello world" is produced in your terminal.

Packaging

Introduction to Helm

We spent a ton of time learning Kubernetes and how it works. We also spent a large amount of time using kubectl but now that you made it to this section, you'll want to know that most of those commands won't be useful once you start using helm -- if the place you work uses helm that is.

Helm is a package manager for kubernetes and it's the best way to find, share, and use software built for kubernetes. It helps you to manage kubernetes applications as code and it's maintained by the CNCF - The Cloud Native Computing Foundation along with Google, Microsoft, Bitnami, and the Helm contributor community.

Helm uses a packing format called "Charts" (get it? "Helm Charts" 🥁) which is a collection of files that describe a set of kubernetes resources. It also comes with it's own CLI tool called helm-cli that has commands very similar to kubectl, kops, and minikube.

A single chart can deploy an application or database and it can have dependencies such as a mysql chart for a wordpress site. You can write your own chart to deploy your application on kubernetes using helm. Charts use templates that are typically developed by a package maintainer. They will generate yaml files that kubernetes understand. You can think of templates as dynamic yaml files which can contain logic and variables.

This is what those variables look like: 

Install Helm and start using it

1. To start using helm, you first need to download the helm client. brew install helm
2. Update your helm repo using helm repo add stable https://charts.helm.sh/stable

You can create your own helm charts to deploy your own applications in kubernetes and it is the recommended method. Packaging the application allows you to deploly it with 1 command instead of kubctl create / apply. Another benefit of helmcharts is that it's version controlled allowing for easy rollbacks.

Creating Helm Charts

1. You can create a new chart for your helm with a simple command helm create mychart💡 Replace the mychart with something unique but stay under the 63 character limit because some kubernetes fields are limited to they by the DNS naming spec.
2. A new folder will have appeared that should have a number of files created under the mychart directory. It should look something like this:

- In chart.yaml you will find some meta data which is all boilerplate.
- In the values.yaml - it contains all the values you will use on your deployment.
- In the templates/ you will find deployment.yaml and service.yaml. These files are the kubernetes yaml files that will be used and you can use the template mechanism to make dynamic yaml files.

3. Inside the values.yaml file you can set the replicaCount: value to any number you want, this will tell kubernetes how many pods to spin up. You can edit other values that will be fed into your deployment.yaml file dynamically
4. Inside the deployment.yaml, service.yaml, and ingress.yaml you can see it's mostly just dynamic fields in a boilerplate .yaml file filled by the values.yaml file above.
5. Run helm install mychart --generate-name to install the helm files on your cluster and you should see an output like this: NAME: mychart-1613969577 LAST DEPLOYED: Sun Feb 21 23:53:00 2021 NAMESPACE: default STATUS: deployed REVISION: 1 NOTES: 1. Get the application URL by running these commands: export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=mychart,app.kubernetes.io/instance=mychart-1613969577" -o jsonpath="{.items[0].metadata.name}") export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}") echo "Visit http://127.0.0.1:8080 to use your application" kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
6. Run kubectl get pods and you should see an output like: NAME READY STATUS RESTARTS AGE mychart-1613969577-f676bff69-rgcdn 1/1 Running 0 98s
7. Run kubectl get deployment and you can see it's deployed. NAME READY UP-TO-DATE AVAILABLE AGE mychart-1613969577 1/1 1 1 2m40s
8. Run export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=mychart,app.kubernetes.io/instance=mychart-1613969577" -o jsonpath="{.items[0].metadata.name}") && echo $POD_NAME and you should see an output like: mychart-1613969577-f676bff69-rgcdn
9. Run export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}") && echo $CONTAINER_PORT and you should see an output of 80.
10. Now we need to port-forward our pod using kubectl port-forward $POD_NAME 8080:80 & and you should see an output like this: Forwarding from 127.0.0.1:8080 -> 80 Forwarding from [::1]:8080 -> 80

💡 That ampersand (&) at the end of that command means "Run this code in the background and don't hold my terminal hostage" or you can run bg

11. Now see if your server is running by opening the browser to http://localhost:8080 or curl http://127.0.0.1:8080 and you should get the default Nginx Load page.
12. To bring your port-forwarding command back to the foreground run fg then close it with control + c and the port-forward will close and our curl wont work anymore.
13. Run helm list to get your helm names and then we are going to terminate it. Your output should look like this: NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION mychart-1613969577 default 1 2021-02-21 23:53:00.096718 -0500 EST deployed mychart-0.1.0 1.16.0
14. Terminate the deployment using helm delete mychart-1613969577.

Helm Repository

Create a chart repository in Amazon S3

1. Use aws configure to login to your aws account from terminal.
2. Create a new file and call it setup.sh in your working directory. touch setup.sh && nano setup.sh
3. We want to create an s3 bucket in AWS and make it create a remote repository for us, here is a command that will do this automatically for you!In your nano editor paste the following: #!/bin/bash set -e # create random string RANDOM_STRING=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | tr '[:upper:]' '[:lower:]' | head -n 1) # > 🚨 be sure to change your `AWS_REGION` to your appropriate AZ DEFAULT_REGION="us-east-1" AWS_REGION="${AWS_REGION:-${DEFAULT_REGION}}" export AWS_REGION # create s3 bucket if [ "$AWS_REGION" == "us-east-1" ] ; then aws s3api create-bucket --bucket helm-${RANDOM_STRING} else aws s3api create-bucket --bucket helm-${RANDOM_STRING} --region $AWS_REGION --create-bucket-configuration LocationConstraint=${AWS_REGION} fi # install helm s3 plugin helm plugin install https://github.com/hypnoglow/helm-s3.git # initialize s3 bucket helm s3 init s3://helm-${RANDOM_STRING}/charts # add repository to helm helm repo add my-charts s3://helm-${RANDOM_STRING}/charts 🚨 be sure to change your DEFAULT_REGION to your appropriate AZ
4. Press control + x, then y, then return.
5. Make that file executable chmod -x setup.sh
6. Run sh setup.sh to execute that bash-script.
7. You can check to see if it was created - Run aws s3 ls -or- if you prefer .json readout you can use aws s3api list-buckets. You should see an output similar to this: 2021-02-21 23:23:54 helm-hlb
8. Run export AWS_REGION=us-east-1🚨 be sure to change your AWS_REGION to your appropriate AZ.
9. Verify your remote repository is set up by running helm repo list and you should see something like this: NAME URL stable https://charts.helm.sh/stable my-charts s3://helm-hlb/charts Now we can add our "mychart" application to s3.
10. Package up the deployment using helm package mychart - a new file should have been created in your working directory called "mychart-0.1.0tgz"
11. export AWS_REGION=us-east-1a
12. Push it using helm s3 push mychart-0.1.0.tgz my-charts💡 We add "my-charts" above because that is the name of the s3 repository we got from step 9 above.
13. We can then see the helm chart in the remote repository by running: helm search repo mychart and you should see an output that looks like this: NAME CHART VERSION APP VERSION DESCRIPTION my-charts/mychart 0.1.0 1.16.0 A Helm chart for Kubernetes
14. You can now install that helm chart using helm install my-charts/mychart --generate-name which will pull from the remote repository called "my-charts"
15. Run helm list to get your helm names and then we are going to terminate it. Your output should look like this: NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION mychart-1613969577 default 1 2021-02-21 23:53:00.096718 -0500 EST deployed mychart-0.1.0 1.16.0
16. Terminate the deployment using helm delete mychart-1613969577.

Commands Quick Reference

Helm Commands

• helm init - Install tiller on the cluster (not a thing on helm3 or helm4)
• helm reset - Remove tiller from the cluster
• helm install - Install helm chart
• helm search - search for a chart
• helm list - list releases (installed charts)
• helm upgrade - upgrade a release
• helm rollback - rollback a release to the previously active version

KOPS Commands

• kops create cluster --name=kubernetes.newtech.academy --state=s3://kops-state-<RANDOMSTRING> --zones=eu-west-1a --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=kubernetes.newtech.academy
• kops update cluster kubernetes.newtech.academy --yes --state=s3://kops-state-<RANDOMSTRING>
• kops delete cluster --name kubernetes.newtech.academy --state=s3://kops-state-<RANDOMSTRING>
• kops delete cluster --name kubernetes.newtech.academy --state=s3://kops-state-<RANDOMSTRING> --yes
• kops validate cluster --wait 10m --state=s3://kops-state-<RANDOMSTRING>: verify your services are created and healthy after running kops create.

DOCKER Commands

• Build image: docker build .
• Build & Tag: docker build -it registry.gitlab.com/drewkarriker/drew-learns-Devops:latest .
• -i : interactive flag, when you run this, the pod will stay alive as long as the shell is active and terminate upon closing
• -t : tag
• Tag image: docker tag imageid registry.gitlab.com/drewkarriker/drew-learns-Devops
• Push image: docker push registry.gitlab.com/drewkarriker/drew-learns-Devops
• List images: docker images
• List all containers: docker ps -a
• Docker Hub has a ton of pre-built images that you can "pull" and try without needing to define your own: docker pull [OPTIONS] NAME[:TAG|@DIGEST]

Kubernetes Commands

• kubectl get po -A will show you all containers, their name space, and status
• kubectl get pod: Get information about all running pods
• kubectl describe pod <pod>: Describe one pod
• kubectl expose pod <pod> --port=444 --name=frontend: Expose the port of a pod (creates a new service)
• kubectl port-forward <pod> 8080: Port forward the exposed pod port to your local machine
• kubectl attach -i: Attach to the pod
• kubectl exec -- command: Execute a command on the pod
• kubectl label pods mylabel=awesome: Add a new label to a pod
• kubectl run -i --tty busybox --image=busybox --restart=Never -- sh: Run a shell in a pod - very useful for debugging
• kubectl get deployments: Get information on current deployments
• kubectl get rs: Get information about the replica sets
• kubectl get pods --show-labels: get pods, and also show labels attached to those pods
• kubectl rollout status deployment helloworld-deployment: Get deployment` status
• kubectl set image deployment helloworld-deployment example=example:2`: Run example with the image label version 2
• kubectl edit deployment/helloworld-deployment: Edit the deployment object
• kubectl rollout status deployment/helloworld-deployment: Get the status of the rollout
• kubectl rollout history deployment helloworld-deployment: Get the rollout history
• kubectl rollout undo deployment helloworld-deployment: Rollback to previous version

-kubectl rollout undo deployment helloworld-deployment --to-revision=n: Rollback to any version version

AWS Commands

• aws ec2 create-volume --size 10 --region us-east-1 --availability-zone us-east-1a --volume-type gp2 This command will create a new volume on US-East-1 AZ
• aws s3 mb s3://<SOMERANDOMSTRING>: Create a new s3 bucket. (mb == make bucket)
• aws s3 rb s3://<SOMERANDOMSTRING>: Delete an s3 bucket. (rb == remove bucket)
• aws elb describe-load-balancers --load-balancer-name my-loadbalancer: Find loadbalancer details
• aws sts get-caller-identity: verify you are able to connect to amazon services with AWS-CLI
• aws s3 ls -or- if you prefer .json readout you can use aws s3api list-buckets to view s3 buckets on your account.

Creating a new key for a new user: openssl genrsa -out myuser.pem 2048

• Creating a certificate request: openssl req -new -key myuser.pem -out myuser-csr.pem -subj "/CN=myuser/O=myteam/"
• Creating a certificate: openssl x509 -req -in myuser-csr.pem -CA /path/to/kubernetes/ca.crt -CAkey /path/to/kubernetes/ca.key -CAcreateserial -out myuser.crt -days 10000

MiniKube

• minikube start will start minikube
• minikube status will tell you the whether minikube is running or not
• minikube service <DeploymentName> will start the service name and open a browser
• minikube stop will stop minikube
• minikube delete will delete the vm cluster and delete the profile from your minikube config

Misc Commands

• ssh-keygen -R api.kubernetes.codingwithdrew.com will fix "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!" when SSH'ing into your cluster.

INDEX

• Objectives
• Is this for me?

Introduction to Kubernetes

• What is Kubernetes?
• What does it do though?
• Architectural Overview of how Kubernetes works
  • Whats a node?
• What is Containerization?
  • What is a container?
  • Virtualization:
  • Containerization:
  • Public Cloud
• Where can you run Kubernetes?
• Devops Methodology to Know and Love
  • The Twelve Factors for stateless applications
• Kubernetes Demo
  • Getting Accounts Setup Before We Begin
  • Install K8s Locally
  • Minikube (pronounced "Mini Cube") Intro
  • Installing Minikube
  • Cluster Setup
  • Install KOPS Demo
  • Deploy to AWS using Kubernetes
  • What is Docker?
  • Benefits:
  • Building Containers
  • Dockerfile
  • Create A Dockerfile
  • Setup your Javascript Bits
  • Building and Running your first Container Image
  • Deploy a Docker Image to a Container Registry
  • Deploy the Demo Application to a New Kubernetes Cluster
• Wait, what's a pod?
• Ok, so now I know what a Pod is but how do I define it?
• Getting familiar with Docker commands
• Demo kubectl with Docker
  • Port forwarding
  • Set Up an External AWS Load balancer
  • ELB Demo Time
• Kubernetes Introduction Summary

Kubernetes Deeper Dive

• Node Architecture
• Scaling Pods
• Deployments
• So what is a deployment?
• Services
  • Demo - Create a service of type NodePort
• Labels
• What is the benefit?
• How do make a resource only use a specific label?
• How do I label objects after I created it?
• Heath checks (Liveliness Probe)
• How does a health check look in Yaml?
• Cool, how do I use that health check though?
• Readiness Probe
• How does a readiness probe look in Yaml?
• Pod State and LifeCycle
• Pod Statuses
• Pod Conditions
• Container State
• Pod LifeCycle
  • 1. Init container
  • 2. Post Start Hook
  • 3. Readiness and Liveliness Probes.
• Secrets
• Using Secrets:
• Web UI

Advanced Topics

• Different "Kinds" you can provision
• ConfigMap
• How do I generate configmaps using files
• How do I use a configmap?
• Ingress & Ingress Controller
• External DNS
• Volumes
• Volume Provisioning
• Pod Presets
• StatefulSets
• Daemon Sets
• Resource Usage Monitoring
• Autoscaling (Horizontal Pod AutoScaling)
• Node Affinity and InterPod (Anti-)Affinity
• Node Affinity
  • Affinity VS anti Affinity
  • InterPod (Anti-)Affinity
• Taints and Tolerations
  • Taint Operators
  • Taint Keys
• Customer Resource Definitions
  • Quotas and Limits
  • Resource limits
  • Object Limits
• Namespaces
• User Management
  • 1. Normal User
  • 2. Service user
• RBAC
• Networking
• Node Maintenance
• High Availability
• TLS on ELB

Packaging

• Introduction to Helm
• Install Helm and start using it
• Creating Helm Charts
• Helm Repository

Commands Quick Reference

• Helm Commands
• KOPS Commands
• DOCKER Commands
• Kubernetes Commands
• AWS Commands
• Creating a new key for a new user
• MiniKube
• Misc Commands

Drew Karriker

Drew is a seasoned DevOps Engineer with a rich background that spans multiple industries and technologies. With foundational training as a Nuclear Engineer in the US Navy, Drew brings a meticulous approach to operational efficiency and reliability. His expertise lies in cloud migration strategies, CI/CD automation, and Kubernetes orchestration. Known for a keen focus on facts and correctness, Drew is proficient in a range of programming languages including Bash and JavaScript. His diverse experiences, from serving in the military to working in the corporate world, have equipped him with a comprehensive worldview and a knack for creative problem-solving. Drew advocates for streamlined, fact-based approaches in both code and business, making him a reliable authority in the tech industry.