Intros

• Hello! We are:

  • Brice (@bdereims, VMware)

  • Jérôme (@jpetazzo, Enix)

• Feel free to interrupt for questions at any time

• Especially when you see full screen container pictures!

logistics.md

A brief introduction

• This was initially written by Jérôme Petazzoni to support in-person, instructor-led workshops and tutorials

• Credit is also due to multiple contributors — thank you!

• You can also follow along on your own, at your own pace

• We included as much information as possible in these slides

• We recommend having a mentor to help you ...

• ... Or be comfortable spending some time reading the Kubernetes documentation ...

• ... And looking for answers on StackOverflow and other outlets

k8s/intro.md

About these slides

• All the content is available in a public GitHub repository:

  https://github.com/jpetazzo/container.training

• You can get updated "builds" of the slides there:

  http://container.training/

Chapter 1

• Pre-requirements

• Our sample application

• Kubernetes concepts

• First contact with kubectl

(auto-generated TOC)

Chapter 2

• Running our first containers on Kubernetes

• Accessing logs from the CLI

• vRLI

• Declarative vs imperative

• Kubernetes network model

• Exposing containers

• NSX-T

• Shipping images with a registry

• Running our application on Kubernetes

(auto-generated TOC)

Chapter 3

• Deploying with YAML

• Setting up Kubernetes

• PKS

• Scaling our demo app

• vROPS

• Rolling updates

• Namespaces

(auto-generated TOC)

Chapter 4

• Volumes

• Managing configuration

• Highly available Persistent Volumes

• vSAN

• Next steps

• Links and resources

(auto-generated TOC)

shared/toc.md

Pre-requirements

• Be comfortable with the UNIX command line

  • navigating directories

  • editing files

  • a little bit of bash-fu (environment variables, loops)

• Some Docker knowledge

  • docker run, docker ps, docker build

  • ideally, you know how to write a Dockerfile and build it
    (even if it's a FROM line and a couple of RUN commands)

• It's totally OK if you are not a Docker expert!

shared/prereqs.md

Hands-on sections

• The whole workshop is hands-on

• We are going to build, ship, and run containers!

• You are invited to reproduce all the demos

• All hands-on sections are clearly identified, like the gray rectangle below

shared/prereqs.md

Navigating slides

• Use arrows to move to next/previous slide

  (up, down, left, right, page up, page down)

• Type a slide number + ENTER to go to that slide

• The slide number is also visible in the URL bar

  (e.g. .../#123 for slide 123)

• Slides will remain online so you can review them later if needed

shared/prereqs.md

Doing or re-doing the workshop on your own?

• Use something like Play-With-Docker or Play-With-Kubernetes

  Zero setup effort; but environment are short-lived and might have limited resources

• Create your own cluster (local or cloud VMs)

  Small setup effort; small cost; flexible environments

• Create a bunch of clusters for you and your friends (instructions)

  Bigger setup effort; ideal for group training

shared/connecting.md

For a consistent Kubernetes experience ...

• If you are using your own Kubernetes cluster, you can use shpod

• shpod provides a shell running in a pod on your own cluster

• It comes with many tools pre-installed (helm, stern...)

• These tools are used in many exercises in these slides

• shpod also gives you completion and a fancy prompt

shared/connecting.md

We will (mostly) interact with node1 only

These remarks apply only when using multiple nodes, of course.

• Unless instructed, all commands must be run from the first VM, node1

• We will only check out/copy the code on node1

• During normal operations, we do not need access to the other nodes

• If we had to troubleshoot issues, we would use a combination of:

  • SSH (to access system logs, daemon status...)

  • Docker API (to check running containers and container engine status)

shared/connecting.md

Terminals

Once in a while, the instructions will say:
"Open a new terminal."

There are multiple ways to do this:

• create a new window or tab on your machine, and SSH into the VM;

• use screen or tmux on the VM and open a new window from there.

You are welcome to use the method that you feel the most comfortable with.

shared/connecting.md

Tmux cheatsheet

Tmux is a terminal multiplexer like screen.

You don't have to use it or even know about it to follow along.
But some of us like to use it to switch between terminals.
It has been preinstalled on your workshop nodes.

• Ctrl-b c → creates a new window
• Ctrl-b n → go to next window
• Ctrl-b p → go to previous window
• Ctrl-b " → split window top/bottom
• Ctrl-b % → split window left/right
• Ctrl-b Alt-1 → rearrange windows in columns
• Ctrl-b Alt-2 → rearrange windows in rows
• Ctrl-b arrows → navigate to other windows
• Ctrl-b d → detach session
• tmux attach → reattach to session

shared/connecting.md

Our sample application

• We will clone the GitHub repository onto our node1

• The repository also contains scripts and tools that we will use through the workshop

(You can also fork the repository on GitHub and clone your fork if you prefer that.)

shared/sampleapp.md

Having a look at the application

shared/sampleapp.md

Viewing the application

• Jérôme is going to wear his developer hat ...

• ... start the application on his developer's machine ...

• ... and wait for the app to be up and running.

shared/sampleapp.md

What's this application?

DockerCoins in the microservices era

• DockerCoins is made of 5 services:

  • rng = web service generating random bytes

  • hasher = web service computing hash of POSTed data

  • worker = background process calling rng and hasher

  • webui = web interface to watch progress

  • redis = data store (holds a counter updated by worker)

• These 5 services are visible in the application's Compose file, docker-compose.yml

shared/sampleapp.md

How DockerCoins works

• worker invokes web service rng to generate random bytes

• worker invokes web service hasher to hash these bytes

• worker does this in an infinite loop

• every second, worker updates redis to indicate how many loops were done

• webui queries redis, and computes and exposes "hashing speed" in our browser

(See diagram on next slide!)

shared/sampleapp.md

Service discovery in container-land

How does each service find out the address of the other ones?

Example in worker/worker.py

redis = Redis("redis")
def get_random_bytes():
    r = requests.get("http://rng/32")
    return r.content
def hash_bytes(data):
    r = requests.post("http://hasher/",
                      data=data,
                      headers={"Content-Type": "application/octet-stream"})

(Full source code available here)

shared/sampleapp.md

Show me the code!

• You can check the GitHub repository with all the materials of this workshop:
  https://github.com/jpetazzo/container.training

• The application is in the dockercoins subdirectory

• The Compose file (docker-compose.yml) lists all 5 services

• redis is using an official image from the Docker Hub

• hasher, rng, worker, webui are each built from a Dockerfile

• Each service's Dockerfile and source code is in its own directory

  (hasher is in the hasher directory, rng is in the rng directory, etc.)

shared/sampleapp.md

Our application at work

• On the left-hand side, the "rainbow strip" shows the container names

• On the right-hand side, we see the output of our containers

• We can see the worker service making requests to rng and hasher

• For rng and hasher, we see HTTP access logs

shared/sampleapp.md

Connecting to the web UI

• "Logs are exciting and fun!" (No-one, ever)

• The webui container exposes a web dashboard; let's view it

A drawing area should show up, and after a few seconds, a blue graph will appear.

shared/sampleapp.md

Kubernetes concepts

• Kubernetes is a container management system

• It runs and manages containerized applications on a cluster

Basic things we can ask Kubernetes to do

Other things that Kubernetes can do for us

• Autoscaling

  (straightforward on CPU; more complex on other metrics)

• Ressource management and scheduling

  (reserve CPU/RAM for containers; placement constraints)

• Advanced rollout patterns

  (blue/green deployment, canary deployment)

k8s/concepts-k8s.md

More things that Kubernetes can do for us

• Batch jobs

  (one-off; parallel; also cron-style periodic execution)

• Fine-grained access control

  (defining what can be done by whom on which resources)

• Stateful services

  (databases, message queues, etc.)

• Automating complex tasks with operators

  (e.g. database replication, failover, etc.)

k8s/concepts-k8s.md

Kubernetes architecture

k8s/concepts-k8s.md

Kubernetes architecture

• Ha ha ha ha

• OK, I was trying to scare you, it's much simpler than that ❤️

k8s/concepts-k8s.md

Credits

• The first schema is a Kubernetes cluster with storage backed by multi-path iSCSI

  (Courtesy of Yongbok Kim)

• The second one is a simplified representation of a Kubernetes cluster

  (Courtesy of Imesh Gunaratne)

k8s/concepts-k8s.md

Kubernetes architecture: the nodes

• The nodes executing our containers run a collection of services:

  • a container Engine (typically Docker)

  • kubelet (the "node agent")

  • kube-proxy (a necessary but not sufficient network component)

• Nodes were formerly called "minions"

  (You might see that word in older articles or documentation)

k8s/concepts-k8s.md

Kubernetes architecture: the control plane

• The Kubernetes logic (its "brains") is a collection of services:

  • the API server (our point of entry to everything!)

  • core services like the scheduler and controller manager

  • etcd (a highly available key/value store; the "database" of Kubernetes)

• Together, these services form the control plane of our cluster

• The control plane is also called the "master"

k8s/concepts-k8s.md

Interacting with Kubernetes

• We will interact with our Kubernetes cluster through the Kubernetes API

• The Kubernetes API is (mostly) RESTful

• It allows us to create, read, update, delete resources

• A few common resource types are:

  • node (a machine — physical or virtual — in our cluster)

  • pod (group of containers running together on a node)

  • service (stable network endpoint to connect to one or multiple containers)

k8s/concepts-k8s.md

Scaling

• How would we scale the pod shown on the previous slide?

• Do create additional pods

  • each pod can be on a different node

  • each pod will have its own IP address

• Do not add more NGINX containers in the pod

  • all the NGINX containers would be on the same node

  • they would all have the same IP address
    (resulting in Address alreading in use errors)

k8s/concepts-k8s.md

Together or separate

• Should we put e.g. a web application server and a cache together?
  ("cache" being something like e.g. Memcached or Redis)

• Putting them in the same pod means:

  • they have to be scaled together

  • they can communicate very efficiently over localhost

• Putting them in different pods means:

  • they can be scaled separately

  • they must communicate over remote IP addresses
    (incurring more latency, lower performance)

• Both scenarios can make sense, depending on our goals

k8s/concepts-k8s.md

Credits

• The first diagram is courtesy of Lucas Käldström, in this presentation

  • it's one of the best Kubernetes architecture diagrams available!

• The second diagram is courtesy of Weave Works

  • a pod can have multiple containers working together

  • IP addresses are associated with pods, not with individual containers

Both diagrams used with permission.

k8s/concepts-k8s.md

First contact with kubectl

• kubectl is (almost) the only tool we'll need to talk to Kubernetes

• It is a rich CLI tool around the Kubernetes API

  (Everything you can do with kubectl, you can do directly with the API)

• On our machines, there is a ~/.kube/config file with:

  • the Kubernetes API address

  • the path to our TLS certificates used to authenticate

• You can also use the --kubeconfig flag to pass a config file

• Or directly --server, --user, etc.

• kubectl can be pronounced "Cube C T L", "Cube cuttle", "Cube cuddle"...

k8s/kubectlget.md

kubectl get

• Let's look at our Node resources with kubectl get!

k8s/kubectlget.md

Obtaining machine-readable output

• kubectl get can output JSON, YAML, or be directly formatted

k8s/kubectlget.md

(Ab)using kubectl and jq

• It's super easy to build custom reports

k8s/kubectlget.md

Type names

• The most common resource names have three forms:

  • singular (e.g. node, service, deployment)

  • plural (e.g. nodes, services, deployments)

  • short (e.g. no, svc, deploy)

• Some resources do not have a short name

• Endpoints only have a plural form

  (because even a single Endpoints resource is actually a list of endpoints)

k8s/kubectlget.md

Viewing details

• We can use kubectl get -o yaml to see all available details

• However, YAML output is often simultaneously too much and not enough

• For instance, kubectl get node node1 -o yaml is:

  • too much information (e.g.: list of images available on this node)

  • not enough information (e.g.: doesn't show pods running on this node)

  • difficult to read for a human operator

• For a comprehensive overview, we can use kubectl describe instead

k8s/kubectlget.md

kubectl describe

• kubectl describe needs a resource type and (optionally) a resource name

• It is possible to provide a resource name prefix

  (all matching objects will be displayed)

• kubectl describe will retrieve some extra information about the resource

(We should notice a bunch of control plane pods.)

k8s/kubectlget.md

Listing running containers

• Containers are manipulated through pods

• A pod is a group of containers:

  • running together (on the same node)

  • sharing resources (RAM, CPU; but also network, volumes)

Namespaces

• Namespaces allow us to segregate resources

Accessing namespaces

• By default, kubectl uses the default namespace

• We can see resources in all namespaces with --all-namespaces

Here are our system pods!

k8s/kubectlget.md

What are all these control plane pods?

• etcd is our etcd server

• kube-apiserver is the API server

• kube-controller-manager and kube-scheduler are other control plane components

• coredns provides DNS-based service discovery (replacing kube-dns as of 1.11)

• kube-proxy is the (per-node) component managing port mappings and such

• weave is the (per-node) component managing the network overlay

• the READY column indicates the number of containers in each pod

  (1 for most pods, but weave has 2, for instance)

k8s/kubectlget.md

Scoping another namespace

• We can also look at a different namespace (other than default)

k8s/kubectlget.md

Namespaces and other kubectl commands

• We can use -n/--namespace with almost every kubectl command

• Example:

  • kubectl create --namespace=X to create something in namespace X

• We can use -A/--all-namespaces with most commands that manipulate multiple objects

• Examples:

  • kubectl delete can delete resources across multiple namespaces

  • kubectl label can add/remove/update labels across multiple namespaces

k8s/kubectlget.md

Services

• A service is a stable endpoint to connect to "something"

  (In the initial proposal, they were called "portals")

ClusterIP services

• A ClusterIP service is internal, available from the cluster only

• This is useful for introspection from within containers

The command above should either time out, or show an authentication error. Why?

k8s/kubectlget.md

Time out

• Connections to ClusterIP services only work from within the cluster

• If we are outside the cluster, the curl command will probably time out

  (Because the IP address, e.g. 10.96.0.1, isn't routed properly outside the cluster)

• This is the case with most "real" Kubernetes clusters

• To try the connection from within the cluster, we can use shpod

k8s/kubectlget.md

Authentication error

This is what we should see when connecting from within the cluster:

$ curl -k https://10.96.0.1
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
  },
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
  "reason": "Forbidden",
  "details": {
  },
  "code": 403
}

k8s/kubectlget.md

Explanations

• We can see kind, apiVersion, metadata

• These are typical of a Kubernetes API reply

• Because we are talking to the Kubernetes API

• The Kubernetes API tells us "Forbidden"

  (because it requires authentication)

• The Kubernetes API is reachable from within the cluster

  (many apps integrating with Kubernetes will use this)

k8s/kubectlget.md

DNS integration

• Each service also gets a DNS record

• The Kubernetes DNS resolver is available from within pods

  (and sometimes, from within nodes, depending on configuration)

• Code running in pods can connect to services using their name

  (e.g. https://kubernetes/...)

k8s/kubectlget.md

Running our first containers on Kubernetes

• First things first: we cannot run a container

Starting a simple pod with kubectl run

• We need to specify at least a name and the image we want to use

Behind the scenes of kubectl run

• Let's look at the resources that were created by kubectl run

What are these different things?

• A deployment is a high-level construct

  • allows scaling, rolling updates, rollbacks

  • multiple deployments can be used together to implement a canary deployment

  • delegates pods management to replica sets

• A replica set is a low-level construct

  • makes sure that a given number of identical pods are running

  • allows scaling

  • rarely used directly

• A replication controller is the (deprecated) predecessor of a replica set

k8s/kubectlrun.md

Our pingpong deployment

• kubectl run created a deployment, deployment.apps/pingpong

NAME                       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/pingpong   1         1         1            1           10m

• That deployment created a replica set, replicaset.apps/pingpong-xxxxxxxxxx

NAME                                  DESIRED   CURRENT   READY     AGE
replicaset.apps/pingpong-7c8bbcd9bc   1         1         1         10m

• That replica set created a pod, pod/pingpong-xxxxxxxxxx-yyyyy

NAME                            READY     STATUS    RESTARTS   AGE
pod/pingpong-7c8bbcd9bc-6c9qz   1/1       Running   0          10m

• We'll see later how these folks play together for:

  • scaling, high availability, rolling updates

k8s/kubectlrun.md

Viewing container output

• Let's use the kubectl logs command

• We will pass either a pod name, or a type/name

  (E.g. if we specify a deployment or replica set, it will get the first pod in it)

• Unless specified otherwise, it will only show logs of the first container in the pod

  (Good thing there's only one in ours!)

k8s/kubectlrun.md

Streaming logs in real time

• Just like docker logs, kubectl logs supports convenient options:

  • -f/--follow to stream logs in real time (à la tail -f)

  • --tail to indicate how many lines you want to see (from the end)

  • --since to get logs only after a given timestamp

k8s/kubectlrun.md

Scaling our application

• We can create additional copies of our container (I mean, our pod) with kubectl scale

Note: what if we tried to scale replicaset.apps/pingpong-xxxxxxxxxx?

We could! But the deployment would notice it right away, and scale back to the initial level.

k8s/kubectlrun.md

Log streaming

• Let's look again at the output of kubectl logs

  (the one we started before scaling up)

• kubectl logs shows us one line per second

• We could expect 3 lines per second

  (since we should now have 3 pods running ping)

• Let's try to figure out what's happening!

k8s/kubectlrun.md

Streaming logs of multiple pods

• What happens if we restart kubectl logs?

kubectl logs will warn us that multiple pods were found, and that it's showing us only one of them.

Let's leave kubectl logs running while we keep exploring.

k8s/kubectlrun.md

Resilience

• The deployment pingpong watches its replica set

• The replica set ensures that the right number of pods are running

• What happens if pods disappear?

k8s/kubectlrun.md

What happened?

• kubectl delete pod terminates the pod gracefully

  (sending it the TERM signal and waiting for it to shutdown)

• As soon as the pod is in "Terminating" state, the Replica Set replaces it

• But we can still see the output of the "Terminating" pod in kubectl logs

• Until 30 seconds later, when the grace period expires

• The pod is then killed, and kubectl logs exits

k8s/kubectlrun.md

What if we wanted something different?

• What if we wanted to start a "one-shot" container that doesn't get restarted?

• We could use kubectl run --restart=OnFailure or kubectl run --restart=Never

• These commands would create jobs or pods instead of deployments

• Under the hood, kubectl run invokes "generators" to create resource descriptions

• We could also write these resource descriptions ourselves (typically in YAML),
  and create them on the cluster with kubectl apply -f (discussed later)

• With kubectl run --schedule=..., we can also create cronjobs

k8s/kubectlrun.md

Scheduling periodic background work

• A Cron Job is a job that will be executed at specific intervals

  (the name comes from the traditional cronjobs executed by the UNIX crond)

• It requires a schedule, represented as five space-separated fields:

  • minute [0,59]
  • hour [0,23]
  • day of the month [1,31]
  • month of the year [1,12]
  • day of the week ([0,6] with 0=Sunday)

• * means "all valid values"; /N means "every N"

• Example: */3 * * * * means "every three minutes"

k8s/kubectlrun.md

Creating a Cron Job

• Let's create a simple job to be executed every three minutes

• Cron Jobs need to terminate, otherwise they'd run forever

k8s/kubectlrun.md

Cron Jobs in action

• At the specified schedule, the Cron Job will create a Job

• The Job will create a Pod

• The Job will make sure that the Pod completes

  (re-creating another one if it fails, for instance if its node fails)

(It will take a few minutes before the first job is scheduled.)

k8s/kubectlrun.md

What about that deprecation warning?

• As we can see from the previous slide, kubectl run can do many things

• The exact type of resource created is not obvious

• To make things more explicit, it is better to use kubectl create:

  • kubectl create deployment to create a deployment

  • kubectl create job to create a job

  • kubectl create cronjob to run a job periodically
    (since Kubernetes 1.14)

• Eventually, kubectl run will be used only to start one-shot pods

  (see https://github.com/kubernetes/kubernetes/pull/68132)

k8s/kubectlrun.md

Various ways of creating resources

• kubectl run

  • easy way to get started
  • versatile

• kubectl create <resource>

  • explicit, but lacks some features
  • can't create a CronJob before Kubernetes 1.14
  • can't pass command-line arguments to deployments

• kubectl create -f foo.yaml or kubectl apply -f foo.yaml

  • all features are available
  • requires writing YAML

k8s/kubectlrun.md

Viewing logs of multiple pods

• When we specify a deployment name, only one single pod's logs are shown

• We can view the logs of multiple pods by specifying a selector

• A selector is a logic expression using labels

• Conveniently, when you kubectl run somename, the associated objects have a run=somename label

k8s/kubectlrun.md

Streaming logs of multiple pods

• Can we stream the logs of all our pingpong pods?

Note: combining -l and -f is only possible since Kubernetes 1.14!

Let's try to understand why ...

k8s/kubectlrun.md

Shortcomings of kubectl logs

• We don't see which pod sent which log line

• If pods are restarted / replaced, the log stream stops

• If new pods are added, we don't see their logs

• To stream the logs of multiple pods, we need to write a selector

• There are external tools to address these shortcomings

  (e.g.: Stern)

k8s/kubectlrun.md

Aren't we flooding 1.1.1.1?

• If you're wondering this, good question!

• Don't worry, though:

  APNIC's research group held the IP addresses 1.1.1.1 and 1.0.0.1. While the addresses were valid, so many people had entered them into various random systems that they were continuously overwhelmed by a flood of garbage traffic. APNIC wanted to study this garbage traffic but any time they'd tried to announce the IPs, the flood would overwhelm any conventional network.

  (Source: https://blog.cloudflare.com/announcing-1111/)

• It's very unlikely that our concerted pings manage to produce even a modest blip at Cloudflare's NOC!

k8s/kubectlrun.md

Accessing logs from the CLI

• The kubectl logs command has limitations:

  • it cannot stream logs from multiple pods at a time

  • when showing logs from multiple pods, it mixes them all together

• We are going to see how to do it better

k8s/logs-cli.md

Doing it manually

• We could (if we were so inclined) write a program or script that would:

  • take a selector as an argument

  • enumerate all pods matching that selector (with kubectl get -l ...)

  • fork one kubectl logs --follow ... command per container

  • annotate the logs (the output of each kubectl logs ... process) with their origin

  • preserve ordering by using kubectl logs --timestamps ... and merge the output

Stern

Stern is an open source project by Wercker.

From the README:

Stern allows you to tail multiple pods on Kubernetes and multiple containers within the pod. Each result is color coded for quicker debugging.

The query is a regular expression so the pod name can easily be filtered and you don't need to specify the exact id (for instance omitting the deployment id). If a pod is deleted it gets removed from tail and if a new pod is added it automatically gets tailed.

Exactly what we need!

k8s/logs-cli.md

Installing Stern

• Run stern (without arguments) to check if it's installed:

  $ stern
  Tail multiple pods and containers from Kubernetes
  Usage:
  stern pod-query [flags]

• If it is not installed, the easiest method is to download a binary release

• The following commands will install Stern on a Linux Intel 64 bit machine:

  sudo curl -L -o /usr/local/bin/stern \
     https://github.com/wercker/stern/releases/download/1.11.0/stern_linux_amd64
  sudo chmod +x /usr/local/bin/stern

• On OS X, just brew install stern

k8s/logs-cli.md

Using Stern

• There are two ways to specify the pods whose logs we want to see:

  • -l followed by a selector expression (like with many kubectl commands)

  • with a "pod query," i.e. a regex used to match pod names

• These two ways can be combined if necessary

k8s/logs-cli.md

Stern convenient options

• The --tail N flag shows the last N lines for each container

  (Instead of showing the logs since the creation of the container)

• The -t / --timestamps flag shows timestamps

• The --all-namespaces flag is self-explanatory

k8s/logs-cli.md

Using Stern with a selector

• When specifying a selector, we can omit the value for a label

• This will match all objects having that label (regardless of the value)

• Everything created with kubectl run has a label run

• We can use that property to view the logs of all the pods created with kubectl run

• Similarly, everything created with kubectl create deployment has a label app

k8s/logs-cli.md

vRLI

Centralize logs

• Compatible with syslog

• Query language

• Dashboards

• High ingest capacity

vmware/vrli.md

Declarative vs imperative

• Our container orchestrator puts a very strong emphasis on being declarative

• Declarative:

  I would like a cup of tea.

• Imperative:

  Boil some water. Pour it in a teapot. Add tea leaves. Steep for a while. Serve in a cup.

Declarative vs imperative

• What declarative would really be:

  I want a cup of tea, obtained by pouring an infusion¹ of tea leaves in a cup.

Declarative vs imperative

• Imperative systems:

  • simpler

  • if a task is interrupted, we have to restart from scratch

• Declarative systems:

  • if a task is interrupted (or if we show up to the party half-way through), we can figure out what's missing and do only what's necessary

  • we need to be able to observe the system

  • ... and compute a "diff" between what we have and what we want

shared/declarative.md

Declarative vs imperative in Kubernetes

• With Kubernetes, we cannot say: "run this container"

• All we can do is write a spec and push it to the API server

  (by creating a resource like e.g. a Pod or a Deployment)

• The API server will validate that spec (and reject it if it's invalid)

• Then it will store it in etcd

• A controller will "notice" that spec and act upon it

k8s/declarative.md

Reconciling state

• Watch for the spec fields in the YAML files later!

• The spec describes how we want the thing to be

• Kubernetes will reconcile the current state with the spec
  (technically, this is done by a number of controllers)

• When we want to change some resource, we update the spec

• Kubernetes will then converge that resource

k8s/declarative.md

19,000 words

They say, "a picture is worth one thousand words."

The following 19 slides show what really happens when we run:

kubectl run web --image=nginx --replicas=3

k8s/deploymentslideshow.md

Kubernetes network model

• TL,DR:

  Our cluster (nodes and pods) is one big flat IP network.

Kubernetes network model: the good

• Everything can reach everything

• No address translation

• No port translation

• No new protocol

• The network implementation can decide how to allocate addresses

• IP addresses don't have to be "portable" from a node to another

  (We can use e.g. a subnet per node and use a simple routed topology)

• The specification is simple enough to allow many various implementations

k8s/kubenet.md

Kubernetes network model: the less good

• Everything can reach everything

  • if you want security, you need to add network policies

  • the network implementation that you use needs to support them

• There are literally dozens of implementations out there

  (15 are listed in the Kubernetes documentation)

• Pods have level 3 (IP) connectivity, but services are level 4 (TCP or UDP)

  (Services map to a single UDP or TCP port; no port ranges or arbitrary IP packets)

• kube-proxy is on the data path when connecting to a pod or container,
  and it's not particularly fast (relies on userland proxying or iptables)

k8s/kubenet.md

Kubernetes network model: in practice

• The nodes that we are using have been set up to use Weave

• We don't endorse Weave in a particular way, it just Works For Us

• Don't worry about the warning about kube-proxy performance

• Unless you:

  • routinely saturate 10G network interfaces
  • count packet rates in millions per second
  • run high-traffic VOIP or gaming platforms
  • do weird things that involve millions of simultaneous connections
    (in which case you're already familiar with kernel tuning)

• If necessary, there are alternatives to kube-proxy; e.g. kube-router

k8s/kubenet.md

Exposing containers

• kubectl expose creates a service for existing pods

• A service is a stable address for a pod (or a bunch of pods)

• If we want to connect to our pod(s), we need to create a service

• Once a service is created, CoreDNS will allow us to resolve it by name

  (i.e. after creating service hello, the name hello will resolve to something)

• There are different types of services, detailed on the following slides:

  ClusterIP, NodePort, LoadBalancer, ExternalName

• HTTP services can also use Ingress resources (more on that later)

k8s/kubectlexpose.md

ClusterIP

• It's the default service type

• A virtual IP address is allocated for the service

  (in an internal, private range; e.g. 10.96.0.0/12)

• This IP address is reachable only from within the cluster (nodes and pods)

• Our code can connect to the service using the original port number

• Perfect for internal communication, within the cluster

k8s/kubectlexpose.md

LoadBalancer

• An external load balancer is allocated for the service

  (typically a cloud load balancer, e.g. ELB on AWS, GLB on GCE ...)

• This is available only when the underlying infrastructure provides some kind of "load balancer as a service"

• Each service of that type will typically cost a little bit of money

  (e.g. a few cents per hour on AWS or GCE)

• Ideally, traffic would flow directly from the load balancer to the pods

• In practice, it will often flow through a NodePort first

k8s/kubectlexpose.md

NodePort

• A port number is allocated for the service

  (by default, in the 30000-32768 range)

• That port is made available on all our nodes and anybody can connect to it

  (we can connect to any node on that port to reach the service)

• Our code needs to be changed to connect to that new port number

• Under the hood: kube-proxy sets up a bunch of iptables rules on our nodes

• Sometimes, it's the only available option for external traffic

  (e.g. most clusters deployed with kubeadm or on-premises)

k8s/kubectlexpose.md

Running containers with open ports

• Since ping doesn't have anything to connect to, we'll have to run something else

• We could use the nginx official image, but ...

  ... we wouldn't be able to tell the backends from each other!

• We are going to use jpetazzo/httpenv, a tiny HTTP server written in Go

• jpetazzo/httpenv listens on port 8888

• It serves its environment variables in JSON format

• The environment variables will include HOSTNAME, which will be the pod name

  (and therefore, will be different on each backend)

k8s/kubectlexpose.md

Creating a deployment for our HTTP server

• We could do kubectl run httpenv --image=jpetazzo/httpenv ...

• But since kubectl run is being deprecated, let's see how to use kubectl create instead

k8s/kubectlexpose.md

Exposing our deployment

• We'll create a default ClusterIP service

k8s/kubectlexpose.md

Services are layer 4 constructs

• You can assign IP addresses to services, but they are still layer 4

  (i.e. a service is not an IP address; it's an IP address + protocol + port)

• This is caused by the current implementation of kube-proxy

  (it relies on mechanisms that don't support layer 3)

• As a result: you have to indicate the port number for your service

• Running services with arbitrary port (or port ranges) requires hacks

  (e.g. host networking mode)

k8s/kubectlexpose.md

Testing our service

• We will now send a few HTTP requests to our pods

Ingress

• Ingresses are another type (kind) of resource

• They are specifically for HTTP services

  (not TCP or UDP)

• They can also handle TLS certificates, URL rewriting ...

• They require an Ingress Controller to function

k8s/kubectlexpose.md

NSX-T

Connect and secure Kubernetes Pods

• Distributed firewall and micro-segmentation for VMs and Pods

• Ingress and LoadBalancer Controller for Kubernetes

• Traceflow for Pods and dynamic routing

vmware/nsxt.md

Shipping images with a registry

• Initially, our app was running on a single node

• We could build and run in the same place

• Therefore, we did not need to ship anything

• Now that we want to run on a cluster, things are different

• The easiest way to ship container images is to use a registry

k8s/shippingimages.md

How Docker registries work (a reminder)

• What happens when we execute docker run alpine ?

• If the Engine needs to pull the alpine image, it expands it into library/alpine

• library/alpine is expanded into index.docker.io/library/alpine

• The Engine communicates with index.docker.io to retrieve library/alpine:latest

• To use something else than index.docker.io, we specify it in the image name

• Examples:

  docker pull gcr.io/google-containers/alpine-with-bash:1.0
  docker build -t registry.mycompany.io:5000/myimage:awesome .
  docker push registry.mycompany.io:5000/myimage:awesome

k8s/shippingimages.md

Running DockerCoins on Kubernetes

• Create one deployment for each component

  (hasher, redis, rng, webui, worker)

• Expose deployments that need to accept connections

  (hasher, redis, rng, webui)

• For redis, we can use the official redis image

• For the 4 others, we need to build images and push them to some registry

k8s/shippingimages.md

Building and shipping images

• There are many options!

• Manually:

  • build locally (with docker build or otherwise)

  • push to the registry

• Automatically:

  • build and test locally

  • when ready, commit and push a code repository

  • the code repository notifies an automated build system

  • that system gets the code, builds it, pushes the image to the registry

k8s/shippingimages.md

Which registry do we want to use?

• There are SAAS products like Docker Hub, Quay ...

• Each major cloud provider has an option as well

  (ACR on Azure, ECR on AWS, GCR on Google Cloud...)

• There are also commercial products to run our own registry

  (Docker EE, Quay...)

• And open source options, too!

• When picking a registry, pay attention to its build system

  (when it has one)

k8s/shippingimages.md

Using images from the Docker Hub

• For everyone's convenience, we took care of building DockerCoins images

• We pushed these images to the DockerHub, under the dockercoins user

• These images are tagged with a version number, v0.1

• The full image names are therefore:

  • dockercoins/hasher:v0.1

  • dockercoins/rng:v0.1

  • dockercoins/webui:v0.1

  • dockercoins/worker:v0.1

k8s/buildshiprun-dockerhub.md

Running our application on Kubernetes

• We can now deploy our code (as well as a redis instance)

k8s/ourapponkube.md

Is this working?

• After waiting for the deployment to complete, let's look at the logs!

  (Hint: use kubectl get deploy -w to watch deployment events)

Connecting containers together

• Three deployments need to be reachable by others: hasher, redis, rng

• worker doesn't need to be exposed

• webui will be dealt with later

k8s/ourapponkube.md

Is this working yet?

• The worker has an infinite loop, that retries 10 seconds after an error

Exposing services for external access

• Now we would like to access the Web UI

• We will expose it with a NodePort

  (just like we did for the registry)

On PKS, replace NodePort with LoadBalancer.

k8s/ourapponkube.md

Accessing the web UI

• We can now connect to any node, on the allocated node port, to view the web UI

Deploying with YAML

• So far, we created resources with the following commands:

  • kubectl run

  • kubectl create deployment

  • kubectl expose

• We can also create resources directly with YAML manifests

k8s/yamldeploy.md

kubectl apply vs create

• kubectl create -f whatever.yaml

  • creates resources if they don't exist

  • if resources already exist, don't alter them
    (and display error message)

• kubectl apply -f whatever.yaml

  • creates resources if they don't exist

  • if resources already exist, update them
    (to match the definition provided by the YAML file)

  • stores the manifest as an annotation in the resource

k8s/yamldeploy.md

Creating multiple resources

• The manifest can contain multiple resources separated by ---

 kind: ...
 apiVersion: ...
 metadata: ...
   name: ...
 ...
 ---
 kind: ...
 apiVersion: ...
 metadata: ...
   name: ...
 ...

k8s/yamldeploy.md

Creating multiple resources

• The manifest can also contain a list of resources

 apiVersion: v1
 kind: List
 items:
 - kind: ...
   apiVersion: ...
   ...
 - kind: ...
   apiVersion: ...
   ...

k8s/yamldeploy.md

Deploying dockercoins with YAML

• We provide a YAML manifest with all the resources for Dockercoins

  (Deployments and Services)

• We can use it if we need to deploy or redeploy Dockercoins

(If we deployed Dockercoins earlier, we will see warning messages, because the resources that we created lack the necessary annotation. We can safely ignore them.)

k8s/yamldeploy.md

Setting up Kubernetes

• How did we set up these Kubernetes clusters that we're using?

kubeadm drawbacks

• Doesn't set up Docker or any other container engine

• Doesn't set up the overlay network

• Doesn't set up multi-master (no high availability)

Other deployment options

• AKS: managed Kubernetes on Azure

• GKE: managed Kubernetes on Google Cloud

• EKS, eksctl: managed Kubernetes on AWS

• kops: customizable deployments on AWS, Digital Ocean, GCE (beta), vSphere (alpha)

• minikube, kubespawn, Docker Desktop, kind: for local development

• kubicorn, the Cluster API: deploy your clusters declaratively, "the Kubernetes way"

k8s/setup-k8s.md

Even more deployment options

• If you like Ansible: kubespray

• If you like Terraform: typhoon

• If you like Terraform and Puppet: tarmak

• You can also learn how to install every component manually, with the excellent tutorial Kubernetes The Hard Way

  Kubernetes The Hard Way is optimized for learning, which means taking the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.

• There are also many commercial options available!

• For a longer list, check the Kubernetes documentation:
  it has a great guide to pick the right solution to set up Kubernetes.

k8s/setup-k8s.md

PKS

Automate and streamline Kubernetes cluster deployment and operations

• Fully automated installation of mainstream Kubernetes

• Scale up, scale down & upgrade clusters

• Highly-available control plane & self-healing features

  (replace nodes automatically when needed and deploy CVE patches)

• Integration with VMware SDDC (Software Defined Data Center) features

  (e.g. vMotion, DRS, Shared Datastore, NSX-T, vREALIZE Suite)

vmware/pks.md

Scaling our demo app

• Our ultimate goal is to get more DockerCoins

  (i.e. increase the number of loops per second shown on the web UI)

• Let's look at the architecture again:

  

• The loop is done in the worker; perhaps we could try adding more workers?

k8s/scalingdockercoins.md

Adding another worker

• All we have to do is scale the worker Deployment

After a few seconds, the graph in the web UI should show up.

k8s/scalingdockercoins.md

Adding more workers

• If 2 workers give us 2x speed, what about 3 workers?

The graph in the web UI should go up again.

(This is looking great! We're gonna be RICH!)

k8s/scalingdockercoins.md

Adding even more workers

• Let's see if 10 workers give us 10x speed!

Why are we stuck at 10 hashes per second?

• If this was high-quality, production code, we would have instrumentation

  (Datadog, Honeycomb, New Relic, statsd, Sumologic, ...)

• It's not!

• Perhaps we could benchmark our web services?

  (with tools like ab, or even simpler, httping)

k8s/scalingdockercoins.md

Benchmarking our web services

• We want to check hasher and rng

• We are going to use httping

• It's just like ping, but using HTTP GET requests

  (it measures how long it takes to perform one GET request)

• It's used like this:

  httping [-c count] http://host:port/path

• Or even simpler:

  httping ip.ad.dr.ess

• We will use httping on the ClusterIP addresses of our services

k8s/scalingdockercoins.md

Obtaining ClusterIP addresses

• We can simply check the output of kubectl get services

• Or do it programmatically, as in the example below

Now we can access the IP addresses of our services through $HASHER and $RNG.

k8s/scalingdockercoins.md

Checking hasher and rng response times

• hasher is fine (it should take a few milliseconds to reply)

• rng is not (it should take about 700 milliseconds if there are 10 workers)

• Something is wrong with rng, but ... what?

k8s/scalingdockercoins.md

Scaling rng

• Let's scale the rng service just like we scaled worker

The web UI graph should go past 10 hashes/second.

kube-fullday.yml

vROPS

Manage Kubernetes and/or PKS clusters

• Automatically add new PKS clusters after deployment

• Supervision

• Capacity management

• Global view of infrastructure

vmware/vrops.md

Rolling updates

• By default (without rolling updates), when a scaled resource is updated:

  • new pods are created

  • old pods are terminated

  • ... all at the same time

  • if something goes wrong, ¯\_(ツ)_/¯

k8s/rollout.md

Rolling updates

• With rolling updates, when a Deployment is updated, it happens progressively

• The Deployment controls multiple Replica Sets

• Each Replica Set is a group of identical Pods

  (with the same image, arguments, parameters ...)

• During the rolling update, we have at least two Replica Sets:

  • the "new" set (corresponding to the "target" version)

  • at least one "old" set

• We can have multiple "old" sets

  (if we start another update before the first one is done)

k8s/rollout.md

Update strategy

• Two parameters determine the pace of the rollout: maxUnavailable and maxSurge

• They can be specified in absolute number of pods, or percentage of the replicas count

• At any given time ...

  • there will always be at least replicas-maxUnavailable pods available

  • there will never be more than replicas+maxSurge pods in total

  • there will therefore be up to maxUnavailable+maxSurge pods being updated

• We have the possibility of rolling back to the previous version
  (if the update fails or is unsatisfactory in any way)

k8s/rollout.md

Checking current rollout parameters

• Recall how we build custom reports with kubectl and jq:

k8s/rollout.md

Rolling updates in practice

• As of Kubernetes 1.8, we can do rolling updates with:

  deployments, daemonsets, statefulsets

• Editing one of these resources will automatically result in a rolling update

• Rolling updates can be monitored with the kubectl rollout subcommand

k8s/rollout.md

Rolling out the new worker service

Give it some time

• At first, it looks like nothing is happening (the graph remains at the same level)

• According to kubectl get deploy -w, the deployment was updated really quickly

• But kubectl get pods -w tells a different story

• The old pods are still here, and they stay in Terminating state for a while

• Eventually, they are terminated; and then the graph decreases significantly

• This delay is due to the fact that our worker doesn't handle signals

• Kubernetes sends a "polite" shutdown request to the worker, which ignores it

• After a grace period, Kubernetes gets impatient and kills the container

  (The grace period is 30 seconds, but can be changed if needed)

k8s/rollout.md

Rolling out something invalid

• What happens if we make a mistake?

What's going on with our rollout?

• Why is our app a bit slower?

• Because MaxUnavailable=25%

  ... So the rollout terminated 2 replicas out of 10 available

• Okay, but why do we see 5 new replicas being rolled out?

• Because MaxSurge=25%

  ... So in addition to replacing 2 replicas, the rollout is also starting 3 more

• It rounded down the number of MaxUnavailable pods conservatively,
  but the total number of pods being rolled out is allowed to be 25+25=50%

k8s/rollout.md

Checking the dashboard during the bad rollout

If you didn't deploy the Kubernetes dashboard earlier, just skip this slide.

k8s/rollout.md

Recovering from a bad rollout

• We could push some v0.3 image

  (the pod retry logic will eventually catch it and the rollout will proceed)

• Or we could invoke a manual rollback

k8s/rollout.md

Rolling back to an older version

• We reverted to v0.2

• But this version still has a performance problem

• How can we get back to the previous version?

k8s/rollout.md

Multiple "undos"

• What happens if we try kubectl rollout undo again?

🤔 That didn't work.

k8s/rollout.md

Multiple "undos" don't work

• If we see successive versions as a stack:

  • kubectl rollout undo doesn't "pop" the last element from the stack

  • it copies the N-1th element to the top

• Multiple "undos" just swap back and forth between the last two versions!

k8s/rollout.md

In this specific scenario

• Our version numbers are easy to guess

• What if we had used git hashes?

• What if we had changed other parameters in the Pod spec?

k8s/rollout.md

Listing versions

• We can list successive versions of a Deployment with kubectl rollout history

We don't see all revisions.

We might see something like 1, 4, 5.

(Depending on how many "undos" we did before.)

k8s/rollout.md

Explaining deployment revisions

• These revisions correspond to our Replica Sets

• This information is stored in the Replica Set annotations

k8s/rollout.md

Rolling back to an older version

• kubectl rollout undo can work with a revision number

k8s/rollout.md

Namespaces

• We would like to deploy another copy of DockerCoins on our cluster

• We could rename all our deployments and services:

  hasher → hasher2, redis → redis2, rng → rng2, etc.

• That would require updating the code

• There has to be a better way!

Identifying a resource

• We cannot have two resources with the same name

  (or can we...?)

Uniquely identifying a resource

• For namespaced resources:

  the tuple (kind, name, namespace) needs to be unique

• For resources at the cluster scope:

  the tuple (kind, name) needs to be unique

k8s/namespaces.md

Pre-existing namespaces

• If we deploy a cluster with kubeadm, we have three or four namespaces:

  • default (for our applications)

  • kube-system (for the control plane)

  • kube-public (contains one ConfigMap for cluster discovery)

  • kube-node-lease (in Kubernetes 1.14 and later; contains Lease objects)

• If we deploy differently, we may have different namespaces

k8s/namespaces.md

Creating namespaces

• Let's see two identical methods to create a namespace

• Some tools like Helm will create namespaces automatically when needed

k8s/namespaces.md

Using namespaces

• We can pass a -n or --namespace flag to most kubectl commands:

  kubectl -n blue get svc

• We can also change our current context

• A context is a (user, cluster, namespace) tuple

• We can manipulate contexts with the kubectl config command

k8s/namespaces.md

Viewing existing contexts

• On our training environments, at this point, there should be only one context

• The current context (the only one!) is tagged with a *

• What are NAME, CLUSTER, AUTHINFO, and NAMESPACE?

k8s/namespaces.md

What's in a context

• NAME is an arbitrary string to identify the context

• CLUSTER is a reference to a cluster

  (i.e. API endpoint URL, and optional certificate)

• AUTHINFO is a reference to the authentication information to use

  (i.e. a TLS client certificate, token, or otherwise)

• NAMESPACE is the namespace

  (empty string = default)

k8s/namespaces.md

Switching contexts

• We want to use a different namespace

• Solution 1: update the current context

  This is appropriate if we need to change just one thing (e.g. namespace or authentication).

• Solution 2: create a new context and switch to it

  This is appropriate if we need to change multiple things and switch back and forth.

• Let's go with solution 1!

k8s/namespaces.md

Updating a context

• This is done through kubectl config set-context

• We can update a context by passing its name, or the current context with --current

k8s/namespaces.md

Using our new namespace

• Let's check that we are in our new namespace, then deploy a new copy of Dockercoins

k8s/namespaces.md

Deploying DockerCoins with YAML files

• The GitHub repository jpetazzo/kubercoins contains everything we need!

If the argument behind -f is a directory, all the files in that directory are processed.

The subdirectories are not processed, unless we also add the -R flag.

k8s/namespaces.md

Viewing the deployed app

• Let's see if this worked correctly!

If the graph shows up but stays at zero, give it a minute or two!

k8s/namespaces.md

Namespaces and isolation

• Namespaces do not provide isolation

• A pod in the green namespace can communicate with a pod in the blue namespace

• A pod in the default namespace can communicate with a pod in the kube-system namespace

• CoreDNS uses a different subdomain for each namespace

• Example: from any pod in the cluster, you can connect to the Kubernetes API with:

  https://kubernetes.default.svc.cluster.local:443/

k8s/namespaces.md

Isolating pods

• Actual isolation is implemented with network policies

• Network policies are resources (like deployments, services, namespaces...)

• Network policies specify which flows are allowed:

  • between pods

  • from pods to the outside world

  • and vice-versa

k8s/namespaces.md

Switch back to the default namespace

• Let's make sure that we don't run future exercises in the blue namespace

Note: we could have used --namespace=default for the same result.

k8s/namespaces.md

Switching namespaces more easily

• We can also use a little helper tool called kubens:

  # Switch to namespace foo
  kubens foo
  # Switch back to the previous namespace
  kubens -

• On our clusters, kubens is called kns instead

  (so that it's even fewer keystrokes to switch namespaces)

k8s/namespaces.md

kubens and kubectx

• With kubens, we can switch quickly between namespaces

• With kubectx, we can switch quickly between contexts

• Both tools are simple shell scripts available from https://github.com/ahmetb/kubectx

• On our clusters, they are installed as kns and kctx

  (for brevity and to avoid completion clashes between kubectx and kubectl)

k8s/namespaces.md

kube-ps1

• It's easy to lose track of our current cluster / context / namespace

• kube-ps1 makes it easy to track these, by showing them in our shell prompt

• It's a simple shell script available from https://github.com/jonmosco/kube-ps1

• On our clusters, kube-ps1 is installed and included in PS1:

  [123.45.67.89] (kubernetes-admin@kubernetes:default) docker@node1 ~

  (The highlighted part is context:namespace, managed by kube-ps1)

• Highly recommended if you work across multiple contexts or namespaces!

k8s/namespaces.md

Volumes

• Volumes are special directories that are mounted in containers

• Volumes can have many different purposes:

  • share files and directories between containers running on the same machine

  • share files and directories between containers and their host

  • centralize configuration information in Kubernetes and expose it to containers

  • manage credentials and secrets and expose them securely to containers

  • store persistent data for stateful services

  • access storage systems (like Ceph, EBS, NFS, Portworx, and many others)

k8s/volumes.md

Volumes ≠ Persistent Volumes

• Volumes and Persistent Volumes are related, but very different!

• Volumes:

  • appear in Pod specifications (see next slide)

  • do not exist as API resources (cannot do kubectl get volumes)

• Persistent Volumes:

  • are API resources (can do kubectl get persistentvolumes)

  • correspond to concrete volumes (e.g. on a SAN, EBS, etc.)

  • cannot be associated with a Pod directly; but through a Persistent Volume Claim

  • won't be discussed further in this section

k8s/volumes.md

Adding a volume to a Pod

• We will start with the simplest Pod manifest we can find

• We will add a volume to that Pod manifest

• We will mount that volume in a container in the Pod

• By default, this volume will be an emptyDir

  (an empty directory)

• It will "shadow" the directory where it's mounted

k8s/volumes.md

Our basic Pod

apiVersion: v1
kind: Pod
metadata:
  name: nginx-without-volume
spec:
  containers:
  - name: nginx
    image: nginx

This is a MVP! (Minimum Viable Pod😉)

It runs a single NGINX container.

k8s/volumes.md

Trying the basic pod

(We should see the "Welcome to NGINX" page.)

k8s/volumes.md

Adding a volume

• We need to add the volume in two places:

  • at the Pod level (to declare the volume)

  • at the container level (to mount the volume)

• We will declare a volume named www

• No type is specified, so it will default to emptyDir

  (as the name implies, it will be initialized as an empty directory at pod creation)

• In that pod, there is also a container named nginx

• That container mounts the volume www to path /usr/share/nginx/html/

k8s/volumes.md

The Pod with a volume

apiVersion: v1
kind: Pod
metadata:
  name: nginx-with-volume
spec:
  volumes:
  - name: www
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: www
      mountPath: /usr/share/nginx/html/

k8s/volumes.md

Trying the Pod with a volume

(We should now see a "403 Forbidden" error page.)

k8s/volumes.md

Populating the volume with another container

• Let's add another container to the Pod

• Let's mount the volume in both containers

• That container will populate the volume with static files

• NGINX will then serve these static files

• To populate the volume, we will clone the Spoon-Knife repository

  • this repository is https://github.com/octocat/Spoon-Knife

  • it's very popular (more than 100K stars!)

k8s/volumes.md

Sharing a volume between two containers

apiVersion: v1
kind: Pod
metadata:
  name: nginx-with-git
spec:
  volumes:
  - name: www
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: www
      mountPath: /usr/share/nginx/html/
  - name: git
    image: alpine
    command: [ "sh", "-c", "apk add --no-cache git && git clone https://github.com/octocat/Spoon-Knife /www" ]
    volumeMounts:
    - name: www
      mountPath: /www/
  restartPolicy: OnFailure

k8s/volumes.md

Sharing a volume, explained

• We added another container to the pod

• That container mounts the www volume on a different path (/www)

• It uses the alpine image

• When started, it installs git and clones the octocat/Spoon-Knife repository

  (that repository contains a tiny HTML website)

• As a result, NGINX now serves this website

k8s/volumes.md

Trying the shared volume

• This one will be time-sensitive!

• We need to catch the Pod IP address as soon as it's created

• Then send a request to it as fast as possible