Difficulty: Introduction
Estimated Time: 15 minutes

Chaos Engineering is the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production. -- Principles of Chaos Engineering

There are several frameworks that you can add to Kubernetes to provide chaos testing. However, creating a simple chaos tester does not mean it has to be complicated. With just a few tricks a simple chaos tester can run amuck in your cluster. Relentlessly messing with your precious snowflakes.

You will learn how to:

  • Use the Kubernetes API with Python.
  • Install a container with a simple script that accesses the Kubernetes API.
  • Invoke a Cron Job with the Chaos script to randomly melt away Pods.
  • Use annotations to define chaos targets.

You don't need a fancy system to carry out the essence of chaos testing. You leveraged the basics of Kubernetes to implement a core Pod destructor using just a Python script in a small custom container then ran it as a CronJob.

With these steps you have learned:

  • ✔ Use the Kubernetes API with Python.
  • ✔ Install a container with a simple script that accesses the Kubernetes API.
  • ✔ Invoke a Cron Job with the Chaos script to randomly melt away Pods.
  • ✔ Use annotations to define chaos targets.

In the last year we've seen Chaos Engineering move from a much talked-about idea to an accepted, mainstream approach to improving and assuring distributed system resilience. As organizations large and small begin to implement Chaos Engineering as an operational process, we're learning how to apply these techniques safely at scale. The approach is definitely not for everyone, and to be effective and safe, it requires organizational support at scale. -- ThoughtWorks Radar

References


For a deeper understanding of these topics and more join
Jonathan Johnson
at various conferences, symposiums, workshops, and meetups.

Software Architectures ★ Speaker ★ Workshop Hosting ★ Kubernetes & Java Specialist

Pure Chaos

Step 1 of 5

Your Kubernetes Cluster

For this scenario, Katacoda has just started a fresh Kubernetes cluster for you. Verify that it's ready for your use:

kubectl version --short && \ kubectl get nodes && \ kubectl get componentstatus && \ kubectl cluster-info

It should list a 2-node cluster and the control plane components should be reporting Healthy. If it's not healthy, try again in a few moments. If it's still not functioning refresh the browser tab to start a fresh scenario instance before proceeding.

The Helm package manager used for installing applications on Kubernetes is also available:

helm version --short

Kubernetes Dashboard

You can administer your cluster with the kubectl CLI tool or use the visual Kubernetes dashboard. The Dashboard can be accessed from the tab labeled Kubernetes Dashboard above the command line. When the Dashboard first appears, it will prompt for an access token. At any time you can run this script to access the Dashboard token:

token.sh

This script will display the token in the terminal. Copy the green text using your browser's copy feature then paste the token into the prompt when the Dashboard is accessed. If the Dashboard is still starting up, then Katacoda will report the access error. Once the dashboard Pod reports the status Running it can be accessed:

kubectl get pods -n kube-system -l app.kubernetes.io/name=kubernetes-dashboard