With those three commands, you have a two-node Kubernetes cluster.

You can view the status of all the nodes in the cluster with kubectl get nodes

See the status of the individual components running with kubectl get pods -n kube-system

Once all the components are "Running", the cluster is ready to start executing workloads.

Kubernetes workloads are managed via kubectl. There are multiple ways to deploy a workload, but a common approach is using a Deployment Object defined using yaml. The yaml that will be deployed is cat inception_k8s.yaml

The deployment object has three key properties.

1) The default defines three replicas. This means that three instances of the server will be running on the cluster.

2) The command runs _tensorflow_modelserver on port 9000 that exports a pre-built model called inception-export.

3) A name defined as a label that are used as selectors for other components to discover the service.

The yaml also includes a Service that exposes the deployment externally. Kubernetes supports multiple networking options, in this case it's of type NodePort. NodePort defines a well-known port that is available externally across the cluster. When a request comes in on the port, it is automatically load balanced between one of the available replicas.

Task

Use kubectl to create the deployment and service for the Tensorflow server. The tensorflow server will return a classification of an image based on the http://www.image-net.org/ dataset.

kubectl create -f inception_k8s.yaml

After a few moments the Pods, otherwise known as containers, will be created and started.

kubectl get pods

The underlying containers can be identified on the worker node.

docker ps | head -n4

The service has also been created allowing the server to be accessed on port 30000.

kubectl get svc

Once the Tensorflow Server Pods have started, workloads can now be scheduled against them. As the server has been exposed using a NodePort on port 30000, any requests to a Kubernetes master or node on port 30000 will be proxied to an available service.

Under the covers, Tensorflow server uses gRPC, a high performance, open source universal RPC framework, for responding to client requests. This means a client needs to be built using gRPC and HTTP requests will be rejected.

Accessing using Docker Container

The Docker Image _katacoda/tensorflowserving includes the client tools for communicating with the Tensorflow server over gRPC. It can be started as a Docker container using the command on the worker node docker run -it -v /root:/data katacoda/tensorflow_serving bash

The current /root directory has been mounted as a volume into the container. This allows the client to access the files on the host.

The following command will send the image cat.jpg to the server for classification.

bazel-bin/tensorflow_serving/example/inception_client --server=[[HOST_IP]]:30000 --image=/data/cat.jpg

After a few moments, a JSON result should be returned with classifications of the cat.jpg.

This approach is great for running a task on a local machine against the remote server.

Accessing using Interactive Pod

An alternate approach is to run an interactive Pod on the Kubernetes cluster. This allows you to run the process on a remote cluster. Start a interactive Pod with the command on the master:

kubectl run --rm -i --tty interactiveclient --image=katacoda/tensorflow_serving -- bash

One limitation is that you cannot easily mount volumes, as such, the command below downloads the cat image into the container before executing the workload.

curl -L https://katacoda.com/courses/tensorflow/deploying-tensorflow-on-kubernetes/cat.jpg -o /cat.jpg; bazel-bin/tensorflow_serving/example/inception_client --server=[[HOST_IP]]:30000 --image=/cat.jpg

The results should be the same.

Exit the interactive containers with the command exit

Interactive containers are useful for development purposes but at a large scale workload they become difficult to manage, especially for long running workloads.

To solve this, Kubernetes has a Batch Job API that allow single, one-off, workloads to be scheduled and executed on the Kubernetes cluster.

cat job.yaml

In this case, because the workload is running inside the Kubernetes cluster there is no need to expose the service publically. Instead, it's possible to use the built in Kubernetes Service Discovery and reference our Tensorflow server using a well-known name inception-deployment that was defined when the Deployment was created. The service is still load balanced, but accessed via port 9000 as it's no longer going via the NodePort service.

Task

Create the workload in the same way of deploying other Kubernetes objects using the command kubectl create -f job.yaml

In thie case, the workload has mounted in the /root directory of where it's running. Kubernetes has advanced storage APIs allowing you to integrate with external storages such as S3, NFS or EBS. Details can be found in the Volume documentation.

This job will be deployed into the Default namespace. Namespaces allow workloads to be defined into their own context, with their own set of security and available resources and useful when sharing a cluster with multiple teams.

The Batch workloads run as Pods on the cluster. View all of the workloads that are running, or have completed, with kubectl get pods -a

To get the logs from the running output, run the following command. This will get the logs from the last pod run.

kubectl logs $(kubectl get pods -a --no-headers | tr -s ' ' | cut -d ' ' -f 1 | head -n1)

This should output the same results as using the interactive containers. The added benefit is the workload can run in the background, with multiple batch jobs running concurrently to solve the problem. This API gives you greater flexibility to manage available resources. If not enough CPU or Memory is available, Kubernetes will queue the workload until space is available.

Extra Workflow

With the server in place, additional workloads can be sent to it, for example, katacoda.jpg.

By creating another Batch object on Kubernetes, the workload will be scheduled and executed.

kubectl create -f job-katacoda.yaml

As before, the logs can be accessed via kubectl.

kubectl logs -f $(kubectl get pods -a --no-headers | grep katacoda | tr -s ' ' | cut -d ' ' -f 1 | head -n1)

What did the classifier think to the Katacoda logo?