BoxBoat Blog
Service updates, customer stories, and tips and tricks for effective DevOps
Building Containers with Kubernetes and Knative
by Cole Kennedy | Friday, Aug 10, 2018 | Kubernetes
Developing for Kubernetes can be a daunting task for any developer not familiar with the ecosystem. The developer needs to understand how to create spec files, author CI/CD scripts with a system such as Jenkins or CircleCI, and instrument logging and tracing. Knative aims to solve some of these issues, abstracting the details of building images away from the developer.
Knative helps developers build, deploy, and manage modern serverless workloads on Kubernetes.
Knative has the backing of industry giants and is designed from lessons learned at companies such as Google, Red Hat, IBM, and SAP. It is a set of components that enables modern CI/CD workflows that are native to kubernetes. The current components that are available today are
- Build - Build containers from source.
- Serving - Serverless, or “Request-driven compute”
- Eventing - Eventing
In this article, we will be focusing on the Build component.
Build Components
Build
The Knative Build component extends Kubernetes with custom resource definitions or CRDs this will give us a new object Build we can call from kubectl ex. kubectl get Builds Knative Build includes a few batteries included BuildTemplate that cover most situations.
BuildTemplate
BuildTemplate encapsulates a shareable build process with some limited parameterization capabilities.
BuildTemplates are provided for
- Bazel - Bazel is a tool that automates software builds and tests.
- Buildpack - Buildpack is a core link in the chain of the Cloud Foundry deployment process
- Jib - Jib builds Docker and OCI images for your Java applications and is available as plugins for Maven and Gradle.
- Kaniko - Kaniko is a tool to build container images from a Dockerfile, inside a container or Kubernetes cluster.
We will be using the Kaniko build template. Kankio is a project that builds containers without elevated privileges solving many of the issues when using the Docker daemon for building containers in a distributed environment.
Kaniko doesn't depend on a Docker daemon and executes each command within a Dockerfile entirely in userspace. This enables building container images in environments that can't easily or securely run a Docker daemon, such as a standard Kubernetes cluster.
ServiceAccount
Service accounts are the way Kubernetes can pass secrets to build templates. Each service account is associated with a single account on a service provider such as GitHub, or Docker Hub.
Builder
A builder is an image that executes a step in the build process. For example, you may have a builder that runs unit tests on your code before it is deployed. We will not be using the Builder component today.
Getting Started
We have a simple web application that we use for smoke testing. I have disabled automatic build on docker hub for this example. At the end of the example, Knative will pull the source code from GitHub, build the Docker image, and push it to Docker Hub with a single Kubernetes command.
Install Knative (On GKE)
I have stood up a isolated cluster to experiment with Knative. I do not recommend installing alpha software, such as Knative on a cluster you rely on. Instructions specific to your provider are found in the Knative GitHub repo. I will be using Google Kubernetes Engine version 1.10.5-gke.3. The install instructions below only apply if you are using GKE. Follow instructions specific to your provider. Remember, this is ALPHA software, things may not work as expected.
Initialize kubectl
gcloud container clusters get-credentials knative-builder --zone us-east1-b \
--project knative-playground
Ensure cluster is running
kubectl get nodes
Give the current user cluster admin rights
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole=cluster-admin --user=$(gcloud config get-value core/account)
Install Knative Build:
kubectl apply -f https://storage.googleapis.com/build-crd/latest/release.yaml
Wait for pods to come online
kubectl -n knative-build get pods -w
NAME READY STATUS RESTARTS AGE
build-controller-b76f56b4c-6km8j 1/1 Running 0 1m
build-webhook-6867b944b4-vbfxq 1/1 Running 0 1m
Check the the CRDs
kubectl get crd
NAME CREATED AT
backendconfigs.cloud.google.com 2018-08-10T03:24:50Z
builds.build.knative.dev 2018-08-10T03:32:37Z
buildtemplates.build.knative.dev 2018-08-10T03:32:37Z
scalingpolicies.scalingpolicy.kope.io 2018-08-10T03:25:10Z
Set project up for Kaniko
Kaniko requires a Dockerfile to be present. Below is the multi-stage Dockerfile for our simple web app. Because the application is written in Go it requires no dependencies allowing us to use the SCRATCH base image for a minimal artifact.
# Multistage Dockerfile
# Build stage
FROM golang:alpine AS build-env
ENV APP hello-world-webbapp
ENV GOOS linux
ENV GOARCH amd64
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 GOOS=${GOOS} GOARCH=${GOARCH} go build -a -o server
# Final
FROM scratch
COPY --from=build-env /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build-env /src/server .
CMD ["./server"]
Authentication
To push to Docker Hub we need to provide the Build object with a ServiceAccount, a ServiceAccount requires a Secret so let us make that first. Note: StringData is converted to a base64 encoded data object when applied to the cluster.
vim secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: basic-user-pass
annotations:
build.knative.dev/docker-0: https://index.docker.io/v1/
type: kubernetes.io/basic-auth
stringData: #NOT Base64 encoded
username: <docker username>
password: <docker password>
kubectl apply -f secret.yaml
Now make a ServiceAccount so we can pass this secret to a Build spec.
vim service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: build-bot
secrets:
- name: basic-user-pass
kubectl apply -f service-account.yaml
Build
Create a Build spec.
vim build.yaml
apiVersion: build.knative.dev/v1alpha1
kind: Build
metadata:
name: docker-build
spec:
serviceAccountName: knative-build #Service account create above
source:
git:
url: https://github.com/boxboat/hello-world-webapp.git
revision: master
steps:
- name: build-and-push
image: gcr.io/kaniko-project/executor:v0.1.0
args: #See kaniko docs for more info about the arguments passed
- --dockerfile=/workspace/Dockerfile
- --destination=docker.io/boxboat/hello-world-webapp
Now cross your fingers and kubectl apply -f build.yaml
Verify
The build should have been kicked off. Let us take a look.
kubectl get pods you should see a pod named docker-build with a postfix.
kubectl logs docker-build-XXXXX -c build-step-build-and-push
...
time="2018-08-10T04:58:40Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-08-10T04:58:41Z" level=info msg="cmd: copy [/etc/ssl/certs/ca-certificates.crt]"
time="2018-08-10T04:58:41Z" level=info msg="dest: /etc/ssl/certs/"
time="2018-08-10T04:58:41Z" level=info msg="Copying file /kaniko/0/etc/ssl/certs/ca-certificates.crt to /etc/ssl/certs/ca-certificates.crt"
time="2018-08-10T04:58:41Z" level=info msg="Taking snapshot of files [/etc/ssl/certs/ca-certificates.crt]..."
time="2018-08-10T04:58:41Z" level=info msg="cmd: copy [/src/server]"
time="2018-08-10T04:58:41Z" level=info msg="dest: ."
time="2018-08-10T04:58:41Z" level=info msg="Copying file /kaniko/0/src/server to /server"
time="2018-08-10T04:58:41Z" level=info msg="Taking snapshot of files [/server]..."
time="2018-08-10T04:58:43Z" level=info msg="cmd: CMD"
time="2018-08-10T04:58:43Z" level=info msg="Replacing CMD in config with [./server]"
time="2018-08-10T04:58:43Z" level=info msg="No files changed in this command, skipping snapshotting."
time="2018-08-10T04:58:43Z" level=info msg="No files were changed, appending empty layer to config."
2018/08/10 04:58:44 pushed blob sha256:95c547f3e953339de0631b594c759ce7fa849723a8c7f73a153c4784f2ead2d7
2018/08/10 04:58:44 pushed blob sha256:e16ad3b2385f4045d00eb3e686e8ab708a5cb282a30866607ec818f8156c44c2
2018/08/10 04:58:46 pushed blob sha256:1c9588050b0ef607466397076c3ca79735a0d96805f766164bd271719e82c8c4
index.docker.io/boxboat/hello-world-webapp:latest: digest: sha256:9335c0b0016c9482012ccf51cb818eb09c2a03f10406615ab430cfffd24e036c size: 590
Take a look at the Build objects.
kubectl describe builds
Name: docker-build
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"build.knative.dev/v1alpha1","kind":"Build","metadata":{"annotations":{},"name":"docker-build","namespace":"default"},"spec":{"serviceAcc...
API Version: build.knative.dev/v1alpha1
Kind: Build
Metadata:
Cluster Name:
Creation Timestamp: 2018-08-10T04:58:10Z
Generation: 1
Resource Version: 11158
Self Link: /apis/build.knative.dev/v1alpha1/namespaces/default/builds/docker-build
UID: fa4afb12-9c59-11e8-b076-42010a8e021e
Spec:
Generation: 1
Service Account Name: build-bot
Source:
Git:
Revision: master
URL: https://github.com/boxboat/hello-world-webapp.git
Steps:
Args:
--dockerfile=/workspace/Dockerfile
--destination=docker.io/boxboat/hello-world-webapp
Image: gcr.io/kaniko-project/executor:v0.1.0
Name: build-and-push
Resources:
Status:
Builder: Cluster
Cluster:
Namespace: default
Pod Name: docker-build-jpnfg
Completion Time: 2018-08-10T04:58:47Z
Conditions:
State: Succeeded
Status: True
Start Time: 2018-08-10T04:58:10Z
Step States:
Terminated:
Container ID: docker://570002d349fc9e4309dc22235e94587bcf5685141f771d146fec9ebd9ed9ec44
Exit Code: 0
Finished At: 2018-08-10T04:58:10Z
Reason: Completed
Started At: 2018-08-10T04:58:10Z
Terminated:
Container ID: docker://86af6b46d41e53297b6d646fc30bb72a239a052899a62c392e845629e8041a14
Exit Code: 0
Finished At: 2018-08-10T04:58:12Z
Reason: Completed
Started At: 2018-08-10T04:58:11Z
Terminated:
Container ID: docker://9c6c952ab653b150d7ad24b50eec0c1040a283f3921b477bf38690c4d927097f
Exit Code: 0
Finished At: 2018-08-10T04:58:46Z
Reason: Completed
Started At: 2018-08-10T04:58:12Z
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Synced 17s (x21 over 8m) build-controller Build synced successfully
Hopefully, you have a similar output. Check Docker Hub and make sure everything pushed correctly.
Caveats
- Knative is new, beyond bleeding edge. It is likely to move fast breaking all of your work.
- Every time you want to make a build you need to push a new build spec, this can be automated with Knative Serving, but that will have to wait for another post.
- Knative requires Istio. Istio adds many complications to your cluster, and it may not be correct for your situation.
Summary
I have shown you how to use Knative to move from source to container only having to define a Dockerfile. While Knative is not ready for production use we are excited about the value it will bring to our customers as it matures.
In the next few weeks, I will be writing follow-on articles that explore more of the Knative eco-system. We will complete the loop by deploying our web app using the Knative Serving component and integrate with GitHub webhooks to automate the build kickoff. Make sure you subscribe to updates (upper right of page).
Don't hesitate to contact boxboat for any of your containerization, devops, and orchestration needs.
