Using Containers to Reduce Your Exposure to Meltdown and Spectre

by Brandon Mitchell | Friday, Jan 5, 2018 | Docker

The release of Meltdown and Spectre has got IT departments everywhere scrambling, and for good reason. If you are running on an Intel, AMD, or ARM platform, data you expected to be secure can now be exposed by a malicious application.

Eliminating the risk from untrusted code getting privileged access is going to take some workarounds in the OS kernel and ultimately updates from the CPU vendors. Until then, since containers run with a shared OS kernel, it is a best practice to segregate workloads of different trust levels onto separate VMs, if not completely separate hardware.

It’s not all bad news, docker also has a role to play in being part of the solution. We can use many of the built in security features of docker containers to lock down applications in a way that prevents untrusted code from ever being run on our host.

Minimal Filesystem

The first step is to keep our filesystem inside our container small. Do not include compilers, script interpreters, or even a login shell if you can avoid it. With a significantly stripped down environment, not only do you make images that are fast to ship from development to production, you make a very small attack surface where attackers may not have the tools they expect to launch a lateral attack from inside the container.

Docker introduced multi-stage builds that allow us to run the application build with a compiler, and package up a minimal runtime environment, both from a single command using a single Dockerfile. The best example of this in action is a statically linked Go binary with the following Dockerfile:

ARG GOLANG_VER=1.8
FROM golang:${GOLANG_VER} as builder
WORKDIR /go/src/app
COPY . .
RUN go-wrapper download
RUN go-wrapper install
 
FROM scratch
COPY --from=builder /go/bin/app /app
CMD ["/app"]

When you build an image with the above, the first stage includes the Go compiler and copies in the application source code. It gets built into an “app” binary. The second stage starts from “scratch” which is a special docker image that is completely empty, no compilers, no shells, not even a libc. Imagine a drive after a format command or “rm -rf” and that is what “scratch” gives you. In there we add a single statically compiled “app” binary and nothing else. If an attacker exploits a hole in this app to get access to run commands, they’ll quickly find that there’s only one command available to run.

Read Only Filesystem

A minimal filesystem doesn’t help if an attacker finds a way to write their own binaries inside the container. Docker gives us a simple way to lock down the container filesystem with one option, read only. Of course there will be applications that need to write to some files while running. We can see this in a running container by using “docker diff” against a running container to list what files and directories have been changed. Once we know what the application needs, we can then setup in memory temporary filesystems for those directories, and configure them without the ability to run executables or even set a suid bit on the binaries. The result looks something like:

docker container run --read-only --tmpfs /run:rw,noexec,nosuid go-app

Do Not Run Containers As Root

Most sysadmins would turn a bright shade of red or find an excuse to start happy hour at lunch if you told them all of your applications will be run as root. And yet that is the default user in docker. As bad as that sounds, there really isn’t a need to panic. Containers should only run a single application so root access could let it hack itself. And root inside a container has fewer capabilities than root on the host OS. You cannot mount filesystems, change the date, modify networking interfaces, or even change the hostname inside of the container without explicit access being added to the container. But since containers run with a same kernel on the host, there is a risk that a container breakout would result in root access on the host.

Two options exist to mitigate this risk. One is to enable user namespaces on the docker daemon. This offsets all uid’s inside the container to be different from those on the host, so root in the container is an unprivileged user on the host. However, this is disabled by default because it impacts access to volumes on the host. It also needs to be configured on every docker host, including your developer workstations.

The second option is much easier, add a user to your company’s base image and use it. Even with this option, you need to deal with uid mapping issues in volumes on your developer workstations. Files created by the container may appear with an owner and permissions that make them unusable outside the container, and vice versa. We commonly solve this using an entrypoint in our containers that runs as root to correct permission issues, and then drops access to the user to run the application.

Use CGroups

CGroups allow you to limit the CPU and Memory being consumed by a container. While they will not stop an attack, they can slow it down. Considering how Meltdown and Spectre are already relatively slow attacks, measured in KB/sec, and highly depend on measuring the timing of actions, adding a CPU throttle could result in bad timing measurements at best, or at least significantly slow down a breach in progress.

Advanced Mode

These options are quick and easy. They reduce the chance of an attacker being able to run malware inside your container. More advanced tools do exist that require a deeper understanding of your application to utilize. They include SELinux for RedHat and CentOS environments, AppArmor for Debian and Ubuntu environments, and Seccomp. Each of these restricts what an executable can do inside the container. The first two are configured by docker to limit access to /proc and /sys by default. And Seccomp limits what syscalls you permit. Neither of these would not stop the Meltdown attack, but they could further limit the attack surface available to Spectre.

Layers of Security

Security needs a layered defense and docker can help provide one of these layers. Containers will not stop an kernel or hardware vulnerability once it is running on the host, but they may prevent an attack on an application from extending into an attack on the host by providing a limited environment in which that application runs. This applies not just to the Meltdown and Spectre attacks, but also to the next attack.