What’s New in Docker 17.06

by Brandon Mitchell | Wednesday, Jun 28, 2017 | Docker

Docker recently released 17.06.0-ce, their latest stable release for the community edition. Full release notes are available at https://github.com/docker/docker-ce/releases/tag/v17.06.0-ce. Here are some of the highlights from this release:

Version 1 Registries Are Deprecated

If you use a 3rd party registry with docker, you'll need to make sure it supports the version 2 protocol. If not, you can reenable version 1 support with --disable-legacy-registry=false but expect version 1 support to be completely removed in the 17.12 release. This doesn't affect users of the docker hub, the open source registry image, or the Docker Trusted Registry. Docker started deprecating version 1 support back with 1.8.3 which means this has been on the way out for almost 2 years now, but switching the default flag from false to true means this is the first time it will be visible to most users. [#33629]

Services Are Now Pinned by the Client

Docker now pins images to a sha256 hash on the client instead of on the swarm manager when deploying and updating services. This also changed the behavior to resolve the image using the tag on the registry rather than first using an image that's already downloaded. So if the tag on the registry gets updated between when you last pulled it and when you deploy a new service, you may be surprised to find a different version of the image deployed to the host, along with long delays to deploy the image while docker first downloads the changed image.

To revert back to the old behavior, you can pass --no-resolve-image to your docker service create command. [#32388]

Stack Deploy Features

Docker continues to add to the new docker stack deploy command including:

• Support for placement preferences has been added to docker stack deploy. That's docker speak for a method to automatically spread your containers across multiple availability zones, avoiding single points of failure. To use this, you label your nodes with labels like region or rack for a single data center, and then the compose.yml can have a placement section like the example below. [cli #35]

version: "3.3"
services:
  app:
    image: registry:5000/app:latest
    deploy:
      placement:
        constraints: [node.role == worker]
        preferences:
        - spread: node.labels.rack

• Specifying DNS name servers and search domains is now possible in the stack definition. The compose syntax includes this at the top level as a list. [#32059]

version: '3'
services:
  app:
    image: registry:5000/app:latest
    dns:
      - 8.8.8.8

• Supporting the read_only option to create secure services that don't allow attackers to write to the filesystem. [cli #73]

version: '3'
services:
  app:
    image: registry:5000/app:latest
    read_only: true

Docker Service Features

Docker adds newer features to the docker service and related commands first, before they get added to the compose definition for creating stacks. So these next features will be an early preview for what is likely to come to stacks in the next releases.

• Configs have been added as a top level and a feature for services. Docker took what they did with secrets and mostly copied it to another feature for config files. These have less security requirements than secrets, and can be mounted anywhere inside of the container (instead of just /run/secrets). This allows you to centrally manage the configuration for your swarm containers, without baking it into the image and without requiring you to setup and manage volumes across your swarm.To use the new service configs, see the options under docker config. [#32336, cli #45, #33169]
• Support --detach and --quiet options with --rollback which is nice since before you would get a warning about the --detach flag behavior changing even though --detach wasn't a valid option. (While this is a service unique feature, the ability to pass the --detach flag to docker service deploy is an open issue with docker.) [cli #144]

$ docker service update --rollback --detach=false restart-test_test
restart-test_test
rollback: manually requested rollback
overall progress: rolling back update: 1 out of 1 tasks
1/1: running
verify: Waiting 1 seconds to verify that tasks are stable...

Swarm Events

Docker now includes swarm events in their event stream. You can monitor these with:

docker events -f scope=swarm

This is going to open up the option for tools to monitor the swarm for changes and update dynamically without polling. For those using jwilder/nginx-proxy, this is going to allow that to work with swarm services. [#32421]

Windows Gets Swarm Secrets Support

Windows support continues to expand, now with secrets support. Note, this isn't quite as secure as the Linux secrets because windows doesn't have RAM disk support, yet. As a result, the Windows secrets will be stored on the disk. Since Windows doesn't support UID/GID, the secrets are accessible by administrators and system inside the container. [#32208]

Swarm Node-Local Networking

Swarm services can now use local networks. This was done by allowing the network to be defined with --scope=swarm even when using a local driver. Defining a network with this scope creates the network id on the swarm manager which is needed when swarm defines a service.

Two other options were added to docker network create, --config-only to define a network on the local node that consists of only the configuration, and --config-from to define a swarm scoped network that uses one of the locally defined --config-only networks. This allows you to have a network id defined at the swarm level, but networks configured locally on each node.

This will be useful for those working with macvlan and ipvlan drivers that want to take advantage of swarm. You may also want to have some containers only talking on a local bridge or attached directly to the host network. If you try to upgrade an existing swarm and take advantage of connecting containers to the default bridge or host network, expect to see the following error:

Error response from daemon: could not find the corresponding predefined swarm network: network host not found

You'll need to leave the swarm and rerun the swarm init (possibly just a docker swarm join for worker nodes) for it to support these existing networks. [#32981]

Manage the Swarm CA

If you ever have the need to rotate not only your swarm join tokens, but the embedded CA certificates themselves in swarm, docker now lets you rotate the certificates on demand from a simple command line. You can also inject your own certificates if the generated ones do not meet your requirements. Since docker ships swarm in a secure by default state, the most likely reason to manage the CA directly is if you have a requirement to give a back door to the security department so they can inspect all of the encrypted swarm traffic. [cli #48, #32993]

$ # show the initial state of the swarm
$ docker info --format '{{ .Swarm.Cluster.TLSInfo.CertIssuerPublicKey | json }}'
"...DQgAET2g4y+uuBO73UfUKLJD/uaw62whNtaO+AX0kNEmVbbabyB3jzNtGUAclWBrEwG5JHGAkByAyZJnd66zkCc2BdA=="

$ # rotate the CA
$ docker swarm ca --rotate
desired root digest: sha256:...
  rotated TLS certificates:  1/1 nodes
  rotated CA certificates:   1/1 nodes
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

$ # you can see the end of the key has changed as a result of the rotation
$ docker info --format '{{ .Swarm.Cluster.TLSInfo.CertIssuerPublicKey | json }}'
"...DQgAE0FRAQRcBI3j/iLUcp712CMLCQSvv020zeNOZxyLy7lSEbmuJx0YTphBvR2W15fLl7+WypKEGTZ8Nnx9Vqk7KAg=="

Format Options For Everyone

docker stack ls, docker history, docker system df all received --format options. This is a nice addition for anyone that does lots of scripting with the docker command line.

My preferred way to find all the values I can use in the format is to use --format {{ . | json }} to JSON encode the resulting lines. Below are some examples with the new flags. [#31557, #30962, #31482]

$ docker stack ls --format '{{. | json}}'
{"Name":"test","Services":"1"}

$ docker history ubuntu --format '{{ . | json }}'
{"Comment":"","CreatedAt":"11 days","CreatedBy":"/bin/sh -c #(nop)  CMD [\"/bin/bash\"]","CreatedSince":"11 days ago","ID":"7b9b13f7b9c0","Size":"0B"}
{"Comment":"","CreatedAt":"11 days","CreatedBy":"/bin/sh -c mkdir -p /run/systemd \u0026\u0026 echo '...","CreatedSince":"11 days ago","ID":"\u003cmissing\u003e","Size:"7B"}
{"Comment":"","CreatedAt":"11 days","CreatedBy":"/bin/sh -c sed -i 's/^#\\s*\\(deb.*universe\\...","CreatedSince":"11 days ago","ID":"\u003cmissing\u003e","Size":"2.76kB"}
{"Comment":"","CreatedAt":"11 days","CreatedBy":"/bin/sh -c rm -rf /var/lib/apt/lists/*","CreatedSince":"11 days ago","ID":"\u003cmissing\u003e","Size":"0B"}
{"Comment":"","CreatedAt":"11 days","CreatedBy":"/bin/sh -c set -xe   \u0026\u0026 echo '#!/bin/sh' \u003e...","CreatedSince":"11 days ago","ID":"\u003cmissing\u003e","Size":"745B"}
{"Comment":"","CreatedAt":"11 days","CreatedBy":"/bin/sh -c #(nop) ADD file:5aff8c59a707833...","CreatedSince":"11 days ago","ID":"\u003cmissing\u003e","Size":"118MB"}

$ docker system df --format '{{ . | json }}'
{"Active":"8","Reclaimable":"5.108GB (90%)","Size":"5.658GB","TotalCount":"81","Type":"Images"}
{"Active":"2","Reclaimable":"2.221kB (99%)","Size":"2.223kB","TotalCount":"9","Type":"Containers"}
{"Active":"1","Reclaimable":"2.432GB (100%)","Size":"2.432GB","TotalCount":"15","Type":"Local Volumes"}

Misc

• Plugins can now read metrics from the metrics socket. [#32874]
• docker login no longer has the --email option. This was unused and deprecated for quite some time, but some may still have this included in their scripts and need to update them to use the new version. [cli #143]
• Swarm networking traffic for management can now be on a separate network interface from swarm networking traffic between containers. This allows you to manage the swarm without impacting production container traffic and lets you manage the swarm even when containers are flooding their data network. [#32717]

What about 17.04 and 17.05

If you've only been installing stable versions, here is what you missed in the last few releases:

• Nodes can join a swarm with their availability set to drain. That gives you time to do some configuration or testing before containers start to get deployed on the machine.
• Multi-stage build support. This was shown at DockerCon 17 and makes a nice addition to those that like to compile their code with Docker containers but don't want to ship the compiler in their resulting image.
• In the Dockerfile, build args can now come before the FROM line so you can adjust your base image with a flag instead of a new Dockerfile.
• The Dockerfile may be passed to the build command on stdin (docker build -f - .) which can be used by those that automatically generate their Dockerfile for their builds.
• The --mount flag was added to the docker run and docker create commands which allows you to mount multiple volumes with different drivers.
• You can now see logs of a swarm service with docker service logs without retrieving them from each container individually.
• The update and rollback order are now configurable (see --update-order and --rollback-order) on services so you can decide if you'd like to start the new instance first before stopping the existing instance. This will help those doing a rolling upgrading of a single replica service avoid any outage.
• You can now add and remove networks from a swarm service via the docker service update command.
• While deploying stacks, you now have the --prune option to remove services that are no longer in the yml stack definition.

Issue with 17.06

An issue was found late in the release process of 17.06 and couldn't be fixed in time. The regression was in how remote tar.gz files are processed with the ADD command in a Dockerfile. Before they would be automatically extracted, and this will be the behavior in 17.06.01. But for 17.06, if you have a Dockerfile that uses this feature, it will not automatically extract the file. [#33849]