What’s New in Docker 1.12 – Part 2

by Brandon Mitchell | Wednesday, Sep 7, 2016 | Docker

This is the second part in a series covering the Docker 1.12 release. In this part, we will cover the changes introduced to networking, the new plugin interface, and the client API. See part 1 for changes introduced to building images, distribution, and logging.

Networking

Docker 1.12 added several changes to the networking stack, including how host networking handles DNS, aliases for container id's, and adding labels to networks. The biggest change of all is the MacVlan driver coming out of experimental.

MacVlan Driver

Pull request #23524

This new driver allows a MacVlan managed on the network to access docker containers directly, removing the overhead associated with the bridged network. To use this, you will need your own router to act as a gateway for the network, and any traffic routing between networks needs to be done by that gateway. By default, broadcast and inter-network communication is allowed but direct host communication is blocked. You are limited to a single MacVlan per network interface on the host, but you may use vlan tags like eth0.10 (vlan 10) to create multiple interfaces per physical NIC.

Presently, each container will have a separate MAC address, so the NIC needs to go into promiscuous mode to watch for traffic to any running containers. When IPVlan comes out of experimental, separate MAC addresses will no longer be required.

The below network diagrams from Docker show how these pieces fit together:

An example of creating one of these networks and running two containers on the network looks like the following:

# define network with macvlan dirver on eth0
host$ docker network create -d macvlan --subnet 172.18.36.0/24 \
  --gateway 172.18.36.1 -o parent=eth0 pub_net

# create alpine node with a static ip
host$ docker run --net pub_net --ip 172.18.36.10 -itd alpine /bin/sh

# create a second alpine node to ping the first
host$ docker run --net pub_net -it --rm alpine /bin/sh
container$ ping 172.18.36.10
# ...

For further details, see the docs on github.

Network Filters

Pull request #22319, pull request #23300, and pull request #21495

Network filters allow you to limit the list of networks returned in docker network ls. They were extended with three different pull requests for the 1.12 release. You're now able to filter networks by driver and networks can now have labels which can be filtered, and the docker ps command was also extended to allow filters on networks. In practice, this looks like:

$ docker network ls -f driver=bridge
...

$ docker ps -a --filter network=bridge
...

$ docker network create --label env=dev --label loc=east east_dev
63a1988084858f9ffd22acbd941c6a957079df53ef52d3caa3bbf1cb1ad4b188

$ docker network ls -f label=env
NETWORK ID          NAME                DRIVER              SCOPE
63a198808485        east_dev            bridge              local

$ docker network ls -f label=env=dev
NETWORK ID          NAME                DRIVER              SCOPE
63a198808485        east_dev            bridge              local

$ docker network ls -f label=env=dev -f label=loc=east
NETWORK ID          NAME                DRIVER              SCOPE
63a198808485        east_dev            bridge              local

$ docker network ls -f label=env=dev -f label=loc=west
NETWORK ID          NAME                DRIVER              SCOPE

More details are available in the network ls documentation.

Add the container ID as a DNS alias

Pull request #21901

Docker now adds the short container ID as a network alias. This was done to streamline the process being performed by Docker Compose when linking containers. In practice, this looks like:

$ docker run -itd --name test_short --network test \
  --hostname testshort debian /bin/bash
863d35ae29023703ae15953e288ccc208aad191689d1cf53929288b3e88d9251

$ docker exec -it test_short /bin/bash

root@testshort:/# ping 863d35ae2902
PING 863d35ae2902 (172.19.0.2): 56 data bytes
64 bytes from 172.19.0.2: icmp_seq=0 ttl=64 time=0.079 ms
64 bytes from 172.19.0.2: icmp_seq=1 ttl=64 time=0.069 ms

DNS and Host files can be updated even with --net=host

Pull request #22408

Previously, if you used --net=host, the /etc/resolv.conf for DNS and /etc/hosts files were mapped directly to the files on the host. Now these are bind mounts to container metadata which means you may adjust the values for DNS and add entries to the /etc/hosts that may be unique per container. In practice, this looks like:

$ docker run --rm -it \
  --net=host --dns=8.8.8.8 --add-host example.com:127.0.0.1 \
  debian /bin/bash

root@bmitch-asusr556l:/# cat /etc/resolv.conf
nameserver 8.8.8.8

root@bmitch-asusr556l:/# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
127.0.0.1       example.com

Other Networking Changes

Docker fixed a bug when renaming containers to automatically update the integrated DNS in pull request #22466.

Docker added a --local-link-ip option to define an address in the 169.254.0.0/16 and fe80::/64 network space, typically used when a DHCP server isn't available on the host, in pull request #23415.

Plugin Infrastructure

Pull request #23446

This is a new capability added to the experimental branch of the docker engine that allows plugins to be installed as easily as images are pulled. Instead of manually installing pluggins on the host environment with unique commands per plugin, the process is now managed with a new docker plugin top level command with install, enable, disable, rm, inspect, and set subcommands. These plugins are even listed on the Docker hub, though you can't pull them with docker pull

Once installed, a plugin is implemented under the hood as a container that starts before other containers, ensuring that the plugin API is available when containers launch. In addition to making it easy to install and manage plugins, it also prompts for access per plugin for finer grained controls. In the past, if a plugin was installed directly on the host, it would effectively have full root privileges.

As an experimental feature, documentation is very limited at the moment. More details can be found in the discussion on issue #20363. Tibor provided the below example with the pull request:

$ docker plugin install tiborvass/no-remove:latest
Plugin "tiborvass/no-remove:latest" requested the following privileges:
 - Networking: host
 - Mounting host path: /data
Do you grant the above permissions? [y/N]

Client and Remote API

With Docker's 1.12 release, the API version was bumped to 1.24 and a handful of changes were included. Most of these are minor tweaks to add filters. The most user visible change was a split in the docker daemon out from the docker binary itself.

Error messages

Pull request #22880 and pull request #22448

Remote API errors are now formatted in JSON. Comparing 1.23 to 1.24 shows the change on a missing page:

$ curl --key ~/.docker/key.pem -E ~/.docker/cert.pem https://127.0.0.1:2376/v1.23/12345
page not found

$ curl --key ~/.docker/key.pem -E ~/.docker/cert.pem https://127.0.0.1:2376/v1.24/12345
{"message":"page not found"}

Docker now returns 403 (forbidden) instead of 500 (server error) when request is rejected by an authorization plugin.

Docker info

Pull request #21172, pull request #20410, and pull request #21945

Several changes were made to the output of docker info, including:

• Adding “Security Options” for apparmor, selinux, seccomp.
• Adding “insecure Registries” that were defined with the daemon option --insecure-registries. Note that secure registries are not tracked since they require no special behavior.
• “Thin Pool Minimum Free Space” was added for those using device mapper with direct-lvm.

Daemon now split from client

Pull request #20639

The docker daemon is now a separate binary, dockerd instead of the subcommand docker daemon. If you call docker daemon in 1.12, it will exec dockerd to make the transition transparent and allow existing startup scripts to continue unchanged. This split in the binaries makes it possible to install Docker as only the client code.

Docker image filters

Pull request #22908

docker images now supports two new filters, before and since to search for images built before or after other images. This looks like:

$ docker images busybox:1.24
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             1.24                47bcc53f74dc        4 months ago        1.113 MB

$ docker images busybox -f since=busybox:1.24
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             1.25                2b8fd9751c4c        6 weeks ago         1.093 MB
busybox             latest              2b8fd9751c4c        6 weeks ago         1.093 MB

$ docker images busybox -f before=busybox:1.24
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             1.23                a84c36ecc374        10 months ago       1.096 MB

Search options

Pull request #23107 and pull request #22369

Search options now allow the default search limit to be changed from 25 to any numbers from 1 to 100 with --limit. A new filter option was also added with possible parameters: is-official, stars, and is-automated. Note that filters are applied after the limit is applied, so if you want to find the top two official images, you may need to pass a limit greater than two to retrieve enough images. Here are some sample commands using these new options:

$ docker search --limit 125 debian
Error response from daemon: Limit 125 is outside the range of [1, 100]

$ docker search --limit 2 debian
NAME             DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
debian           Debian is a Linux distribution that's comp...   1546      [OK]
eboraas/debian   Debian base images, for all currently-avai...   5                    [OK]

# note how only one result is returned even when the limit is set to 2
$ docker search --limit 2 --filter is-official=true debian
NAME      DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
debian    Debian is a Linux distribution that's comp...   1546      [OK]

$ docker search --limit 5 --filter is-official=true debian
NAME          DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
debian        Debian is a Linux distribution that's comp...   1546      [OK]
neurodebian   NeuroDebian provides neuroscience research...   26        [OK]

$ docker search --limit 20 --filter stars=5 debian
NAME                       DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
debian                     Debian is a Linux distribution that's comp...   1546      [OK]
neurodebian                NeuroDebian provides neuroscience research...   26        [OK]
jesselang/debian-vagrant   Stock Debian Images made Vagrant-friendly ...   8                    [OK]
eboraas/debian             Debian base images, for all currently-avai...   5                    [OK]

# out of the top 100 "debian" search results, 8 of them have at least 5 stars (add one for the header)
$ docker search --limit 100 --filter stars=5 debian | wc -l
9

Other API changes

Docker fixed a race condition in docker run -i --restart always in pull request #22777.

Docker stats were changed to show values in bytes (MiB vs MB) to match inputs in docker run -m in pull request #21773.

Windows clients always enable a VT to avoid extra unprintables appearing in output in pull request #23878.

What's left for part 3

We are getting close to the end of this dig through the release notes. Part 3 includes changes to the runtime, swarm, volumes, and deprecated features.