BoxBoat Blog

Service updates, customer stories, and tips and tricks for effective DevOps

x ?

Get Hands-On Experience with BoxBoat's Cloud Native Academy

Open Source CVE Scanner Round-Up: Clair vs Anchore vs Trivy

by David Widen | Friday, Apr 24, 2020 | Docker Security

featured.png

As packaging applications inside Docker images becomes ubiquitous, more organizations are investing in hardening their Docker images. After you've run your application code through static and dynamic analysis tools, organizations typically leverage a CVE image scanner installed in their Docker registry. This allows you to identify known CVEs before containers are deployed, reducing your risk profile.

There are many vendors that provide CVE scanning tools for Docker images. Unfortunately, they don't all catch every single CVE. In fact, they often disagree about which CVEs are present in the same Docker image.

We decided to run several Docker images through three different open source CVE scanners to see how closely related the results were.

First, we uploaded the following images to our Harbor image registry:

  • alpine:latest
  • nginx:latest
  • nginx:alpine
  • openjdk:jre-alpine
  • ubuntu:latest

Harbor is the open source Docker image registry tool that we typically recommend to our clients. It is feature rich and we've seen a lot of success with our enterprise customers. Our internal Harbor Registry has Clair, Anchore, and Trivy installed, so we ran each image through those scanning tools and collected the results.

OSS CVE ScannerVendorInstallation
ClairRed Hatdocs
AnchoreAnchoredocs
TrivyAquasecdocs

Alpine Base Image

One of the most important “best practices” that companies can invest it when it comes to creating secure Docker images is to pick a lightweight base image. This minimizes the additional software that malicious actors can use to lateral into different processes and perform a privilege escalation.

Most organizations choose Alpine because of its lack of known vulnerabilities. When we ran the image scans, both Anchore and Clair reported zero CVEs; however, Trivy reported that the image is susceptible to CVE-2020-1967, which is a vulnerability introduced by openssl.

Anchore Scan Results for Alpine

Clair Scan Results for Alpine

Trivy Scan Results for Alpine

Based on these results, is alpine:latest secure?

Let's find out. When we examined the Trivy CVE report for CVE-2020-1967, it notes that version 1.1.1d-r3 of openssl is affected, and is fixed in 1.1.1g-r0.

To determine which scanner is correct, we must determine which version of openssl is in this image.

If you want to try this out yourself, the hash of the image is a187dde48cd2, and it was created sometime near the end of March, 2020. We pulled down that image from our Harbor registry, created a container on the CLI from that image, and opened up a shell.

After a little bit of investigative work, we figured out where the openssl library is stored on the filesystem, Then we generated the strings from the shared object file and tried to grep for a few variations of “OpenSSL” and “version” until we got the version information.

$ docker container run -t -i a187dde48cd2 sh
$ strings /lib/libssl.so.1.1 | grep "^OpenSSL"
OpenSSL 1.1.1d 10 Sept 2019

We can see that version 1.1.1d from September 2019 is installed, so this image is almost certainly vulnerable. However, if you were only using Anchore or Clair, you wouldn't know to upgrade this library to harden your application.

However, as I'm sure some of you have already noticed, we are being a bit harsh. This CVE was reported on April 21st 2020, and we're writing this article on April 23rd 2020. We can't realistically fault Clair and Anchore for not detecting this vulnerability just because it isn't in their database. In fact, we may have updated Harbor's CVE database prior to that vulnerability being discovered, so it wouldn't have a chance of being detected.

However, the fact remains that not all CVE image scanners will detect the same vulnerabilities, which you'll see more of below.

OpenJDK JRE Alpine Base Image

Java is a widely used framework, so let's look at the results from the openjdk:jre-alpine image next. Ignoring the fact that using Java will introduce significant vulnerabilities into your code base, all of the scan results are different, some substantially. This makes it difficult for enterprise security teams to evaluate how secure images actually are:

Anchore Scan Results for Java

Clair Scan Results for Java

Trivy Scan Results for Java

What's even more interesting is that Clair only found 10 CVEs but was not able to classify their severity. Anchore found 55 more CVEs than Trivy, but Trivy detected a critical CVE that eluded Anchore.

What is the critical CVE that Trivy found?

CVE-2019-2201 - Critical - libjpeg-turbo - current version: 1.5.3-r1 Description: In generate_jsimd_ycc_rgb_convert_neon of jsimd_arm64_neon.S, there is a possible out of bounds write due to a missing bounds check. This could lead to remote code execution in an unprivileged process with no additional execution privileges needed. User interaction is needed for exploitation.Product: AndroidVersions: Android-8.0 Android-8.1 Android-9 Android-10Android ID: A-120551338

First of all, this looks like an Android library so most server-side Java applications should never touch this code, making the chances of exploiting this unlikely. However, its still a critical vulnerability that exists in code and only one of the image scanners detected it.

In addition, it was reported in 2019, so it should be in the CVE databases for each of the three scanners.

Do Any of the CVE Scanners Agree?

No. Unfortunately, they do not. Let's take a look at the image scan results from the standard NGiNX image, as well as Ubuntu.

NGiNX CVE Scan Results

AnchoreClairTrivy

The scanners can't agree on NGiNX. Does it have 2 critical CVEs, or only 4 medium and 56 negligible?

NGiNX:Alpine CVE Scan Results

AnchoreClairTrivy

These results are similar to those from standard Alpine - Trivy was able to detect the CVE, but the others were not. Hopefully all three scanners will detect this CVE when we scan these images next week.

Ubuntu CVE Scan Results

AnchoreClairTrivy

What about Ubuntu? Depending on which CVE scanner you use, it varies from “maybe we should fix this” to “let's just ask the platform team to pull all nighters this week."

How Should you Scan and Secure Docker Images?

Scanning Docker images for CVEs is only one small part of creating a secure software supply chain. While there aren't any COTS tools today that will automatically leverage multiple CVE scanners and create a superset of their results to inform your security team, choosing the right scanner and strong CI/CD processes will help with this process. We can help you identify the right CVE scanner to harden your Docker images and create the necessary CI/CD automation to harden your secure software supply chain.