When you’re done with this tutorial you’ll have a small YAML file that describes your docker image’s desired state. This will allow you to test this:

docker run -p 8080:80 nginx

With this:

dgoss run -p 8080:80 nginx

The tests will run in about a half second, and output the following:

INFO: Starting docker container
INFO: Container ID: 25d23a24
INFO: Running Tests
File: /var/log/nginx/error.log: exists: matches expectation: [true]
File: /var/log/nginx/error.log: linkedto: matches expectation: ["/dev/stderr"]
File: /var/log/nginx/error.log: filetype: matches expectation: ["symlink"]
File: /var/log/nginx/access.log: exists: matches expectation: [true]
File: /var/log/nginx/access.log: linkedto: matches expectation: ["/dev/stdout"]
File: /var/log/nginx/access.log: filetype: matches expectation: ["symlink"]
Process: nginx: running: matches expectation: [true]
Port: tcp:80: listening: matches expectation: [true]
Port: tcp:80: ip: matches expectation: [["0.0.0.0"]]
HTTP: http://localhost: status: matches expectation: [200]
HTTP: http://localhost: Body: matches expectation: [Welcome to nginx!]
Package: nginx: installed: matches expectation: [true]
Package: nginx: version: matches expectation: [["1.11.10-1~jessie"]]

Total Duration: 0.012s
Count: 13, Failed: 0, Skipped: 0
INFO: Deleting container

Note: If you’re more of a visual person, there’s a video version of this tutorial at the bottom of this blog post.

Prerequisites

In order to follow along this tutorial, you need to have Docker installed.

Installing our test tool

We’ll be using Goss as the testing tool of choice. The installation is simple and only takes a few seconds.

curl -fsSL https://goss.rocks/install | sh

NOTE: If you are weary of curl | sh , you can install it manually by following the instructions here.

So.. What is Goss?

Goss is a YAML based serverspec alternative tool for validating a server’s configuration. It eases the process of writing tests by allowing the user to generate tests from the current system state. Once the test suite is written they can be executed, waited-on, or served as a health endpoint.

More info can be found on the Goss webpage.

Writing our test

dgoss is a helper tool that comes with Goss to make it extremely easy to write docker tests. Let’s say we run our docker image in the following way:

docker run -p 8080:80 nginx:1.11.10

To start writing our tests we substitute docker run with dgoss edit:

dgoss edit -p 8080:80 nginx:1.11.10

This will run the docker image, and drop us into an interactive shell with Goss installed in the container, you will see an output similar to this:

INFO: Starting docker container
INFO: Container ID: 9468c0c3
INFO: Run goss add/autoadd to add resources

Once in the interactive shell we can start writing our test suite. Paste in the following commands and they will derive tests from the running container’s state:

goss a file /var/log/nginx/access.log /var/log/nginx/error.log
goss a process nginx
goss a port 80
goss a package nginx
goss a http http://localhost
exit

Note: “a” is short for “add” in Goss.

After we exit the container will be deleted, and you will now see a goss.yaml in your current directory that contains the following:

file:
  /var/log/nginx/access.log:
    exists: true
    mode: "0777"
    size: 11
    owner: root
    group: root
    linked-to: /dev/stdout
    filetype: symlink
    contains: []
  /var/log/nginx/error.log:
    exists: true
    mode: "0777"
    size: 11
    owner: root
    group: root
    linked-to: /dev/stderr
    filetype: symlink
    contains: []
package:
  nginx:
    installed: true
    versions:
    - 1.11.10-1~jessie
port:
  tcp:80:
    listening: true
    ip:
    - 0.0.0.0
process:
  nginx:
    running: true
http:
  http://localhost:
    status: 200
    allow-insecure: false
    no-follow-redirects: false
    timeout: 5000
    body: []

This can be executed by running:

dgoss run nginx

Modifying our tests by hand

The YAML file we generated in the last section can be written by hand. The dgoss edit command is intended as a helper to ease the process of writing tests. However, in most cases the generated YAML will need to be modified. Let's go ahead and do that..

For the files section, we only care that the log files are symlinked to stdout/stderr, so let’s remove the mode, size, owner, group, and contains checks. Our files section should look as follows:

file:
  /var/log/nginx/access.log:
    exists: true
    linked-to: /dev/stdout
    filetype: symlink
  /var/log/nginx/error.log:
    exists: true           
    linked-to: /dev/stderr
    filetype: symlink

For the http check, let’s remove the https related flags and validate that the “Welcome to nginx!” message appears in the response body:

http:
  http://localhost:
    status: 200
    timeout: 5000
    body:
    - Welcome to nginx!

Our YAML file should now be 28 lines:

file:
  /var/log/nginx/access.log:
    exists: true
    linked-to: /dev/stdout
    filetype: symlink
  /var/log/nginx/error.log:
    exists: true
    linked-to: /dev/stderr
    filetype: symlink
package:
  nginx:
    installed: true
    versions:
    - 1.11.10-1~jessie
port:
  tcp:80:
    listening: true
    ip:
    - 0.0.0.0
process:
  nginx:
    running: true
http:
  http://localhost:
    status: 200
    timeout: 5000
    body:
    - Welcome to nginx!

Same as before, executing this is simply:

dgoss run nginx

What’s next?

This tutorial is meant as a quick start showcasing a simple test scenario. For more advanced use cases take see the following resources:

• Goss — The tool used in this tutorial
• dgoss — Helper tool packaged with goss, also used in this tutorial
• dgoss-examples — Repo containing more complex examples of using dgoss to validate docker images
• Goss manual — Goss documentation

Video version of this tutorial

If you found this article beneficial, please recommend it to others by clicking the heart.

Edit: I recently released a new tool Miniswarm, in this section of the tutorial video I show Goss health checks being used with Docker Swarm.

So.. What is Goss?

Note: Unless you’re familiar with Goss v0.2.0, I recommend you read this section.

Goss is a YAML based serverspec alternative tool for validating a server’s configuration. It eases the process of writing tests by allowing the user to generate tests from the current system state. Once the test suite is written they can be executed, waited-on, or served as a health endpoint.

More info can be found on the Goss webpage.

Example goss.yaml test file:

The above goss.yaml file can also be generated by using the following commands on a configured system:

goss add dns mysql

# "a" is short for "add"
goss a addr mysql:3306
goss a command '/goss/checks/db_login.sh'

Once we have a test suite, we can:

• run it once

goss validate

• keep running it until the system enters a valid state or we timeout

goss validate --retry-timeout 30s --sleep 1s

• serve the tests as a health endpoint

goss serve &
curl localhost:8080/healthz

# JSON endpoint
goss serve --format json &
curl localhost:8080/healthz

Docker Challenge: Running more than one HEALTHCHECK command

Docker 1.12 introduced the concept of HEALTHCECK, but only allows you to run one command. By letting Goss manage your checks. The goss.yaml file can define as many checks or reference as many scripts/commands as you like.

FROM alpine:3.4

# Install Goss
RUN apk add --no-cache --virtual=goss-dependencies curl ca-certificates && \
    curl -fsSL https://goss.rocks/install | sh && \
    apk del goss-dependencies

# Add our checks
ADD db_login.sh /goss/checks/db_login.sh
ADD goss.yaml /goss/goss.yaml

# Use goss to run all our other checkes
HEALTHCHECK --interval=1s --timeout=6s CMD goss -g /goss/goss.yaml validate

You can view the container health using the following commands:

# View health check status
docker ps container_name

# Print out the text of the last 5 checks
docker inspect -f '{{ range .State.Health.Log }}{{ println "======\nStart:" .Start }}{{ .Output }}{{end}}' container_name

Docker Challenge: Delaying container startup until dependent services are up and running

If your application can’t gracefully handle missing dependent services, wrap the CMD in a script that calls Goss validate with a retry.

# Alternatively, the -r option can be set
# using the GOSS_RETRY_TIMEOUT env variable
CMD goss -g /goss/goss.yaml validate -r 5m && exec my_command..

Kubernetes Challenge: Monitoring 3rdParty applications that do not provide a /healthz endpoint

Create a multi-container pod and use Goss as a /healthz endpoint. Set the /healthz endpoint as a readiness and/or liveliness probe to control traffic and/or container status.

To create a healthz container:

• Create goss.yaml and optionally a checks folder in the following structure:

.
|-- Dockerfile
`-- goss/
    |-- checks/
    |   `-- db_login.sh
    `-- goss.yaml

• Use the goss:onbuild image to build our healtz image:

FROM goss:onbuild
CMD ["serve"]

Example of a mysql deployment with /healthz endpoint:

Kubernetes Challenge: Ensure container waits for dependencies to before starting up

Use the goss:onbuild image to create an image that waits for certain conditions to be met (perhaps another service’s /heathz endpoint). Set it as an init-container. For more information on init-containers, see the Kubernets docs here and here.

Example Dockerfile:

FROM goss:onbuild
CMD ["validate", "--retry-timeout", "5m"]

Parting thoughts

Check out Kelsey Hightower’s great talk at Monitorama to see some of the above patterns in action.

If you found this article informative, please recommend it to others by clicking the heart.