In this article, I’m going to show you how to fetch private Cargo dependencies and source them when building a Docker image. This solves a key issue with Docker of not copying over SSH keys when building an image

The code for this blog post can be found in the Github repository below

c-ameron/rocket-add

Example Application: Rocket-Add

I’ve developed an example API called Rocket-Add which is built on top of a very cool web framework in Rust called Rocket

This application extends the ‘Hello World’ example with a new API call add/<num1>/<num2> which adds two numbers and outputs the result

$ curl localhost:8000/add/5/10
The sum of 5 and 10 is 15

The API endpoint is written as a function

Looking closely, you can see the addition calls an external library math_utils with the call math_utils::add(num1, num2)

The math_utils crate is from a private Github repository https://github.com/c-ameron/rust-math-utils.git

For reference, this is the code for the add function in math_utils being used

Fetching crates in private repositories

Before we go any further, we’re going to take a page from Ruby Bundler’s book and create a local .cargo folder in the project root. This will provide a project-independent place to store our crates and Cargo config

git clone git@github.com:c-ameron/rocket-add.git
cd rocket-add
mkdir -p $(git rev-parse --show-toplevel)/.cargo
export CARGO_HOME=$(git rev-parse --show-toplevel)/.cargo

Currently Cargo by default won’t fetch from private repositories

To get around this, create a .cargo/config file which tells Cargo to use the git cli for fetching

$ cat .cargo/config
[net]
git-fetch-with-cli = true

Then in our Cargo.toml we tell Cargo to fetch our crate with SSH

math_utils = { version = "0.1.0", git = "ssh://git@github.com/c-ameron/rust-math-utils.git"}

Now we can run cargo fetch and it will download all the crates to the local .cargo folder

Building a Rust Docker image with private dependencies

A frequently encountered problem when building a Docker image is downloading private dependencies without mounting an SSH key

To solve this problem, we can copy over our pre-fetched .cargo folder to the Docker image and build from there. Now Cargo won’t need to use any SSH keys to fetch from private repositories

Note that we’re having to set the WORKDIR and CARGO_HOME inside the Docker image for Cargo to resolve its crates correctly

We can also take advantage of multi-stage builds in Docker, and create a smaller binary image

This approach saves time downloading the same crates each build and we also don’t need to pass any SSH keys to Docker. However, if there’s any change to our dependencies it will require a rebuild. For a larger application with a lot of changing dependencies, this often means rebuilding from scratch several times a day

Notes

• Docker has implemented experimental support for SSH forwarding when building images in 18.09, potentially negating the need for copying .cargo in the future
• If you want to fetch or build an image completely offline with Cargo, you can use the -Z offline flag. See https://doc.rust-lang.org/cargo/reference/unstable.html#offline-mode
• With recently released Rust 1.34, Cargo now has support for alternative registries other than crates.io