Category: Docker

Creating a tiny Docker image of a Rust project

I am building a toy project in Rust to help me learn how to deploy things in AWS. I’m considering using Elastic Beanstalk (AWS’s platform-as-a-service) and also Kubernetes. Both of these support deploying via Docker containers, so I am learning how to package a Rust executable as a Docker image.

My program is a small web site that uses Redis as a back end database. It consists of some Rust code and a couple of static files.

Because Rust has good support for building executables with very few dependencies, we can actually build a Docker image with almost nothing in it, except my program and the static files.

Thanks to Alexander Brand’s blog post How to Package Rust Applications Into Minimal Docker Containers I was able to build a Docker image that:

  1. Is very small
  2. Does not take too long to build

The main concern for making the build faster is that we don’t download and build all the dependencies every time. To achieve that we make sure there is a layer in the Docker build process that includes all the dependencies being built, and is not re-built when we only change our source code.

Here is the Dockerfile I ended up with:

# 1: Build the exe
FROM rust:1.42 as builder
WORKDIR /usr/src

# 1a: Prepare for static linking
RUN apt-get update && \
    apt-get dist-upgrade -y && \
    apt-get install -y musl-tools && \
    rustup target add x86_64-unknown-linux-musl

# 1b: Download and compile Rust dependencies (and store as a separate Docker layer)
RUN USER=root cargo new myprogram
WORKDIR /usr/src/myprogram
COPY Cargo.toml Cargo.lock ./
RUN cargo install --target x86_64-unknown-linux-musl --path .

# 1c: Build the exe using the actual source code
COPY src ./src
RUN cargo install --target x86_64-unknown-linux-musl --path .

# 2: Copy the exe and extra files ("static") to an empty Docker image
FROM scratch
COPY --from=builder /usr/local/cargo/bin/myprogram .
COPY static .
USER 1000
CMD ["./myprogram"]

The FROM rust:1.42 as build line uses the newish Docker feature multi-stage builds – we create one Docker image (“builder”) just to build the code, and then copy the resulting executable into the final Docker image.

In order to allow us to build a stand-alone executable that does not depend on the standard libraries in the operating system, we use the “musl” target, which is designed to be statically linked.

The final Docker image produced is pretty much the same size as the release build of myprogram, and the build is fast, so long as I don’t change the dependencies in Cargo.toml.

A couple more tips to make the build faster:

1. Use a .dockerignore file. Here is mine:

/target/
/.git/

2. Use Docker BuildKit, by running the build like this:

DOCKER_BUILDKIT=1 docker build  .

Build with a different Java version (e.g. 11) using Docker

To spin up a temporary environment with a different Java version without touching your real environment, try this Docker command:

docker run -i -t --mount "type=bind,src=$PWD,dst=/code" openjdk:11-jdk bash

(Change “11-jdk” to the version you want as listed on the README.)

Then you can build the code inside the current directory something like this:

cd code
./gradlew test

Or similar for other build tools, although you may need to install them first.

Run bash inside any version of Linux using Docker

Docker is useful for some things, and not as useful as you think for others.

Here’s something massively useful: get a throwaway bash prompt inside any version of any Linux distribution in one command:

docker run -i -t --mount "type=bind,src=$HOME/Desktop,dst=/Desktop" ubuntu:18.10 bash

This command downloads a recent Ubuntu 18.10 image, mounts my desktop as /Desktop in the container, and gives me a bash prompt. From here I can install any packages I want and then use them.

For example, today I used it to decrypt a file that was encrypted with a cipher my main OS did not have a package for.

When I exit bash, the container stops and I can find it with docker ps -a then remove it with docker rm. To really clean up I can find the downloaded images with docker image ls and remove them with docker image rm.

ZX Spectrum BASIC Web Server

Finally, you can write your web sites in ZX Spectrum BASIC, using my new project, ZX Spectrum BASIC CGI server .

How it works

Here’s what happens when a request comes in:

  • Apache HTTPD receives the request, and runs the CGI script that does the magic.
  • The CGI script (written in Bash) generates some BASIC code that provides the HTTP meta-vars (e.g. PATH_INFO for the path of the request) as DATA statements and wraps some other boilerplate around the code of the .basic file asked for in the request, and writes out another .basic file which is ready to be executed.
  • Then it uses BAS2TAP (my mirror here) to create an audio tape file representing the program.
  • Next it launches the Fuse spectrum emulator (inside an Xvfb wrapper to make it think it has a display), telling it to load from the audio tape and run the program.
  • The program runs, and writes its HTTP responses to the Spectrum printer using LPRINT statements.
  • Fuse uses very basic OCR to understand what characters are printed, and writes them into a text file.
  • The CGI script monitors the output file, waiting for a line containing an end marker (“ENDSPECTRUMBASIC”). When it finds it, it kills the emulator (since it can’t made to auto-exit, I think).
  • The CGI script sends the contents of the output file (minus the end marker) back to Apache, which returns it as a response.

Simple.

Why?

Originally designed to demonstrate how Docker can help isolate other services from “interesting” dependencies, this became a bit of a labour of love.

I’ll never stop being grateful to the designers of the ZX Spectrum, or the authors of “Further Programming for the ZX Spectrum”.

Changing the Docker daemon options in systemd on Ubuntu 15.04

Update: now documenting the better way, as described in issue 14513.

Update 2: I think this way is even better (works in Ubuntu 16.04, Docker 1.12.2, Reference: dockerd command line):

sudo -s -H
echo '{"insecure-registries":["myreg.example.com:5000"]}' > /etc/docker/daemon.json
exit
sudo service docker restart

End of update 2

On earlier versions of Ubuntu (14.04 and before), changing the command line options to the Docker daemon (e.g. to allow using an insecure private registry) was just a matter of editing /etc/default/docker and uncommenting the line starting with #DOCKER_OPTS=.

On Ubuntu 15.04, even though that file still exists, editing it does not have any effect, and I found it quite tricky to work out how to change Docker’s command line, so I wrote it up here.

I wanted to use an insecure private docker registry for Docker on Ubuntu 15.04, which uses systemd.

Under systemd, we must create a config file that overrides the default Docker command line by typing sudo systemctl edit docker. In the editor which pops up, type:

[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --insecure-registry=myreg.example.com:5000

Note: the first “ExecStart=” line is necessary.

The second “ExecStart=” line should contain whatever command line options you want to include.

Note: for docker versions before 1.10, replace “daemon” with “-d”.

With this config in place, restart the Docker service:

$ sudo systemctl restart docker

Check everything looks right for the Docker service:

$ systemctl status docker

And confirm the command line arguments have been applied with:

$ ps axwww | grep /usr/bin/docker

The instructions here: Control and configure Docker with systemd and issues: 14513 and 15859 suggest that the Docker team are not planning to make this any easier in the short term.