Publishing Docker image to Dockerhub using Automated builds

Published March 8, 2019 in   docker, dockerhub, sass, multi-stage, automation, labels, label-schema.org

Although there are some resources on how to create, label and publish images to hub.docker.com, I have found myself in need to put all those pieces together. This article is the complete example of how to create, build and test, label and publish docker image using Automated Builds with source code hosted at github.com.

In this article we will:

  • create small docker image, build it and test locally
  • create and link hub.docker.com repository with github repository
  • setup automated builds of the image
  • alter the automated builds setup to include build-time labels
  • verify the build

Requirements

First of all, we need to:

Create the docker image

As an example software to bundle as a docker image, we will use SASS. At the time of writing, I was unable to find any docker image, which would distribute the dart-sass binary version. This may be, cause the ruby version of SASS is officially deprecated and will become unmaintained. Maybe majority of the world use sassc+libsass which already has its docker version. Nevertheless, I decided to create my own small docker image (~80MB) and publish it.

Dockerfile

We will create a two stage Dockerfile as recommended by dockerfile best practices. See the multistage builds for more information.

Dockerfile:

FROM ubuntu:18.04 as build
ADD https://github.com/sass/dart-sass/releases/download/1.17.2/dart-sass-1.17.2-linux-x64.tar.gz /opt/
RUN tar -C /opt/ -xzvf /opt/dart-sass-1.17.2-linux-x64.tar.gz

FROM ubuntu:18.04 as final

COPY --from=build /opt/dart-sass /opt/dart-sass

CMD [ "/opt/dart-sass/sass", "--watch", "/sass:/css" ]

This dockerfile first uses ubuntu:18.04 as a build stage, in which it just downloads the binary distribution of dart-sass and unpacks its contents to /opt/dart-sass directory.

This build stage is then thrown away and only the /opt/dart-sass directory and its contents are embedded into final docker image using COPY instruction.

The run command for this image is the one, which a web developer usually runs locally:

/opt/dart-sass/sass --watch /sass:/css

The docker image is designed, to be run by a web developer using bind mounted directories:

  • <source code>/sass (host machine) —> /sass (docker container)
  • <source code>/css (host machine) —> /css (docker container)

This way, the sass binary inside docker container will watch for changes from host machine, rebuild the CSS files, which propagate back to host machine.

The appropriate run command for this image would be:

docker run -v $PWD/sass:/sass/ -v $PWD/css:/css/ --init -it michalklempa/dart-sass:latest

Lets build the image:

docker build -t michalklempa/dart-sass .
Sending build context to Docker daemon  2.048kB
Step 1/6 : FROM ubuntu:18.04 as build
 ---> 47b19964fb50
Step 2/6 : ADD https://github.com/sass/dart-sass/releases/download/1.17.2/dart-sass-1.17.2-linux-x64.tar.gz /opt/
Downloading [==================================================>]  11.84MB/11.84MB
 ---> db91f4f2622b
Step 3/6 : RUN tar -C /opt/ -xzvf /opt/dart-sass-1.17.2-linux-x64.tar.gz
 ---> Running in 0015f971339d
dart-sass/src/dart
dart-sass/src/DART_LICENSE
dart-sass/src/sass.dart.snapshot
dart-sass/src/SASS_LICENSE
dart-sass/dart-sass
dart-sass/sass
Removing intermediate container 0015f971339d
 ---> 42ab633cad2e
Step 4/6 : FROM ubuntu:18.04 as final
 ---> 47b19964fb50
Step 5/6 : COPY --from=build /opt/dart-sass /opt/dart-sass
 ---> 26645a730c07
Step 6/6 : CMD [ "/opt/dart-sass/sass", "--watch", "/sass:/css" ]
 ---> Running in dc5d4299b299
Removing intermediate container dc5d4299b299
 ---> 1dda340b7fe8
Successfully built 1dda340b7fe8
Successfully tagged michalklempa/dart-sass:latest

To test the image, we will need to create two directories: sass and css and create at least sass/main.sass file:

mkdir -p css
mkdir -p sass
cat >./sass/main.sass <<'EOL'
$font-stack:    Helvetica, sans-serif
$primary-color: #333

body
  font: 100% $font-stack
  color: $primary-color
EOL

And test image locally:

docker run -v $PWD/sass:/sass/ -v $PWD/css:/css/ --init -it michalklempa/dart-sass:latest

Expected output is:

Compiled sass/main.sass to css/main.css.
Sass is watching for changes. Press Ctrl-C to stop.

And the generated CSS file css/main.css:

body {
  font: 100% Helvetica, sans-serif;
  color: #333;
}

/*# sourceMappingURL=main.css.map */

Now that we have our image, we can push it to github repository and set-up the link between docker hub and github.

Create a github repository

Lets create a github.com repository named docker-dart-sass and make it public so the hub.docker.com could access the source files of our repository.

As we already have done some work, we do not clone the reposity, just link our directory with it.

Initialiaze empty repository in our current directory:

git init

and add remote:

git remote add origin git@github.com:michalklempa/docker-dart-sass.git

Now we can checkout master branch:

git checkout master

Add our files:

git add Dockerfile
git commit -m"Our dockerfile"
git push -u origin master

Setup Automated Build

At dockehub settins link your account to github account. At the builds tab of the docker repository settings page, set up Automated Builds.

Automated Builds Setup
Automated Builds Setup

We should see a build being done:

Building in Docker Cloud's infrastructure...
Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '192.30.253.113' to the list of known hosts.
Reset branch 'master'
Your branch is up-to-date with 'origin/master'.
KernelVersion: 4.4.0-1060-aws
Components: [{u'Version': u'18.03.1-ee-3', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.4.0-1060-aws', u'Os': u'linux', u'BuildTime': u'2018-08-30T18:42:30.000000000+00:00', u'ApiVersion': u'1.37', u'MinAPIVersion': u'1.12', u'GitCommit': u'b9a5c95', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.10.2'}}]
Arch: amd64
BuildTime: 2018-08-30T18:42:30.000000000+00:00
ApiVersion: 1.37
Platform: {u'Name': u''}
Version: 18.03.1-ee-3
MinAPIVersion: 1.12
GitCommit: b9a5c95
Os: linux
GoVersion: go1.10.2
Starting build of index.docker.io/michalklempa/dart-sass:latest...
Step 1/13 : FROM ubuntu:18.04 as build
---> 47b19964fb50
Step 2/13 : ADD https://github.com/sass/dart-sass/releases/download/1.17.2/dart-sass-1.17.2-linux-ia32.tar.gz /opt/
...

If everything went well, we can now pull the image and inspect it:

docker pull michalklempa/dart-sass:latest
docker inspect michalklempa/dart-sass

Search for the Config.Labels:

...
 "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/opt/dart-sass/sass",
                "--watch",
                "/sass:/css"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:26645a730c0736c512fd65bf4f61355fd10321f8ef002f5fb0322e7091ed8ae8",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
...

As we can see, labes are empty at the moment. Labels add metadata to the image which are used also by hub.docker.com user interface (e.g. repository url).

Add metadata to image using labels

Some set of labels and the steps needed are described in Ross Fairbanks blog on labels together with the code. We will do basically same steps and add more labels. For those interested, look at label-schema.org for full list of labels.

Great example to start with is Dockerfile made by Justin J. Novack:

FROM scratch

ARG BRANCH="none"
ARG COMMIT="local-build"
ARG DATE="1970-01-01T00:00:00Z"
ARG URL="https://github.com/jnovack/dockerhub-hooks"
ARG VERSION="dirty"

LABEL org.label-schema.schema-version="1.0" \
    org.label-schema.build-date=$DATE \
    org.label-schema.vendor="Justin J. Novack" \
    org.label-schema.name="jnovack/dockerhub-hooks" \
    org.label-schema.description="Sample Docker Hub Build Hooks" \
    org.label-schema.version="$VERSION" \
    org.label-schema.vcs-url=$URL \
    org.label-schema.vcs-branch=$BRANCH \
    org.label-schema.vcs-ref=$COMMIT

ENV BRANCH "$BRANCH"
ENV COMMIT "$COMMIT"
ENV DATE "$DATE"
ENV VERSION "$VERSION"

We will extend this set of labels with:

    org.label-schema.url="https://hub.docker.com/r/michalklempa/dart-sass" \
    org.label-schema.docker.cmd="docker run -v $PWD/sass:/sass/ -v $PWD/css:/css/ --init -it michalklempa/dart-sass:latest"

and also add MAINTAINER to our Dockerfile:

FROM ubuntu:18.04 as build
ADD https://github.com/sass/dart-sass/releases/download/1.17.2/dart-sass-1.17.2-linux-x64.tar.gz /opt/
RUN tar -C /opt/ -xzvf /opt/dart-sass-1.17.2-linux-x64.tar.gz

FROM ubuntu:18.04 as final
ARG BRANCH
ARG COMMIT
ARG DATE
ARG URL
ARG VERSION

MAINTAINER michal.klempa@gmail.com

LABEL org.label-schema.schema-version="1.0" \
    org.label-schema.build-date=$DATE \
    org.label-schema.vendor="Michal Klempa" \
    org.label-schema.name="michalklempa/dart-sass" \
    org.label-schema.description="sass/dart-sass docker image for web development purposes. Runs sass --watch on provided volumes." \
    org.label-schema.url="https://hub.docker.com/r/michalklempa/dart-sass" \
    org.label-schema.docker.cmd="docker run -v $PWD/sass:/sass/ -v $PWD/css:/css/ --init -it michalklempa/dart-sass:latest" \
    org.label-schema.version="$VERSION" \
    org.label-schema.vcs-url=$URL \
    org.label-schema.vcs-branch=$BRANCH \
    org.label-schema.vcs-ref=$COMMIT

COPY --from=build /opt/dart-sass /opt/dart-sass

CMD [ "/opt/dart-sass/sass", "--watch", "/sass:/css" ]

Now building this image will require us to specify multiple --build-arg arguments. To achieve this arguments appear on commandline during automated build, we will override a build hook. Create file ./hooks/build and put the great hook from Justin there:

#!/bin/bash
# hooks/build
# https://docs.docker.com/docker-cloud/builds/advanced/

# $IMAGE_NAME var is injected into the build so the tag is correct.
echo "[***] Build hook running"

docker build \
  --build-arg VERSION=$(git describe --tags --always) \
  --build-arg COMMIT=$(git rev-parse HEAD) \
  --build-arg URL=$(git config --get remote.origin.url) \
  --build-arg BRANCH=$(git rev-parse --abbrev-ref HEAD) \
  --build-arg DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ") \
-t $IMAGE_NAME .

Now commit and push the changes. We should see the metadata added on dockerhub also on microbadger.com Or we can re-pull the image and inspect it again:

docker pull michalklempa/dart-sass
docker inspect michalklempa/dart-sass

...
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "org.label-schema.build-date": "2019-03-08T15:57:06Z",
                "org.label-schema.description": "sass/dart-sass docker image for web development purposes. Runs sass --watch on provided volumes.",
                "org.label-schema.docker.cmd": "docker run -v /sass:/sass/ -v /css:/css/ --init -it michalklempa/dart-sass:latest",
                "org.label-schema.name": "michalklempa/dart-sass",
                "org.label-schema.schema-version": "1.0",
                "org.label-schema.url": "https://hub.docker.com/r/michalklempa/dart-sass",
                "org.label-schema.vcs-branch": "master",
                "org.label-schema.vcs-ref": "dfc9fec637a677aaa267a290d267c9afda0ed6d4",
                "org.label-schema.vcs-url": "git@github.com:michalklempa/docker-dart-sass.git",
                "org.label-schema.vendor": "Michal Klempa",
                "org.label-schema.version": "v1.17.2-02-1-gdfc9fec"
            }
        },
        "Architecture": "amd64",
        "Os": "linux",
...

Conclusion

We have created and published our Docker image using Automated builds with properly assigned labels on it. All the source code is available at github.com/michalklempa/docker-dart-sass under GPLv3. The docker image is available at hub.docker.com/r/michalklempa/dart-sass.

Any comments are welcome.

Note 1: We have not been discussing git tagging the source repository to generate corresponding tags in automated builds. However, example source repository and automated builds rule are set properly, to accomodate git tags in format: v1.0.0-06 into proper docker build tags. Version format is similar to one used in Debian packaging (v<upstream>-<package_maintainer_version> see deb-version)

Posted by Michal Klempa on 8 March 2019
automation, docker, dockerhub, label-schema.org, labels, multi-stage, sass