Running Ansible from inside Docker image for CI/CD pipeline

In this article we prepare simple Docker image packed with our Ansible roles, which will be ready-made for provisioning just by running the container from this image.

In this article we describe process of encapsulating ansible executable, Ansible roles, dependent galaxy roles, SSH key material and group variables into a docker image for CI/CD use. We also present a way to run prepared image from command-line without installing Ansible.

Introduction

Often times, the CI/Cd pipeline needs to install, deploy or update configuration on some remote machine. For example to reflect the changes in source code after compile, we may want the development cloud-base environment to run latest artifacts.

In the time of Docker, the compile and tests phases are carried out by docker builds, lets put a light on how to encapsulate Ansible into docker image, so the deployment phase can also be carried out by docker run.

Requirements

A working docker installation is required. Ansible or python is not required, as all Ansible related dependencies will be installed inside a docker image.

Prepare basic Ansible structure

Lets prepare basic Ansible structure. Create directory tree structure like this:

.
├── ansible/
│   ├── ansible.cfg
│   ├── base.yml
│   ├── group_vars/
│   │   └── all.yml
│   ├── inventories/
│   └── roles/
└── requirements.yml

Lets suppose we want each and every of our servers to have NTP daemon set up and configured with our timezone. For this purpose, lets re-use the galaxy role geerlingguy/ansible-role-ntp. To add this role into dependencies, we edit the ./requirements.yml:

- src: git+https://github.com/geerlingguy/ansible-role-ntp.git
  version: master
  name: geerlingguy/ansible-role-ntp

But we do not download or setup the required role on our computer. It will be downloaded during docker image build later.

We may want to setup some configuration in ansible/ansible.cfg, but it is not a necessity (just here to demonstrate it can be done).

In ansible/group_vars/all.yml we have put variables for ntp role and ssh extra args:

ansible_ssh_extra_args: "-o 'PasswordAuthentication no' -o 'IdentitiesOnly yes'"
ntp:
  timezone: Europe/Bratislava
  area: europe
  manage_config: true

And finally the playbook ansible/base.yml contains steps to run ntp role on every host:

---
- hosts: all
  become: True
  roles:
    - role: geerlingguy/ansible-role-ntp
      tags: ntp

Basic Docker image for Ansible roles

Maybe you wondered why the directory structure included ansible related files in a separate subdirectory. It is because we will put a Dockerfile into root project directory.

Lets create./Dockerfile with some minimal software needed:

FROM alpine:3.11
RUN apk add --no-cache openssh-client ansible git

You probably have some ssh key which should be able to access destination host, or if not, create ssh key.

Add you id_rsa and id_rsa.pub into a subdirectory ./docker in the project. So the directory structure would be:

.
├── ansible/
│   ├── ansible.cfg
│   ├── base.yml
│   ├── group_vars/
│   │   └── all.yml
│   ├── inventories/
│   └── roles/
├── docker/
│   ├── id_rsa
│   └── id_rsa.pub
├── Dockerfile
└── requirements.yml

Next we add the SSH key files into a docker image:

FROM alpine:3.11
RUN apk add --no-cache openssh-client ansible git

RUN mkdir -p /root/.ssh
COPY ./docker/id_rsa /root/.ssh/id_rsa
COPY ./docker/id_rsa.pub /root/.ssh/id_rsa.pub

RUN chmod 600 /root/.ssh/id_rsa \
 && chmod 640 /root/.ssh/id_rsa.pub \
 && echo "Host *" > /root/.ssh/config && echo " StrictHostKeyChecking no" >> /root/.ssh/config

The setup is very straightforward, just copy the key id_rsa, public key id_rsa.pub into root users home just as it would be done by ssh-keygen (mind proper modes).

The StrictHostKeyChecking is here as quick-and-dirty solution for this article. In real world, you should supply baked known_hosts file or maybe you will use dynamic inventories.

Finally we install role with ansible-galaxy as a part of docker image build and we copy our source files into docker image:

FROM alpine:3.11
RUN apk add --no-cache openssh-client ansible git

RUN mkdir -p /root/.ssh
COPY ./docker/id_rsa /root/.ssh/id_rsa
COPY ./docker/id_rsa.pub /root/.ssh/id_rsa.pub

RUN chmod 600 /root/.ssh/id_rsa \
 && chmod 640 /root/.ssh/id_rsa.pub \
 && echo "Host *" > /root/.ssh/config && echo " StrictHostKeyChecking no" >> /root/.ssh/config

COPY ./requirements.yml /ansible/requirements.yml

RUN ansible-galaxy install -n -p /ansible/roles -r /ansible/requirements.yml --ignore-errors

COPY ./ansible /ansible

WORKDIR /ansible

CMD [ "" ]

Build the image locally:

$ 
docker build -t michalklempa/ansible-base .

Sending build context to Docker daemon  63.49kB
Step 1/11 : FROM alpine:3.11
 ---> f70734b6a266
Step 2/11 : RUN apk add --no-cache openssh-client ansible git
 ---> Running in 970d979bedba
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/35) Installing libbz2 (1.0.8-r1)
(2/35) Installing expat (2.2.9-r1)
...
(34/35) Installing libedit (20191211.3.1-r0)
(35/35) Installing openssh-client (8.1_p1-r0)
Executing busybox-1.31.1-r9.trigger
Executing ca-certificates-20191127-r1.trigger
OK: 222 MiB in 49 packages
Removing intermediate container 970d979bedba
 ---> e7e9b5799345
Step 3/11 : RUN mkdir -p /root/.ssh
 ---> Running in cae695d22cea
Removing intermediate container cae695d22cea
 ---> 0528b467edcf
Step 4/11 : COPY ./docker/id_rsa /root/.ssh/id_rsa
 ---> 7e07a691e824
Step 5/11 : COPY ./docker/id_rsa.pub /root/.ssh/id_rsa.pub
 ---> e9545a923de6
Step 6/11 : RUN chmod 600 /root/.ssh/id_rsa  && chmod 640 /root/.ssh/id_rsa.pub  && echo "Host *" > /root/.ssh/config && echo " StrictHostKeyChecking no" >> /root/.ssh/config
 ---> Running in 93f285c12848
Removing intermediate container 93f285c12848
 ---> 5eb4a72b2f8d
Step 7/11 : COPY ./requirements.yml /ansible/requirements.yml
 ---> 3ba02c827420
Step 8/11 : RUN ansible-galaxy install -n -p /ansible/roles -r /ansible/requirements.yml --ignore-errors
 ---> Running in f070091b3450
- extracting geerlingguy/ansible-role-ntp to /ansible/roles/geerlingguy/ansible-role-ntp
- geerlingguy/ansible-role-ntp (master) was installed successfully
Removing intermediate container f070091b3450
 ---> 48306a66c4fc
Step 9/11 : COPY ./ansible /ansible
 ---> 6c4934752e47
Step 10/11 : WORKDIR /ansible
 ---> Running in ced43a736a88
Removing intermediate container ced43a736a88
 ---> 013428a6fdca
Step 11/11 : CMD [ "" ]
 ---> Running in c8ecf9dcfb23
Removing intermediate container c8ecf9dcfb23
 ---> b0a80602361d
Successfully built b0a80602361d
Successfully tagged michalklempa/ansible-base:latest

Test docker image

To test our new docker image, we need some inventory. Create a servers.yml file in some other directory:

all:
  hosts:
    example-01.michalklempa.com:
      ansible_host: "161.35.67.27"
      ansible_user: "root"

We will run Ansible docker image and provide the inventory as a bind mount into the container:

$ 
docker run -t -v ${PWD}/servers.yml:/ansible/inventories/servers.yml \
michalklempa/ansible-base ansible -i inventories/servers.yml -m ping all

example-01.michalklempa.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}

The arguments are:

-t to provide tty, this keeps nice coloring of Ansible output (in automated scripts, like Jenkins, you want this turned off)
-v volume (more precisely bind mount) with the inventory YAML from host into docker container
michalklempa/ansible-base image name
ansible -i inventories/servers.yml -m ping all command and arguments to run in container.

We can try ansible-playbook:

$ 
docker run -t -v ${PWD}/servers.yml:/ansible/inventories/servers.yml \
michalklempa/ansible-base ansible-playbook -i inventories/servers.yml base.yml

PLAY [all] ******************************************************************

TASK [Gathering Facts] ******************************************************
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Include OS-specific variables.] ********
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Include OS-Release specific variables on RHEL 6.]
skipping: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Set the ntp_package variable.] *********
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Set the ntp_config_file variable.] *****
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Ensure NTP package is installed.] ******
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Ensure tzdata package is installed (Linux).]
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : include_tasks] *************************
skipping: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Set timezone] **************************
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Ensure NTP is running and enabled as configured.]
ok: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Ensure NTP is stopped and disabled as configured.]
skipping: [example-01.michalklempa.com]

TASK [geerlingguy/ansible-role-ntp : Generate ntp configuration file] ******
skipping: [example-01.michalklempa.com]

PLAY RECAP *****************************************************************
example-01.michalklempa.com : 
ok=8    changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0

Add your own role

So far we worked with existing role from Ansible Galaxy. What if we want to incorporate our own role into the setup? Lets create some sample role for the purpose of demonstration. Our role will be a generic package install role to populate our server with favorite package we like to use.

Create directory: roles/michalklempa/packages structure for new role:

mkdir -p roles/michalklempa/packages/defaults
mkdir -p roles/michalklempa/packages/tasks

The defaults/main.yml file will contain an empty list of packages to install and to remove:

---
packages:
  present:
  remove:

The role will define only two tasks, one to install packages and one to remove (tasks/main.yml):

---
- name: "Install Packages"
  package:
    name: ""
    state: present
  become: true

- name: "Remove Packages"
  package:
    name: ""
    state: absent
  become: True

Tree structure is now:

.
├── ansible
│   ├── ansible.cfg
│   ├── base.yml
│   ├── group_vars
│   │   └── all.yml
│   ├── inventories
│   └── roles
│       └── michalklempa
│           └── packages
│               ├── defaults
│               │   └── main.yml
│               └── tasks
│                   └── main.yml
├── docker
│   ├── id_rsa
│   └── id_rsa.pub
├── Dockerfile
└── requirements.yml

We need to alter the base.yml to add our new role into a playbook plan:

---
- hosts: all
  become: True
  roles:
    - role: michalklempa/packages
      tags: packages
    - role: geerlingguy/ansible-role-ntp
      tags: ntp

Rebuild the docker image:

docker build -t michalklempa/ansible-base .

To test, we extend the inventory file servers.yml to provide variable value to install for example, package htop:

all:
  hosts:
    example-01.michalklempa.com:
      ansible_host: "161.35.67.27"
      ansible_user: "root"
  vars:
    packages:
      present:
        - htop

No run the dockerized ansible:

docker run -t -v ${PWD}/servers.yml:/ansible/inventories/servers.yml \
michalklempa/ansible-base ansible-playbook -i inventories/servers.yml base.yml --tags packages

PLAY [all] *************************************************

TASK [Gathering Facts] *************************************
ok: [example-01.michalklempa.com]

TASK [michalklempa/packages : Install Packages] ************
changed: [example-01.michalklempa.com]

TASK [michalklempa/packages : Remove Packages] *************
ok: [example-01.michalklempa.com]

PLAY RECAP *************************************************
example-01.michalklempa.com : 
ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

You can start extending the example repository with some basic roles, which is available on github: docker-ansible-base

Set up bash alias to run dockerized Ansible

Although for running dockerized Ansible in scripts the setup describe above is sufficient, one can also run the docker image from local machine. To make this more convenient, we provide a few lines to put into your ~/.bashrc file:

function ansible() {
        docker run -t ${1} ansible ${@:2}
}
function ansibleplaybook() {
        docker run -t ${1} ansible-playbook ${@:2}
}
alias ansible-playbook="ansibleplaybook"

This provides a way to execute simply:

ansible michalklempa/ansible-base -i inventories/servers.yml -m ping all

This assumes, you build your inventory into a docker image.

Conclusion

All the project files and roles are available in github repository.