Continuous Integration with Jenkins and Docker

Continuous Integration is the corner stone of development these days. A right CI tool will ensure that code written is suitable for deployment. Several organizations run a CI suite comprising of tests and build steps on every commit or a merge to their production branch. This ensures that the code is of high quality and is always ready to deploy. Jenkins is a very popular Continuous Integration tool used by small and large organizations alike and in this lab we will see how we can build Docker images and deploy Docker containers.

Installing Jenkins

Jenkins can be installed easily using the rpm or deb package. Since we have been using CentOS, we would use the rpm package. So let us install Java and add the Jenkins repository.

$ sudo yum -y install wget git java-1.8.0-openjdk
$ sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
$ sudo rpm --import http://pkg.jenkins-ci.org/redhat-stable/jenkins-ci.org.key
$ sudo yum install jenkins

Running Jenkins CI and Docker

Let us start the Jenkins service and see if we can browse the Jenkins UI

$ sudo systemctl start jenkins

Now open the browser and go to the port 8080 of the Jenkins server and you should see this. Jenkins welcome

For Docker, we have to make the daemon listen on a port. We should also add jenkins user to docker group. This will let Jeknins fire docker commands.

$ sudo docker daemon -H :1234
$ sudo usermod -G docker -a jenkins

Using Jenkins to Deploy Docker

We can use Jenkins to build, test and deploy Docker containers.

Defining a Continuous Integration workflow

Let us define three stages of our CI pipeline.

  1. Fetch the code and run basic tests
  2. If stage 1 passes, build the Docker image
  3. If stage 2 passes, deploy the Docker image to run Docker container.

In reality, these pipelines can be complex involving multiple testing stages, both before and after building or compiling code. For this pipeline, we will start with creation of these stages individually.

Creating and configuring Jenkins Jobs

To create a new job, on the welcome screen, click on the New Item link on the sidebar. This will open a page with several options. Here we will choose to create a Freestyle project like in the image below. enter image description here

After choosing the Freestyle project, Jenkins will ask us to configure various parameters of the job. We will name the job and write a suitable description. In Build section, we will click on Add build step and then select Execute shell. Anything that we write here will be executed on the shell. For this stage, we need to fetch the code and verify that we have a Dockerfile. Put the lines below in the Command text area of Execute shell section.

rm -rf docker-owncloud-orchestration
git clone https://github.com/training-devops/docker-owncloud-orchestration.git
stat docker-owncloud-orchestration/Dockerfile/Dockerfile

enter image description here Jenkins uses a different workspace for each job that is created. Since we are going in work in stages, we want the workspace to be the same. To override the default workspace option, click on the Advanced... button in Advanced Project Options section and select Use custom workspace and set the Directory as docker-ci. Now we will use this workspace for all the stages.

We will create the next stages similarly. We will set the workspace as docker-ci. In stage2, we will build the Docker image. The commands for the Execute shell section are: Stage 2

cd docker-owncloud-orchestration/Dockerfile/
docker -H :1234 build -t oc9-ci -f Dockerfile .

We want this stage to trigger automatically once the stage 1 has finished successfully. For that, we will select Build after other projects are built option in Build Triggers section and write the name of the stage 1.

Similarly, we will build the stage 3 with the shared workspace and stage 2 Build Triggers. The Execute Shell section would look like this:

docker -H :1234 stop oc9 || true
docker -H :1234 rm oc9 || true
docker -H :1234 run -d --name oc9 -p 80:80 oc9-ci

Now that we have all the stages defined. Let us execute them. We will go the dashboard and click on the clock against the stage 1 job. enter image description here Since our jobs are tied into stages, successful completion of one stage will automatically trigger the next stage. When all the stages complete successfully, we will see the ownCloud container, which we built, successfully running.

Using Docker Containers as Jenkins Slaves

Another paradigm where we can use and achieve high productivity is to use Docker containers as Jenkins slave. Ideally, we can dedicate a few machines as Jenkins slave but typically, running a single slave instance on a machine is wasteful. Docker comes to rescue here by supplying Jenkins slave on demand. Since Docker containers are very fast to boot, the different in running the slave in Docker container and in a dedicated machine, in terms of start time, is quite insignificant. However, now we can boot multiple slaves per machine, utilizing the resources in a much better way and achieving higher parallel processing capability for the same amount of hardware.

Setup Jenkins Docker Plugin

Installing plugins in Jenkins is quite simple. From the left sidebar, choose Manage Jenkins and then Manage Plugins. Click on the Available tab and search for docker. Now just use the check box to select the Docker Plugin and install it.

Configuring Docker Plugin

The documentation for Docker plugin is quite good, however it assumes that we know have reasonable experience with Jenkins. We need to setup Docker daemon as a cloud endpoint where we can spin up and destroy containers on demand. To setup a cloud, go to Manage Jenkins and then to Configure System. Click on the Add a new cloud button at the end of the page. The Docker plugin configuration will open. enter image description here Set Name and Docker URL appropriately and then click on the Add Docker Template button next to Images section. Here we have to give information about the slave instance. To run the Docker container as slave, bare minimal requirement for the image is to have ssh deamon and java installed in the image. We can build our own image or use one provided by the plugin authors evarga/jenkins-slave. enter image description here So we will put this information along with the credentials in this section. The image evarga/jenkins-slave uses jenkins as username and password. Note that using such simple passwords is not recommended as such.

We have configured the Docker plugin to use as cloud. Now we want to stop using master to run jobs. For that, in this page, find Usage field which is located around the top and set that to Only build jobs with label restrictions matching this node.

Now we can configure any job and that would get offloaded to the Docker container as Jenkins slave.