Skip to content

Docker Hub Travis CI Workflow

Jump to bottom
Jean-Michaël Celerier edited this page Sep 5, 2016 · 9 revisions

This page explains how to build and deploy AppImages from Travis CI, using Docker.

The objective is to have a pipeline that :

  • Automatically builds a binary working on most recent desktop distributions
  • When some conditions on a commit are met, upload it to github releases or a provider of your choice.

This tutorial assumes familiarity with the creation of AppImages.

Prerequisites

First, multiple things are required :

  • A GitHub repository with a program that you want to embed in an AppImage.
  • A Travis CI account. Travis is a continuous integration service. It provides virtual machines for open source projects that are triggered on each new commit in the GitHub repository.
  • A Docker Hub account. This optionally allows uploading pre-built Docker images, in order to reduce the build time on Travis. It is also useful if no images provide the packages you need (for instance, if you need a really recent version of GCC, Qt or Clang, or a system library with specific build options).

Creation of a Docker image

The first thing to do is to set-up your AppImage on an old system, for instance CentOS.

This can easily be done with docker :

$ docker pull centos:6
$ docker run -i -t centos:6 /bin/bash

These commands drop you to a shell running in a CentOS container. From there, you can update, install more recent packages, etc :

# yum -y update 
# yum -y install epel-release

epel-release is a repository that contains very recent versions of some packages. For instance, it provides Qt 5.6. This is useful for modern development toolchains.

Recipe.deps

Write down all the commands used to "populate" the container in a script, this is useful for rebuilding it automatically if more updates come down the pipeline. For instance :

#!/bin/bash -eux 

yum -y install epel-release
yum -y update 
yum -y install qt5-qtbase-devel gcc gcc-c++

[ ... git clone AppImage, AppImageKit, etc. ...]

Recipe

Then, write your AppImage recipe and try it on the container (and try to build to to check that everything works !). The recipe should clone or pull the repository, build the software, and ends by creating the AppImage : once it is run, there should be a folder with YourSoftware.AppImage somewhere.

#!/bin/bash -eux

git clone https://github.com/my/software
cmake, etc...

Examples of recipes : https://github.com/probonopd/AppImages/tree/master/recipes

Dockerfile

Finally, we create a Docker file that explains how to build the image :

FROM centos:6

ADD Recipe.deps /Recipe.deps
RUN bash -ex Recipe.deps 

ADD Recipe /Recipe
RUN bash -ex Recipe

Note : the Dockerfile is split in two scripts, one for the dependencies, and one for the building of your software. This is not a necessity. However, this allows Docker to cache the dependencies part, which can be quite long if it implies the rebuilding of big software such as GCC & friends. This way, when fine-tuning the actual Recipe for your software, re-building the image will be faster as it will only re-run the scripts that have changed.

Once done, you can do a test build of the container :

$ ls
Dockerfile Recipe Recipe.deps
$ docker build -t my_image .
$ docker create --name my_container my_image

You can see the container id with docker ps. The created AppImage can be copied to the host with docker cp :

$ docker cp my_container:/path/to/created.AppImage created.AppImage

Try it :

$ ./created.AppImage

The docker container can then be uploaded to Docker hub :

$ docker tag my_image dockerhub_user_name/my_container:latest
$ docker push dockerhub_user_name/my_container:latest

It can just be re-uploaded again if the image has to change. Due to the limited 40 minutes time of Travis CI open-source instance, it is better to stuff most things in the Docker image in order to leave time for the build and tests.

The built image is uploaded to Docker hub; you will be able to pull it from Travis CI.

Travis set-up

Some options are necessary, as of june 2016, to use Docker in Travis. The principle is simple :

  • Fetch the uploaded docker image
  • Run the appimage recipe
  • Copy the appimage to the travis host in order to upload it.
  • If the deployment condition is met, upload to the provider.

.travis.yml

dist: trusty
sudo: required
services:
  - docker

script: 
  - docker pull dockerhub_user_name/my_container
  - docker run --name buildvm dockerhub_user_name/my_container /bin/bash Recipe
  - docker cp buildvm:/path/to/created.AppImage created.AppImage 

deploy:
  - provider: releases # Uploads to github
    api_key:
      secure:  # See Travis CI documentation
    file: created.AppImage
    skip_cleanup: true
    on: # Put here conditions for deployment. 
      tags: true # This condition will only deploy on tagged commits.
Clone this wiki locally