Continuous Integration

riscos.info has a Continuous Integration server. Continuous Integration (usually abbreviated to CI, or relatedly CD for Continuous Delivery) involves one or more servers that repeatedly build software projects, either every version control commit or on a regular interval. Motivation for CI varies, but includes checking software remains buildable in the face of developer testsuites, running testsuites, and building chains of software based on its dependencies. Wikipedia has more, and there are plenty of other resources in the software engineering community.

We use Jenkins, a web-based CI system. The web interface can be found at [1] Jenkins is read-only for visitors, while those with an account can start and edit builds.

We use Jenkins to build riscos.info projects, like the RISC OS GCC compiler toolchain. We also use Jenkins to run the RISC OS autobuilder to cross-compile Unix ports and native packaged software.

The purpose of running this under CI is to run builds in a controlled environment, one free of the vagaries of running it on a developer's machine with whatever packages they might have installed: the Jenkins build starts with a blank slate, pulling in the compiler, libraries and so on from other Jenkins jobs, which are 'upstream' of it. When successfully built, a build emits RISC OS packages and libraries which can be consumed by 'downstream' builds. The output files of a build (usually zips or tarballs in our case) are called 'artifacts' in Jenkins terminology.

The second aim of building under CI is to ensure timely builds: if upstream changes, we should rebuild. Due to some details of how the complex package dependency graph interacts with Jenkins, we aren't quite there yet. Building is currently a matter of logged-in users clicking a button on the package(s) in question.

ci.riscos.info

The riscos.info service comprises a Jenkins web GUI - ci.riscos.info - and a number of build 'slaves', where the actual compilation takes place. All these run Ubuntu 14.04 LTS and communicate with the GUI over SSH using Jenkins operations set up in the GUI: the idea is the slaves are stateless vanilla Ubuntu installs, and all special setup is represented in the Jenkins configuration.

Jenkins has a number of top-level jobs for notable components like GCC 4.1, GCC 4.7 and pre-setup jobs for autobuilder projects. Each job follows roughly the same pattern: check out some sources from Subversion or other repositories, possibly import some build outputs from other jobs, build using a shell script, save the output files as artifacts, and report success or failure to Jenkins. The red/yellow/green state indicates whether a build was successful (yellow meaning it built but failed tests).

You can drill down into a job, for instance clicking on gcc-4.7-native (the GCC compiler to run natively on RISC OS) gives you a list of builds on the left hand side, some successful, some not. If you click on a build number (#nnn) and go to Console Output, you can see what happened during the build. If it was red, something went wrong. The build number screen also shows you the versions of source that was used to build this project, and any commit messages since the last time Jenkins built it. This means the build number is a record of the state of the project at a given point in time, and the build log records what happened. Both these pages can be hyperlinked to, to discuss what happened in a given build. If you find a successful build, it will also allow you to download the build products as artifacts. Jenkins also stores the files from the most recent build in the 'Workspace', so you can access intermediate files.