Accelerating GCCSDK autobuilder using apt-proxy

When developing autobuilder scripts it is inefficient to keep downloading the source packages each time, particularly since many packages are large and internet connections may be slow. The following enables the source packages to be downloaded once and kept locally, or on another machine on your network. It only works for those autobuilder packages that use Debian sources as their upstream.

We use the program [apt-proxy], which is a specialised web proxy for Debian packages: essentially it builds a local package repository with only those files that have been requested through it.

Debian-based distributions

On Debian-based distributions (including Ubuntu, Mepis, Knoppix, etc) the autobuilder downloads packages using the 'apt' program. From a root shell, install apt-proxy:

debian# apt-get install apt-proxy

apt-proxy is configured with a configuration file in /etc/apt-proxy/ (in Debian sarge it's /etc/apt-proxy/apt-proxy-v2.conf). Notable entries are:

;; Server IP to listen on
address = 127.0.0.1

;; Server port to listen on
port = 9999

;; Cache directory for apt-proxy
cache_dir = /var/cache/apt-proxy

[debian]
;; The main Debian archive
;; Backend servers, in order of preference
backends = 
        http://ftp.us.debian.org/debian
        http://ftp.de.debian.org/debian
        http://ftp2.de.debian.org/debian
        ftp://ftp.uk.debian.org/debian

So here we create our special proxy on port 9999 and we only allow connections from localhost (this machine) to access it. You may need to change these if you want to store the cache on another machine. The cache will be built up in /var/cache/apt-proxy - create this directory and give it the aptproxy owner:

debian# mkdir /var/cache/apt-proxy
debian# chown aptproxy /var/cache/apt-proxy

The amount of disc space it'll require depends on how many packages you download: at any time you can delete the contents of the directory and start again if you want to recover some space.