What I've Been Doing Lately... Or Better Said, Coding Lately.

Lately I've been doing research to get an Exchange resource working and usable for KDE. During this time, I've had to have a working trunk build since OpenSuSE's Build Service tends to have custom patches for their build of KDE that make some things not behave or look how the default KDE trunk tree acts/looks (that's not a comment that that's bad, just not my want for when I do development, but I digress...)

To facilitate this, I'd been using the shell scripts that KDE has on Techbase and on the OpenSuSE wiki to build trunk. I've found however, that my time would be better spent working on code, instead of waiting on code to compile when at home. Since I wanted something that would even fully track Qt's Git tree and email me when builds ran or error-ed out, kdesvn-build was not going to meet my needs. What came out of it was a hacky little Perl script called KDEBuilder that was fairly fragile and frankly not designed at all.

While thinking over the mess that was KDEBuilder, I realized that what I *really* wanted was something that was flexible, feature rich, and wasn't an overblown ported shell script on steroids. Out of this came the design for Direktor, a continuous build system written in a clean Perl code style (heavily influenced by the Google Perl Style guide....) To better describe Direktor, I've pasted the ReadMe.txt from the source tree below:

---

DOCUMENTATION FOR DIRECTOR:

Direktor is a continuous build system primarily meant for use by the KDE open source project. The goal was to create a system that built KDE and it's various dependencies efficiently and allowed the user to be notified and to monitor the progress of the builds.

To achieve these goals, Direktor can be configured to e-mail out the build announcements and build failures. This is augmented by the HTML data that is generated to create a dashboard for the build.

DESIGN:
Originally, Direktor was only a simple Perl script that would naively build KDE trunk and announce simple e-mailed snippets to an email address. This became fairly ungainly as time went on with the need to add in Qt4 to the build. This is when it became obvious that the build methods needed to be more abstracted away to allow any software (including KDE's base dependencies if so desired) to be built with this system.

To facilitate this, Direktor was re-factored to use description files to detail various aspects about the build and source control system per module or project. These description files are nothing more than INI files with required entries to make the system work.

A typical description file follows:

fig. 1: qt4.direktor

---


# Direktor 1.0
name = qt
repository_type = git
in_source_build = TRUE
source_repository = git://gitorious.org/+kde-developers/qt/kde-qt.git
source_directory = \$HOME/sources/qt-kde
prefix = \$HOME/installs/qt-kde
configuration_flags = -qt-gif -debug -fast -no-separate-debug-info -system-libpng -system-libjpeg -system-zlib -dbus -webkit -plugin-sql-mysql -nomake examples -nomake demos -prefix \$QTDIR
environment = QTDIR=$HOME/installs/qt-kde \
PATH=$QTDIR/bin:$PATH \
QT_PLUGIN_PATH=$HOME/installs/trunk/lib/KDE4/plugins:$QTDIR/plugins \
PKG_CONFIG_DIR=$QTDIR/lib/pkgconfig:$PKG_CONFIG_DIR \
LD_LIBRARY_PATH=$QTDIR/lib:$LD_LIBRARY_PATH \
QTEST_COLORED=1
configuration_command = configure
build_command = make
build_flags = -j2
install_command = make install


---

As can be seen, this allows the system to be very versatile with builds allowing one to build any package. The configuration_command, build_command, and install_command options allow can be any conceivable valid Linux command, even
scripts.

The fields in a direktor file are required. Lets review the fields and their format:

# Direktor 1.0
This is the description and format version of a description file as used by this application.

name:
This is an arbitrary string value. It is mainly used to present information about the module to the user.

repository_type:
This is a string value that takes one of the following values: svn, git, file, cvs, or p4. This tells Direktor which type of repository the module lives in. The file option tells Direktor that the source is not in a revision control system, but rather is a release tar.bz2, tar.gz, tgz, or zip archive.

in_source_build:
This attribute takes either a TRUE or FALSE. This boolean value tells Direktor to build the binaries from within or outside the source directory. Increasingly, many of the open source applications are built outside the source directory.

source_repository:
This is the location of the source repository or the full path location of the archive that contains the sources of the project to build.

source_directory:
This is the location where the unpacked or checked out files will be placed on the local hard disk.

prefix:
This is the prefix where the resultant binaries will be installed to.

configuration_flags:
These are the flags to pass to the script or application called to configure the source tree for compilation.

environment:
This describes the environment variables and values that must be set to allow the build of the source code to complete correctly. These are placed one per line with the continuation character being the forward slash.

configuration_command:
This tells Direktor what command or script to run in the source directory to configure the source code for compilation.

build_command:
This tells Direktor what command or script to run from the build directory to compile the source code into binary object and executable code.

build_flags:
This instructs Direktor to use the requested flags with the command to build the source tree. In many cases, this is to instruct the compiler to use non-standard library or header locations, or to do the build in parallel allowing the job to complete faster.

install_command:
This instructs Direktor what command or script to run from the build directory to install the resultant binaries produced in to the prefix earlier specified.

Note that any time an environment variable is requested that has a dollar sign in it must be escaped to properly be interpolated by the shell that is invoked by Direktor.

REGISTERING MODULES:
To register the modules, simply add the module direktor file names without the .direktor extension to the .direktor.modules_registry located in the build user's home directory. The header line declares the major series of Direktor that can read this format. An example .direktor.modules_registry follows:

fig. 2: .direktor.modules_registry:

---


# Direktor 1.0
qt4
kdesupport
kdelibs
kdepimlibs
kdebase
kdepim
kdemultimedia


---

WORK FLOW:
When Direktor starts, it clears the current environment and then sets up the global parameters defined in it's configuration stored in the home directory of the user running the build. This file is called $HOME/.direktor.cfg, and is in an extended INI format.

fig. 3: .direktor.cfg

---


# Direktor 1.0
[GENERAL]
user = somebody
group = users
registry = $HOME/.direktor.modules_registry
announce_start_via_email = 1
announce_builds_via_email = 1
announce_errors_via_email = 1
announce_completion_via_email = 1
use_dashboard = 1
; these next two are mutually exclusive!
use_watcher_to_dispatch_builds = 0
use_scheduler_to_dispatch_builds = 1
; needed for any non-generic initial repo setup that you will need. Remember,
; this gets run if the lock file $HOME/.direktor.firstrun does not exist.
first_time_startup_script = firsttimesetup.sh

[EMAIL]
from_address = someone@somewhere.net
to_address = somelist@somewhere.net
cc_addresses = bob@somewhere.net,bugs@somewhere.net
subject_prefix = MY BUILDER

[DASHBOARD]
html_upload_path = somehost.somewhere.net:/home/builder/public_html/
upload_user = someone
upload_cioher = arcfour

[WATCHER]
email_inbox = scm-monitor@somewhere.net
action_to_take = queue_all

[SCHEDULER]
queue_file = \$HOME/.direktor.schedulerrc
action_to_take = fullbuild


---

Once the global configuration is loaded, Direktor checks whether it was started in interactive or daemon mode. Once the execution mode is determined and acted on, Direktor scans the modules_registry to see which modules it needs to retrieve and build.

After it knows which targets to build, it loads the .direktor file for the given target in. Then it announces the start of the first target to the pre-defined E-Mail address and acts on the required options accordingly by setting up the additional environment needed for the compilation of the sources in this target, checking out or downloading the sources to the local disk and unpacking it if needs be to the source directory, configuring the sources, running the build command for the source target, and then installing it into a predefined prefix. Along the way, if any of the steps fail, an E-Mail announcement is sent out to notify the user building the code that the build has failed.

While this is all going on, Direktor will generate HTML pages to display the current status of the builds, including whether a build is running, the target's build completion status (success or failure), and when the build is done, what warnings were output during the compilation.

If the compile fails, the compilation log is displayed back to the last successful portion of the configuration, build, or installation.

CONTINUOUS BUILDS:
The most significant feature of Direktor is it's ability to run automatic and prescheduled builds with user specified actions. The two modes of this allow it to be used for continuous build assurance allowing projects to assure that all commits to a branch that is monitored to always build.

The first method, using the watcher system, requires that a post-commit action be set on your source control system to notify Direktor when a check-in has occurred which is delivered to a selected address configured for retrieval by a fetchmail-alike and then processed by procmail, or similar application. During the processing by procmail, the message is piped to our watcher script which then asks the scheduler to start a build as soon as the current server is idle and sends an email notification to a selected e-mail address that a commit to one of the modules has caused a new build to be queued. This very literally will on very active repositories cause the build to always be queued, so be sure that is truly the desired outcome.

The second method kicks off builds at predetermined clock times. The schedule is re-read every 15 minutes to add newly desired times that may have been added to the specified configuration file. Note that no two builds can run at the same time, so be sure that the schedule is long enough to accommodate the completion of all builds.