May 27, 2012
OS X downgrade....
Unsure of the cause, but OS X 10.7.4 has been a wash. Not only is MainStage horribly broken under it, so is QuickTime (and it seems anything else that uses the QuickTime libraries, for that matter), along with the screensaver engine. Due to this, a) I've downgraded to 10.7.3 and b) trying to find a way to block the update for now until I can confirm that the issues I saw are resolved. Apple dropped the ball on this one, in my opinion.....
May 11, 2012
Status as of May 13, 2012
Real life kicked in and side tracked me a couple days (along with having to reinstall my iMac to fix broken beta updates from Apple....) All that done, I got the VM instance of the build server running again, and have written a couple scripts to simplify some mundane tasks needed for builds.
As my boss hasn't cleared a maintenance to occur at work, I'll be able to spend a bit of time this weekend getting closer to the point of being able to have all of the "core" OS build dependencies finished before building RPM and then rebuilding everything again, but this time as packages. This will also help determine whether some of the tools are fully self-hosting.
I'll update with more info as I continue.
As my boss hasn't cleared a maintenance to occur at work, I'll be able to spend a bit of time this weekend getting closer to the point of being able to have all of the "core" OS build dependencies finished before building RPM and then rebuilding everything again, but this time as packages. This will also help determine whether some of the tools are fully self-hosting.
I'll update with more info as I continue.
April 29, 2012
Status update for AltimatOS - April 29, 2012
Short and quick update:
April 27, 2012
Progress on AltimatOS
Some know I've been working off and on in my time on a new Linux distribution, and today I decided to start writing up the process to allow better feedback as I work on the software and packages involved, and better
show what goes into building a new flavour of distribution. To start, I guess I should cover what my goals with this are:
The rationale for using VMware for this is simple, While working on this, I can still have a usable system to use for my day to day until AltimatOS is to a reasonable state to use as my main desktop.
As the weeks roll on, I'll chronicle the work being done and at times may ask questions of my readers to see if you've ideas to make this even better. Thanks for joining me on this ride, comments always welcome.
- Home and Business User-centric: This means that I'll be forgoing a number of UNIX-isms in favour of user needs and wants. While the power of the GNU project's tools, KDE, and other free/open source tools are the building blocks of AltimatOS, the intent is to allow the average accountant, secretary, high school student, or end-user to easily use their computer without getting in their way, or exposing them to archaic command-line prompts to do normal tasks.
- Desktop oriented: AltimatOS is not, and never will be a server platform. While the underside of the operating system is based on similar software as other GNU/Linux distributions, AltimatOS' design and filesystem layout are not targeted at the duties of acting as a high performance server.
- Not another "Linux": AltimatOS is not another GNU/Linux distribution in that the normal GNU/Linux distributions mostly adhere to filesystem layouts and design philosophies that run counter to good end-user desktop and corporate desktop operating system design. AltimatOS aims to avoid many of these design pitfalls that have kept "Linux on the Desktop" from becoming a reality.
- The entire operating system must hide away the command line as much as is possible - This means that by default (unless in debug mode) the system should not expose the command-line heritage of GNU/Linux to the end-user. (Turn off the virtual consoles, disallow switching from X11 to the other consoles, etc.)
- No normal-case configuration settings should require the command line - Printing, file-sharing, hardware configuration, security configuration, and network settings must be done graphically and should not require jumping to the command line.
- Job scheduling (cron and others) must be graphically configured - self explanatory.
- Filesystem layout must be simple to navigate - While the FHS and LSB standards are intended on "standardizing" GNU/Linux, they've been largely designed by UNIX engineers, which drive them from being useful to the typical end user. To this end I'm taking a page out of Apple's play book; AltimatOS will radically restructure the filesystem hierarchy to simplify the user experience. While the traditional filesystem hierarchy will be preserved with symlinks for some level of compatibility, software explicitly designed for AltimatOS should not rely on these compatibility symlinks.
- The compatibility symlinks will be hidden from end-user view. The method by which we do so will be through patches to KDE's KIO and Dolphin to honour our distribution specific named extended file attribute: user.visibility.
- Authentication must be done through the local LDAP and Kerberos services running on the system. The flat file method of authentication inspired from UNIX, while simple is dated and inflexible to extend.
- In a central authentication environment, we must support LDAP+Kerberos - Simplifying enterprise integration with standard OpenLDAP and MIT Kerberos on a GNU/Linux server.
- System configuration is tightly coupled to the Puppet configuration engine - Software packages and configuration tools must work through the custom Puppet registration on the local system, or a centrally manage puppet-master.
The rationale for using VMware for this is simple, While working on this, I can still have a usable system to use for my day to day until AltimatOS is to a reasonable state to use as my main desktop.
As the weeks roll on, I'll chronicle the work being done and at times may ask questions of my readers to see if you've ideas to make this even better. Thanks for joining me on this ride, comments always welcome.
July 14, 2011
Vixie Cron How I Loath Thee.....
As some know, I work as the senior systems administrator for a company in the southeast SF Bay Area. Recently, we had to move to a new security system, after having been broken into four times in the last _four_ years.
One of the features of this new system is that it ties into our phone system to alert the building that the alarm is about to be armed. To do this, a dial out to a special phone number needs to occur at the desired hour and then voila! Instant Alert throughout our suite! So, we've setup an email to fax service that we can easily trigger by a simple email.
So I wrote a REALLY simple shell script shown below (with the fax and email addresses elided):
As I said, simple.
From there I setup my crontab on one of the Linux box in my cube to have it go off at the apropriote time, 7:55PM Monday through Friday:
Tested out the script, thinking that "Hey, I'm all done". Wrong.
I come to find out that Vixie Cron can't seem to tell time! I get the alert emails at 8:31PM from the system!
At this point, looks like I'll be replacing Vixie Cron on my workstation (CentOS 5). Can anyone advise a good Cron Daemon replacement that has enough similarities to vixie cron that I won't have to relearn too much and can actually interpret time settings in the crontab as more than mere suggestions?
One of the features of this new system is that it ties into our phone system to alert the building that the alarm is about to be armed. To do this, a dial out to a special phone number needs to occur at the desired hour and then voila! Instant Alert throughout our suite! So, we've setup an email to fax service that we can easily trigger by a simple email.
So I wrote a REALLY simple shell script shown below (with the fax and email addresses elided):
#!/bin/bash
set -e
echo "Sending Security Page NOW!"
/usr/sbin/sendmail -t <<EOF
FROM: SOMERANDOMSMUCK@minervanetworks.com
TO: 1234567890@fax.com
CC: SOMERANDOMSMUCK@minervanetworks.com
SUBJECT: Alarm Set
Set.
EOF
RET=$?
if [ $? -eq 0 ]; then
echo "SUCCESS"
fi
As I said, simple.
From there I setup my crontab on one of the Linux box in my cube to have it go off at the apropriote time, 7:55PM Monday through Friday:
MAILTO=SOMERANDOMSMUCK@minervanetworks.com
50 19 * * 1-5 /usr/bin/securityfax.sh
Tested out the script, thinking that "Hey, I'm all done". Wrong.
I come to find out that Vixie Cron can't seem to tell time! I get the alert emails at 8:31PM from the system!
At this point, looks like I'll be replacing Vixie Cron on my workstation (CentOS 5). Can anyone advise a good Cron Daemon replacement that has enough similarities to vixie cron that I won't have to relearn too much and can actually interpret time settings in the crontab as more than mere suggestions?
February 11, 2011
Qt, Nokia, and the Internet Rambling.....
So, most of you who read my blog know about Nokia's deal with Microsoft. To re-iterate the best image posting I've seen on planetkde via John Layt's blog: Keep Calm and Carry On.
Yes, Microsoft has been a major negative force on computing, especially with regards to the freedoms of end users. This however does not warrant the extreme and in many ways self-defeating knee-jerk reactions that have been permeating the Interwebs oft lately on the F/OSS blogs.
Lets rather, take a wait and see approach to this. There really isn't enough information on the ground at this point in the game to know what is or is not going to occur in the immediate or even long-term for Qt Development Frameworks at Nokia. Lets trust in our friends at Nokia to gather up the information that the community is craving to know and get it ready for public consumption.
Until then, Nothing to See Here, Move Along.
Yes, Microsoft has been a major negative force on computing, especially with regards to the freedoms of end users. This however does not warrant the extreme and in many ways self-defeating knee-jerk reactions that have been permeating the Interwebs oft lately on the F/OSS blogs.
Lets rather, take a wait and see approach to this. There really isn't enough information on the ground at this point in the game to know what is or is not going to occur in the immediate or even long-term for Qt Development Frameworks at Nokia. Lets trust in our friends at Nokia to gather up the information that the community is craving to know and get it ready for public consumption.
Until then, Nothing to See Here, Move Along.
October 13, 2009
What I've Been Doing Lately... Or Better Said, Coding Lately.
Lately I've been doing research to get an Exchange resource working and usable for KDE. During this time, I've had to have a working trunk build since OpenSuSE's Build Service tends to have custom patches for their build of KDE that make some things not behave or look how the default KDE trunk tree acts/looks (that's not a comment that that's bad, just not my want for when I do development, but I digress...)
To facilitate this, I'd been using the shell scripts that KDE has on Techbase and on the OpenSuSE wiki to build trunk. I've found however, that my time would be better spent working on code, instead of waiting on code to compile when at home. Since I wanted something that would even fully track Qt's Git tree and email me when builds ran or error-ed out, kdesvn-build was not going to meet my needs. What came out of it was a hacky little Perl script called KDEBuilder that was fairly fragile and frankly not designed at all.
While thinking over the mess that was KDEBuilder, I realized that what I *really* wanted was something that was flexible, feature rich, and wasn't an overblown ported shell script on steroids. Out of this came the design for Direktor, a continuous build system written in a clean Perl code style (heavily influenced by the Google Perl Style guide....) To better describe Direktor, I've pasted the ReadMe.txt from the source tree below:
DOCUMENTATION FOR DIRECTOR:
Direktor is a continuous build system primarily meant for use by the KDE open source project. The goal was to create a system that built KDE and it's various dependencies efficiently and allowed the user to be notified and to monitor the progress of the builds.
To achieve these goals, Direktor can be configured to e-mail out the build announcements and build failures. This is augmented by the HTML data that is generated to create a dashboard for the build.
DESIGN:
Originally, Direktor was only a simple Perl script that would naively build KDE trunk and announce simple e-mailed snippets to an email address. This became fairly ungainly as time went on with the need to add in Qt4 to the build. This is when it became obvious that the build methods needed to be more abstracted away to allow any software (including KDE's base dependencies if so desired) to be built with this system.
To facilitate this, Direktor was re-factored to use description files to detail various aspects about the build and source control system per module or project. These description files are nothing more than INI files with required entries to make the system work.
A typical description file follows:
fig. 1: qt4.direktor
As can be seen, this allows the system to be very versatile with builds allowing one to build any package. The configuration_command, build_command, and install_command options allow can be any conceivable valid Linux command, even
scripts.
The fields in a direktor file are required. Lets review the fields and their format:
# Direktor 1.0
This is the description and format version of a description file as used by this application.
name:
This is an arbitrary string value. It is mainly used to present information about the module to the user.
repository_type:
This is a string value that takes one of the following values: svn, git, file, cvs, or p4. This tells Direktor which type of repository the module lives in. The file option tells Direktor that the source is not in a revision control system, but rather is a release tar.bz2, tar.gz, tgz, or zip archive.
in_source_build:
This attribute takes either a TRUE or FALSE. This boolean value tells Direktor to build the binaries from within or outside the source directory. Increasingly, many of the open source applications are built outside the source directory.
source_repository:
This is the location of the source repository or the full path location of the archive that contains the sources of the project to build.
source_directory:
This is the location where the unpacked or checked out files will be placed on the local hard disk.
prefix:
This is the prefix where the resultant binaries will be installed to.
configuration_flags:
These are the flags to pass to the script or application called to configure the source tree for compilation.
environment:
This describes the environment variables and values that must be set to allow the build of the source code to complete correctly. These are placed one per line with the continuation character being the forward slash.
configuration_command:
This tells Direktor what command or script to run in the source directory to configure the source code for compilation.
build_command:
This tells Direktor what command or script to run from the build directory to compile the source code into binary object and executable code.
build_flags:
This instructs Direktor to use the requested flags with the command to build the source tree. In many cases, this is to instruct the compiler to use non-standard library or header locations, or to do the build in parallel allowing the job to complete faster.
install_command:
This instructs Direktor what command or script to run from the build directory to install the resultant binaries produced in to the prefix earlier specified.
Note that any time an environment variable is requested that has a dollar sign in it must be escaped to properly be interpolated by the shell that is invoked by Direktor.
REGISTERING MODULES:
To register the modules, simply add the module direktor file names without the .direktor extension to the .direktor.modules_registry located in the build user's home directory. The header line declares the major series of Direktor that can read this format. An example .direktor.modules_registry follows:
fig. 2: .direktor.modules_registry:
WORK FLOW:
When Direktor starts, it clears the current environment and then sets up the global parameters defined in it's configuration stored in the home directory of the user running the build. This file is called $HOME/.direktor.cfg, and is in an extended INI format.
fig. 3: .direktor.cfg
Once the global configuration is loaded, Direktor checks whether it was started in interactive or daemon mode. Once the execution mode is determined and acted on, Direktor scans the modules_registry to see which modules it needs to retrieve and build.
After it knows which targets to build, it loads the .direktor file for the given target in. Then it announces the start of the first target to the pre-defined E-Mail address and acts on the required options accordingly by setting up the additional environment needed for the compilation of the sources in this target, checking out or downloading the sources to the local disk and unpacking it if needs be to the source directory, configuring the sources, running the build command for the source target, and then installing it into a predefined prefix. Along the way, if any of the steps fail, an E-Mail announcement is sent out to notify the user building the code that the build has failed.
While this is all going on, Direktor will generate HTML pages to display the current status of the builds, including whether a build is running, the target's build completion status (success or failure), and when the build is done, what warnings were output during the compilation.
If the compile fails, the compilation log is displayed back to the last successful portion of the configuration, build, or installation.
CONTINUOUS BUILDS:
The most significant feature of Direktor is it's ability to run automatic and prescheduled builds with user specified actions. The two modes of this allow it to be used for continuous build assurance allowing projects to assure that all commits to a branch that is monitored to always build.
The first method, using the watcher system, requires that a post-commit action be set on your source control system to notify Direktor when a check-in has occurred which is delivered to a selected address configured for retrieval by a fetchmail-alike and then processed by procmail, or similar application. During the processing by procmail, the message is piped to our watcher script which then asks the scheduler to start a build as soon as the current server is idle and sends an email notification to a selected e-mail address that a commit to one of the modules has caused a new build to be queued. This very literally will on very active repositories cause the build to always be queued, so be sure that is truly the desired outcome.
The second method kicks off builds at predetermined clock times. The schedule is re-read every 15 minutes to add newly desired times that may have been added to the specified configuration file. Note that no two builds can run at the same time, so be sure that the schedule is long enough to accommodate the completion of all builds.
To facilitate this, I'd been using the shell scripts that KDE has on Techbase and on the OpenSuSE wiki to build trunk. I've found however, that my time would be better spent working on code, instead of waiting on code to compile when at home. Since I wanted something that would even fully track Qt's Git tree and email me when builds ran or error-ed out, kdesvn-build was not going to meet my needs. What came out of it was a hacky little Perl script called KDEBuilder that was fairly fragile and frankly not designed at all.
While thinking over the mess that was KDEBuilder, I realized that what I *really* wanted was something that was flexible, feature rich, and wasn't an overblown ported shell script on steroids. Out of this came the design for Direktor, a continuous build system written in a clean Perl code style (heavily influenced by the Google Perl Style guide....) To better describe Direktor, I've pasted the ReadMe.txt from the source tree below:
DOCUMENTATION FOR DIRECTOR:
Direktor is a continuous build system primarily meant for use by the KDE open source project. The goal was to create a system that built KDE and it's various dependencies efficiently and allowed the user to be notified and to monitor the progress of the builds.
To achieve these goals, Direktor can be configured to e-mail out the build announcements and build failures. This is augmented by the HTML data that is generated to create a dashboard for the build.
DESIGN:
Originally, Direktor was only a simple Perl script that would naively build KDE trunk and announce simple e-mailed snippets to an email address. This became fairly ungainly as time went on with the need to add in Qt4 to the build. This is when it became obvious that the build methods needed to be more abstracted away to allow any software (including KDE's base dependencies if so desired) to be built with this system.
To facilitate this, Direktor was re-factored to use description files to detail various aspects about the build and source control system per module or project. These description files are nothing more than INI files with required entries to make the system work.
A typical description file follows:
fig. 1: qt4.direktor
# Direktor 1.0
name = qt
repository_type = git
in_source_build = TRUE
source_repository = git://gitorious.org/+kde-developers/qt/kde-qt.git
source_directory = \$HOME/sources/qt-kde
prefix = \$HOME/installs/qt-kde
configuration_flags = -qt-gif -debug -fast -no-separate-debug-info -system-libpng -system-libjpeg -system-zlib -dbus -webkit -plugin-sql-mysql -nomake examples -nomake demos -prefix \$QTDIR
environment = QTDIR=$HOME/installs/qt-kde \
PATH=$QTDIR/bin:$PATH \
QT_PLUGIN_PATH=$HOME/installs/trunk/lib/KDE4/plugins:$QTDIR/plugins \
PKG_CONFIG_DIR=$QTDIR/lib/pkgconfig:$PKG_CONFIG_DIR \
LD_LIBRARY_PATH=$QTDIR/lib:$LD_LIBRARY_PATH \
QTEST_COLORED=1
configuration_command = configure
build_command = make
build_flags = -j2
install_command = make install
As can be seen, this allows the system to be very versatile with builds allowing one to build any package. The configuration_command, build_command, and install_command options allow can be any conceivable valid Linux command, even
scripts.
The fields in a direktor file are required. Lets review the fields and their format:
# Direktor 1.0
This is the description and format version of a description file as used by this application.
name:
This is an arbitrary string value. It is mainly used to present information about the module to the user.
repository_type:
This is a string value that takes one of the following values: svn, git, file, cvs, or p4. This tells Direktor which type of repository the module lives in. The file option tells Direktor that the source is not in a revision control system, but rather is a release tar.bz2, tar.gz, tgz, or zip archive.
in_source_build:
This attribute takes either a TRUE or FALSE. This boolean value tells Direktor to build the binaries from within or outside the source directory. Increasingly, many of the open source applications are built outside the source directory.
source_repository:
This is the location of the source repository or the full path location of the archive that contains the sources of the project to build.
source_directory:
This is the location where the unpacked or checked out files will be placed on the local hard disk.
prefix:
This is the prefix where the resultant binaries will be installed to.
configuration_flags:
These are the flags to pass to the script or application called to configure the source tree for compilation.
environment:
This describes the environment variables and values that must be set to allow the build of the source code to complete correctly. These are placed one per line with the continuation character being the forward slash.
configuration_command:
This tells Direktor what command or script to run in the source directory to configure the source code for compilation.
build_command:
This tells Direktor what command or script to run from the build directory to compile the source code into binary object and executable code.
build_flags:
This instructs Direktor to use the requested flags with the command to build the source tree. In many cases, this is to instruct the compiler to use non-standard library or header locations, or to do the build in parallel allowing the job to complete faster.
install_command:
This instructs Direktor what command or script to run from the build directory to install the resultant binaries produced in to the prefix earlier specified.
Note that any time an environment variable is requested that has a dollar sign in it must be escaped to properly be interpolated by the shell that is invoked by Direktor.
REGISTERING MODULES:
To register the modules, simply add the module direktor file names without the .direktor extension to the .direktor.modules_registry located in the build user's home directory. The header line declares the major series of Direktor that can read this format. An example .direktor.modules_registry follows:
fig. 2: .direktor.modules_registry:
# Direktor 1.0
qt4
kdesupport
kdelibs
kdepimlibs
kdebase
kdepim
kdemultimedia
WORK FLOW:
When Direktor starts, it clears the current environment and then sets up the global parameters defined in it's configuration stored in the home directory of the user running the build. This file is called $HOME/.direktor.cfg, and is in an extended INI format.
fig. 3: .direktor.cfg
# Direktor 1.0
[GENERAL]
user = somebody
group = users
registry = $HOME/.direktor.modules_registry
announce_start_via_email = 1
announce_builds_via_email = 1
announce_errors_via_email = 1
announce_completion_via_email = 1
use_dashboard = 1
; these next two are mutually exclusive!
use_watcher_to_dispatch_builds = 0
use_scheduler_to_dispatch_builds = 1
; needed for any non-generic initial repo setup that you will need. Remember,
; this gets run if the lock file $HOME/.direktor.firstrun does not exist.
first_time_startup_script = firsttimesetup.sh
[EMAIL]
from_address = someone@somewhere.net
to_address = somelist@somewhere.net
cc_addresses = bob@somewhere.net,bugs@somewhere.net
subject_prefix = MY BUILDER
[DASHBOARD]
html_upload_path = somehost.somewhere.net:/home/builder/public_html/
upload_user = someone
upload_cioher = arcfour
[WATCHER]
email_inbox = scm-monitor@somewhere.net
action_to_take = queue_all
[SCHEDULER]
queue_file = \$HOME/.direktor.schedulerrc
action_to_take = fullbuild
Once the global configuration is loaded, Direktor checks whether it was started in interactive or daemon mode. Once the execution mode is determined and acted on, Direktor scans the modules_registry to see which modules it needs to retrieve and build.
After it knows which targets to build, it loads the .direktor file for the given target in. Then it announces the start of the first target to the pre-defined E-Mail address and acts on the required options accordingly by setting up the additional environment needed for the compilation of the sources in this target, checking out or downloading the sources to the local disk and unpacking it if needs be to the source directory, configuring the sources, running the build command for the source target, and then installing it into a predefined prefix. Along the way, if any of the steps fail, an E-Mail announcement is sent out to notify the user building the code that the build has failed.
While this is all going on, Direktor will generate HTML pages to display the current status of the builds, including whether a build is running, the target's build completion status (success or failure), and when the build is done, what warnings were output during the compilation.
If the compile fails, the compilation log is displayed back to the last successful portion of the configuration, build, or installation.
CONTINUOUS BUILDS:
The most significant feature of Direktor is it's ability to run automatic and prescheduled builds with user specified actions. The two modes of this allow it to be used for continuous build assurance allowing projects to assure that all commits to a branch that is monitored to always build.
The first method, using the watcher system, requires that a post-commit action be set on your source control system to notify Direktor when a check-in has occurred which is delivered to a selected address configured for retrieval by a fetchmail-alike and then processed by procmail, or similar application. During the processing by procmail, the message is piped to our watcher script which then asks the scheduler to start a build as soon as the current server is idle and sends an email notification to a selected e-mail address that a commit to one of the modules has caused a new build to be queued. This very literally will on very active repositories cause the build to always be queued, so be sure that is truly the desired outcome.
The second method kicks off builds at predetermined clock times. The schedule is re-read every 15 minutes to add newly desired times that may have been added to the specified configuration file. Note that no two builds can run at the same time, so be sure that the schedule is long enough to accommodate the completion of all builds.
October 2, 2009
Sprint-A-Thon!
What has two homes on either side of the planet, attracts geeks by the storm, and the letter Q?
No, not a rabid Qbert collector, rather Qt Dev Days!
This year, to start things out right we've managed to acquire through the clandestine (ok, maybe not so much....) work of the Nokia Qt Development Frameworks division (formerly known as Trolltech) F/OSS Community Manager, Knut Yrvin, a small stash of passes for KDE developers to the event. We thought what better way to draw in interested KDE developers than to hold a pre-Qt Dev Days Sprint?!
What will the sprint cover, you might ask? Well, we're interested in hearing (hopefully sooner than later) what YOU, the KDE hacker wants to work on/brainstorm on at this sprint! To better get an idea of whom will or will not be coming, please stop by our Facebook event page and RSVP. If you choose not to use FB to let us know, feel free to leave a blurb on my blog. Thanks :D
UPDATE: Hmmm, it would seem I for got to post the LINK to said FB event..... Here you go: Facebook Dev Days Sprint.
No, not a rabid Qbert collector, rather Qt Dev Days!
This year, to start things out right we've managed to acquire through the clandestine (ok, maybe not so much....) work of the Nokia Qt Development Frameworks division (formerly known as Trolltech) F/OSS Community Manager, Knut Yrvin, a small stash of passes for KDE developers to the event. We thought what better way to draw in interested KDE developers than to hold a pre-Qt Dev Days Sprint?!
What will the sprint cover, you might ask? Well, we're interested in hearing (hopefully sooner than later) what YOU, the KDE hacker wants to work on/brainstorm on at this sprint! To better get an idea of whom will or will not be coming, please stop by our Facebook event page and RSVP. If you choose not to use FB to let us know, feel free to leave a blurb on my blog. Thanks :D
UPDATE: Hmmm, it would seem I for got to post the LINK to said FB event..... Here you go: Facebook Dev Days Sprint.
August 21, 2009
Update....
Thanks to Brad Hards' quick intervention, I have a tarball of a recent SVN snapshot for OpenChange. I'll build the code and start working on reviewing the API tomorrow.
Stuck....
This is a special request:
The last couple days, the OpenChange project's website has fallen off the 'Net. As one of the steps in getting my development environment up is installing libmapi++ (which no distribution seems to have packaged at all), I'm stuck. If anyone knows the folks maintaining the OpenChange project's website, please find out what is going on. Also, if anyone has a current check-out of the libmapi++ tree, please let me know so I can get it from you. Thanks.
You may now return to your regular blogging enjoyment.
The last couple days, the OpenChange project's website has fallen off the 'Net. As one of the steps in getting my development environment up is installing libmapi++ (which no distribution seems to have packaged at all), I'm stuck. If anyone knows the folks maintaining the OpenChange project's website, please find out what is going on. Also, if anyone has a current check-out of the libmapi++ tree, please let me know so I can get it from you. Thanks.
You may now return to your regular blogging enjoyment.
August 20, 2009
Trudging Through Another Build....
After spending an evening getting my development environment updated to trunk, I think I've come to a few conclusions: 1) I need to learn more about CMake, 2) learn how to deal with missing headers better (thank you ervin) 3. don't resort to sledgehammer methodology as quickly to fix a problem, 4) work on my my patience level more.
After committing the fix from being the guinea pig for the fix, I'm onto building kdepim, and then I'll start working on getting the code fixed for the openchange resource so it compiles.
After committing the fix from being the guinea pig for the fix, I'm onto building kdepim, and then I'll start working on getting the code fixed for the openchange resource so it compiles.
August 18, 2009
Answering Questions.
Just so those that are curious know, I'll repost my response to the over a dozen comments to my posting last night......
Just a small fyi for those that have been commenting:
Yes, my aim is for 100% parity with Outlook's capabilities with MAPI. This means that Mail, Contacts, Calendar, Notes, Journal, Public Folders, Tasks, and Projects are on my TODO list.
Whether this will use openchange for all of that functionality has yet to be decided based off of testig and feature completion.
Also, I think I should make it clear: the code I'll be working on was started in development around 2007 by Brad Hards and continued by Alan Alvares for his 2008 GSoC project. This WILL be an akonadi resource that talks Extended MAPI/RPC to the Exchange server and DBUS back to the Akonadi session daemon. Where necessary, I might call things in the various libraries from the Samba4 project to mimic an Active Directory backed Exchange client to make my work easier, we'll see if I have to do that or if I can rely on some things from LDAP..... Onwards and Upwards :)
With this, I need to stress that it won't be instant progress on this, so please all, be patient :)
Just a small fyi for those that have been commenting:
Yes, my aim is for 100% parity with Outlook's capabilities with MAPI. This means that Mail, Contacts, Calendar, Notes, Journal, Public Folders, Tasks, and Projects are on my TODO list.
Whether this will use openchange for all of that functionality has yet to be decided based off of testig and feature completion.
Also, I think I should make it clear: the code I'll be working on was started in development around 2007 by Brad Hards and continued by Alan Alvares for his 2008 GSoC project. This WILL be an akonadi resource that talks Extended MAPI/RPC to the Exchange server and DBUS back to the Akonadi session daemon. Where necessary, I might call things in the various libraries from the Samba4 project to mimic an Active Directory backed Exchange client to make my work easier, we'll see if I have to do that or if I can rely on some things from LDAP..... Onwards and Upwards :)
With this, I need to stress that it won't be instant progress on this, so please all, be patient :)
August 17, 2009
Hello planetkde!
This is my first posting on the planet (I hope... the aggregator doesn't seem to be showing my feed.... Here goes crossing fingers....)
Some people in the community may know me as the guy that likes Perl, Qt, KDE, and does a hell of a lot of IT administration at a very un-KDE job. I've helped out at a few F/OSS related events around the San Francisco Bay Area, including hosting lodging for three of the people that helped at the O'Reilly OSCON event earlier this month.
A large portion of the stuff you'll read about from my feed will be related to my (mis) adventures with getting KDE integrated with Exchange 2003 and later. I'll get into that a little more in the next post.
Some people in the community may know me as the guy that likes Perl, Qt, KDE, and does a hell of a lot of IT administration at a very un-KDE job. I've helped out at a few F/OSS related events around the San Francisco Bay Area, including hosting lodging for three of the people that helped at the O'Reilly OSCON event earlier this month.
A large portion of the stuff you'll read about from my feed will be related to my (mis) adventures with getting KDE integrated with Exchange 2003 and later. I'll get into that a little more in the next post.
KDE, PIM, and MAPI
Enough acronyms for you? I thought so.... For those that might not have heard, I'll be taking on the challenge of getting KDE PIM fully Exchange capable. This comes from my desire to kill Outlook and Entourage from my life and not force me to use a tool not integrated very well into my desktop of choice (Evolution).
So what is MAPI, you might ask? Well, MAPI stands for Messaging Application Programming Interface, which is the tier one option if you want a mail client to talk to Microsoft's Exchange server. This interface has gone through a number of changes over the years and thus far has been an elusive target for F/OSS projects. Lately a project called Openchange has been working on a library that much of my work on kdepim will be focused around integrating into KDE. This library covers all of the mess of having to write our own library to get access to these servers. As some know, this library is already used in a native MAPI method in Evolution that was released recently in the routine releases of GNOME. I'll just leverage that work and make KDE just as accessible to Enterprise users stuck in an MS powered shop.
I'll try to cover all the things I do to make this a reality, so stay tuned, Fun stuff ahead :)
So what is MAPI, you might ask? Well, MAPI stands for Messaging Application Programming Interface, which is the tier one option if you want a mail client to talk to Microsoft's Exchange server. This interface has gone through a number of changes over the years and thus far has been an elusive target for F/OSS projects. Lately a project called Openchange has been working on a library that much of my work on kdepim will be focused around integrating into KDE. This library covers all of the mess of having to write our own library to get access to these servers. As some know, this library is already used in a native MAPI method in Evolution that was released recently in the routine releases of GNOME. I'll just leverage that work and make KDE just as accessible to Enterprise users stuck in an MS powered shop.
I'll try to cover all the things I do to make this a reality, so stay tuned, Fun stuff ahead :)
August 2, 2009
APP-Get Progress
I've gotten the code for the SourceManager mostly done, need a dumper function at this point and then I can utilise it in app-update. After that LWP code will be high on my list as I add the DownloadManager functions.
May 13, 2009
Code tally....
While this normally falls under the "pissing match" category, I thought this would be fun to show how much code can be shaved off of a project that is written in pure Perl versus C++ and a Shell wrapper. The old genbasedir from Apt-RPM clocks in at 1650 lines of code, not counting the includes in the C++ source files. The NEW genbasedir.pl from APP-Get clocks in at 473 lines of code. A whopping 2/3 code reduction alone. Mind you, I could have cut corners, and trimmed the code down even further, however after looking over the code that genbasedir from Apt-RPM have between the shell script and the two C++ files, there are more than a few areas that look dodgy and lack proper error checking. Tomorrow, I'll move most of the functions from genbasedir.pl to a module to make it unit testable. Stay tuned to further developments :D
May 10, 2009
Reworking genbasedir....
To make genbasedir easier to work on in the future (when we move to using some SQL engine for the data store for metadata), I reworked the functions to allow us to use an integer based pkgId instead of using the string name. This should speed up the dependency map functions in app-get considerably when doing package look ups since string comparisons aren't the fastest operations in the world in any language.
As an aside, I've gotten buy in from Jeff Johnson, the maintainer of the version of RPM at www.rpm5.org to add in some Debian inspired tags to make my work on app-get easier. :)
As an aside, I've gotten buy in from Jeff Johnson, the maintainer of the version of RPM at www.rpm5.org to add in some Debian inspired tags to make my work on app-get easier. :)
May 7, 2009
New APP in town :D
Just an update, I'm working on a new dep solver for RPM and the eventual general successor OPM. The utility is called APP-get, a clone of apt-get that doesn't rely on private API in the underlying package manager to work and crufty code that Apt-RPM does. To get a feel of how APP-get will retrieve the metadata, I've been reimplementing genbasedir.
At this time, I've much of the new genbasedir written. It writes out a new plain text database format for binary and source package metadata called a lst.db. Each file is a single table of fields delimited by three pipe characters in a row. With this new set of lst.db files, a person can generate the metadata using standard UNIX tools if they so wish (though I don't know why you'd want to, since genbasedir does such a nice job of it :D).
While writing this, I've been studying the dependency map algorithms used by Apt and SmartPM. Armed with the knowledge of how they determine the dependencies, I should have a fairly easy time of writing the dependency manager in APP-get.
As I've been writing the code, I'm noting a lot of places that I'll be able to clear out Debian-isms from the design, which makes the codebase and design far simpler. At the same time, I'm seeing features I'd like integrated into RPM5 to make some aspects simpler, such as package priorities.
In Apt-RPM priorities are implemented as a kludgy list file (rpmpriorities) in /etc/apt/ that is checked for a string representing the name of a given package that uses a YAML-like format that defines packages according to named priorities which are more like the Essential tag in dpkg. I'd like to change this to be INSIDE the package's header to allow distributions to mark certain packages that don't normally have a require against another as Priority: Essential, or Priority: Normal. Also if this was merged into the header of the package, it would allow rpm to gain the ability to stop people from shooting themselves in the foot with rpm -e or app-get remove.
Well, enough chatter for now. Look forward to more progress on APP-get.
At this time, I've much of the new genbasedir written. It writes out a new plain text database format for binary and source package metadata called a lst.db. Each file is a single table of fields delimited by three pipe characters in a row. With this new set of lst.db files, a person can generate the metadata using standard UNIX tools if they so wish (though I don't know why you'd want to, since genbasedir does such a nice job of it :D).
While writing this, I've been studying the dependency map algorithms used by Apt and SmartPM. Armed with the knowledge of how they determine the dependencies, I should have a fairly easy time of writing the dependency manager in APP-get.
As I've been writing the code, I'm noting a lot of places that I'll be able to clear out Debian-isms from the design, which makes the codebase and design far simpler. At the same time, I'm seeing features I'd like integrated into RPM5 to make some aspects simpler, such as package priorities.
In Apt-RPM priorities are implemented as a kludgy list file (rpmpriorities) in /etc/apt/ that is checked for a string representing the name of a given package that uses a YAML-like format that defines packages according to named priorities which are more like the Essential tag in dpkg. I'd like to change this to be INSIDE the package's header to allow distributions to mark certain packages that don't normally have a require against another as Priority: Essential, or Priority: Normal. Also if this was merged into the header of the package, it would allow rpm to gain the ability to stop people from shooting themselves in the foot with rpm -e or app-get remove.
Well, enough chatter for now. Look forward to more progress on APP-get.
March 31, 2009
Updating to a new build root....
Yes, this is necessary (sadly). GCC 4.1.2 and Binutils 2.17 is fairly old at this point, and isn't safe to build KDE 4 against from what I've found due to linking and visibility bugs. At this point, I'm documenting my work in a script to allow future build roots to be built far faster for the update and security trees that will be following.
September 24, 2008
Updates....
The build root is coming along, and the AltimatOS Installer is also seeing much needed love. On other fronts, we finally have our registered PEN from the IANA to allow us to create the necessary schema for LDAP in AltimatOS.
Outside AltimatOS, I've been working a lot lately, which has slowed some things up, but never fear, I will get a release out eventually. :D
Outside AltimatOS, I've been working a lot lately, which has slowed some things up, but never fear, I will get a release out eventually. :D
Subscribe to:
Posts (Atom)