Karl Pietrzak's Google Summer of Code 2006 Proposal

Introduction

About the Author

My name is Karl Pietrzak, and I am currently a senior at the Rochester Institute of Technology in Rochester, NY, where I will graduate with a BS in Computer Science in November 2006. I was born in Kraków, Poland and I came to the United States at the tender age of five. As such, I can speak, read, and write Polish, as well an a tinge of Spanish.

I am writing this document in order to receive at a SCons Google Summer of Code 2006 project. My interest in SCons started with my massive frustrations with the autotools (i.e., autoconf, automake, etc.); I feel the world could use something better, and SCons--with the power of Python behind--is that alternative.

Throughout the past few years of mySCons usage for both commercial and personal use, I have contributed a few small patches where I saw a bug or a need for a minor improvement. Now, however, I would love the opportunity to become a SCons developer so that my involvement becomes more substantial and permanent. I am very excited to get the opportunity to work on this proposal for this summer!

Contact Information

I can be contacted by any the following methods:

Summary of Proposal

Despite all the power and functionality, it is difficult to when, how, and why the SCons engine is doing something. A logging framework would mitigate this problem by allowing SCons users to monitor what the engine is doing, and by allowing SCons developers to trace functions with minimal overhead.

The end goal is to have an XML build log, along with a plethora of minor features such as colorized output. These features will be able to be "chained", so a developer will be able to output colorized gcc output and an XML build log at the same time. This will also enable powerful integration with build management tools such asBuildBot.

Because of my class schedule, I would only work during the first half of the summer (until July 10th). I fully understand I will not receive the full $4500 from Google.

Qualifications

The qualifications for this project can be divided into the following sections.

Skill Set

I possess the following skills which I feel make me a qualified candidate for this SCons project:

• implemented a SCons build system for the Software Research and Development Group at Thomson Legal and Regulatory

• contributed a few patches to the SCons project

• large interest in Python and build systems

• large experience in using Ant

• giving a variety of presentations on both technical and non-technical audiences

• experience in a variety of bug trackers, including BugZilla and TeamShare

• years spent developing Java software using Eclipse on both Linux and Microsoft Windows

• UML modeling using Umbrello UML modeler, Borland Together, and Rational Rose.

• used a variety of SCMs, including Subversion and CVS

• used a variety of unit testing frameworks, especially JUNIT , CppUnit , and the logging module in Python

Open-source Project Experience

I have contributed to a variety of open-source projects in a variety of capacities, ranging from tester to developer. I have:

• contributed a few small patches to the PyDev project for better auto-completion of Python code

• submitted a few bug reports to the PyDev project

  • PyDev refactoring crash, Bug 1426745

• submitted a few small patches also to the SCons project, a Python-based build management tool used by Blender 3D and many other projects

• tested a few releases and submitted bug reports for the Autopackage project

• reported bugs to a plethora of open-source projects, including KDE, Umbrello UML modeler, openSUSE

  • Umbrello UML Modeler

    • changing case in file names, Bug 124273

    • diagrams print HUGE, Bug 124333

    • File menu should have print, Bug 124330

    • deleting association name in state diagram may cause crash, Bug 124587

    • cannot put subsystems within subsystems in Component diagram, Bug 124278

  • openSUSE

    • Python path does not contain /usr/local, Bug 149809

    • Python's site-packages should contain a directory in /usr/local, Bug 149843

    • helix backend for Amarok uses lots of memory, Bug 163481

    • latest Scribus fails because of 32bit / 64bit Python issues, Bug 163849

  • SCons

    • contributed to many Wiki articles

    • started my own Wiki page dealing with using having an RPATH of $ORIGIN using SCons

    • created Wiki page that deals with delegating SCons responsibilities to help with development

    • better Sun CC compiler support, Patch 1192558

    • missing function now documented in man page Patch 1208536

    • recursive code fix, Patch 1195203

    • improper shared object suffix when using Sun CC compiler, Patch 1191940

• contributed to many Wikipedia articles

In addition to those, I have experience using the following open-source tools:

• Subversion

• CVS

• Eclipse

Commercial Experience

I have worked on a variety of projects for IBM, Advanced Micro Devices, and Thomson Legal and Regulatory throughout my (so far) short career.

These have included, but are not limited to:

• Oracle database manipulation using JDBC

• XML parsing
• XML creation

• advanced GUI programming in Java's Swing

• RPM creation

• middle-ware programming using XMPP

• BIOS testing
• giving presentations on a variety of topics
• demo-ing software for non-technical users

I have used SCons in a commercial setting to create a hierarchical build for an internal library that would run using the Microsoft Visual Studio compiler, gcc, and the Sun CC compiler.

Development Methodology

My development methodology can be described succinctly as careful, tested, and documented. I'm a big fan of software assistance in any kind of endeavour, specially software engineering. I tend to use project management, bug trackers, IDEs, and unit testing frameworks.

Specifically, the following will explain my development methodology in more detail:

• Unit testing is a big part of my methodology, no matter what language it is. I have used JUNIT for unit testing in Java, CppUnit in C++, and the logging module in Python. I tend to sleep better at night knowing my code is unit tested.

• I feel documentation is extremely important, and not just code documentation. I believe that a picture is worth a thousand words, and as a result, I tend to use UML modeling tools. Specifically, I plan on using the Umbrello UML Modeler, which has (limited) Python support.

Purpose

Although I have contributed to a variety of open-source projects, I have not become a major developer for any of them due to time constraints and other circumstances. The Google Summer of Code allows me to become the full-time open-source project developer I've always wanted to!

I have been followed the SCons mailing lists (both scons-user and scons-devel) for a few years now. I would really enjoy being a full-time SCons developer, and I feel I can make great impact on not only SCons development but also its user base.

The following sections list the tasks I would like to perform for the summer.

Logging Framework

Use Cases

A full-fledged logging framework would allow easy transfer of information from SCons to other build management tools by having easily-parsible log files which will contain all build information. There are current hacks to enable some of this (e.g., BuildLog), but they are not nearly complete or usable by external tools.

The log file must contain at least the following details:

• a list of all targets
• whether target passed or failed
• the types of each target
• the dependencies
• any input and output of the target
• timestamps

The final goal would be to allow SCons to integrate easily with build tools such as:

• CruiseControl

• BuildBot

• Maven

Without a full-fledged logging framework, it is close to impossible to gather the aforementioned data because parsing standard out is not scalable, and it's just bad practice. After the completion of this project, a tool like BuildBot will be able to parse the log XML file, and, for example:

• spit out to an IRC channel that libSDL.so failed

• email the developer in charge of Windows support that EnvironmentVariablesParsingTest failed

• provide a list of tests that did not run because the class XmppNetworking did not compile

With the notation of build listeners and event handlers will come the ability to filter:

• unneeded build information in a "build-simple.log" file, for example
• add as much information as possible in a "build-complete.log" file, for example

Independent software vendors (i.e., ISVs) will be able to create their own handlers, listeners, and filters to create their own complete build solution. Python's large software collection will allow some vendors, to, for example, send each build.log to a database, or just send a message to an IRC channel. Combined with a tool like BuildBot, the possibilities are limitless.

Description

Developers need a logging framework for SCons, similar to loggers and listeners in Ant. This is one area that SCons has been lacking in compared to the competition. Currently the only way to get results from SCons is to capture standard out and parse it. This is not very useful, however, as there is no:

• distinction which target is being built
• whether what's being built is a executable, shared library, etc.
• overall result document that provides a summary

Ideally, there would be multiple ways to output and format results. This lends itself to a concept similar to the Python logging module. Specifically, there will be a notion of formatters and handlers, interchangeable and joinable like LEGO parts:

• handlers

  • ConsoleHandler: prints to stdout/stderror

  • FileHandler: prints to a File

  • DBHandler: sends results to a DB
• formatters

  • SimpleFormatter: just prints out everything

  • XMLFormatter: print out an XML to some predefined schema

  • FancyFormatter: simply prints to stdout/stderr, but gives more information such as time, etc.

Using this vocabulary, the current SCons default would be a ConsoleHandler with a SimpleFormatter. The following UML diagram illustrates this relationship:

I would develop the following components using this framework:

• ColorFormatter to colorize the output of gcc and other compiles

• XMLFormatter using an XML schema

• PipeHandler which writes output to a named pipe

• GUI which would parse the log files and let you select--dynamically--how much output you want to see

Performance Measuring

Care will have to be taken to ensure that the logging framework does not use too many resources. Of course, the more complicated the handlers and formatters, the longer it will take to execute (e.g., XMLFormatter with a RotatingFileHandler will take much longer than SimpleFormatter with a ConsoleHandler).

Benchmarks will be taken and compared to the current SCons implemention to ensure that the new logging framework does not place have significant impacts on performance.

Architecture Re-work

If there is enough time, I will also work on some architecture rework, as described on the main SCons Wiki for Summer of Code page. This will entail using the Configure context stuff to find tools, instead of the hard-coded generate() methods in each Tool module. This will relieve SCons of the burden of having to the availability of each Tool on the given machine.

Timeline

I would like to point out that all of my ideas for this proposal are official SCons proposals. I have decided to perform a few large-size tasks:

Conclusion

I am very qualified for this SCons Google Summer of Code proposal. I have worked with the SCons team for a number of years, and I know SCons well.