Karl Pietrzak's Google Summer of Code 2006 Proposal
- Karl Pietrzak's Google Summer of Code 2006 Proposal
About the Author
My name is Karl Pietrzak, and I am currently a senior at the Rochester Institute of Technology in Rochester, NY, where I will graduate with a BS in Computer Science in November 2006. I was born in Kraków, Poland and I came to the United States at the tender age of five. As such, I can speak, read, and write Polish, as well an a tinge of Spanish.
I am writing this document in order to receive at a SCons Google Summer of Code 2006 project. My interest in SCons started with my massive frustrations with the autotools (i.e., autoconf, automake, etc.); I feel the world could use something better, and SCons--with the power of Python behind--is that alternative.
Throughout the past few years of mySCons usage for both commercial and personal use, I have contributed a few small patches where I saw a bug or a need for a minor improvement. Now, however, I would love the opportunity to become a SCons developer so that my involvement becomes more substantial and permanent. I am very excited to get the opportunity to work on this proposal for this summer!
I can be contacted by any the following methods:
PGP Key ID
Summary of Proposal
Despite all the power and functionality, it is difficult to when, how, and why the SCons engine is doing something. A logging framework would mitigate this problem by allowing SCons users to monitor what the engine is doing, and by allowing SCons developers to trace functions with minimal overhead.
The end goal is to have an XML build log, along with a plethora of minor features such as colorized output. These features will be able to be "chained", so a developer will be able to output colorized gcc output and an XML build log at the same time. This will also enable powerful integration with build management tools such asBuildBot.
Because of my class schedule, I would only work during the first half of the summer (until July 10th). I fully understand I will not receive the full $4500 from Google.
The qualifications for this project can be divided into the following sections.
I possess the following skills which I feel make me a qualified candidate for this SCons project:
contributed a few patches to the SCons project
large interest in Python and build systems
large experience in using Ant
- giving a variety of presentations on both technical and non-technical audiences
Open-source Project Experience
I have contributed to a variety of open-source projects in a variety of capacities, ranging from tester to developer. I have:
contributed a few small patches to the PyDev project for better auto-completion of Python code
submitted a few bug reports to the PyDev project
tested a few releases and submitted bug reports for the Autopackage project
- contributed to many Wiki articles
started my own Wiki page dealing with using having an RPATH of $ORIGIN using SCons
created Wiki page that deals with delegating SCons responsibilities to help with development
better Sun CC compiler support, Patch 1192558
missing function now documented in man page Patch 1208536
recursive code fix, Patch 1195203
improper shared object suffix when using Sun CC compiler, Patch 1191940
contributed to many Wikipedia articles
In addition to those, I have experience using the following open-source tools:
These have included, but are not limited to:
Oracle database manipulation using JDBC
- XML parsing
- XML creation
advanced GUI programming in Java's Swing
- RPM creation
middle-ware programming using XMPP
- BIOS testing
- giving presentations on a variety of topics
- demo-ing software for non-technical users
I have used SCons in a commercial setting to create a hierarchical build for an internal library that would run using the Microsoft Visual Studio compiler, gcc, and the Sun CC compiler.
My development methodology can be described succinctly as careful, tested, and documented. I'm a big fan of software assistance in any kind of endeavour, specially software engineering. I tend to use project management, bug trackers, IDEs, and unit testing frameworks.
Specifically, the following will explain my development methodology in more detail:
Unit testing is a big part of my methodology, no matter what language it is. I have used JUNIT for unit testing in Java, CppUnit in C++, and the logging module in Python. I tend to sleep better at night knowing my code is unit tested.
I feel documentation is extremely important, and not just code documentation. I believe that a picture is worth a thousand words, and as a result, I tend to use UML modeling tools. Specifically, I plan on using the Umbrello UML Modeler, which has (limited) Python support.
Although I have contributed to a variety of open-source projects, I have not become a major developer for any of them due to time constraints and other circumstances. The Google Summer of Code allows me to become the full-time open-source project developer I've always wanted to!
I have been followed the SCons mailing lists (both scons-user and scons-devel) for a few years now. I would really enjoy being a full-time SCons developer, and I feel I can make great impact on not only SCons development but also its user base.
The following sections list the tasks I would like to perform for the summer.
A full-fledged logging framework would allow easy transfer of information from SCons to other build management tools by having easily-parsible log files which will contain all build information. There are current hacks to enable some of this (e.g., BuildLog), but they are not nearly complete or usable by external tools.
The log file must contain at least the following details:
- a list of all targets
- whether target passed or failed
- the types of each target
- the dependencies
- any input and output of the target
The final goal would be to allow SCons to integrate easily with build tools such as:
Without a full-fledged logging framework, it is close to impossible to gather the aforementioned data because parsing standard out is not scalable, and it's just bad practice. After the completion of this project, a tool like BuildBot will be able to parse the log XML file, and, for example:
- spit out to an IRC channel that libSDL.so failed
email the developer in charge of Windows support that EnvironmentVariablesParsingTest failed
provide a list of tests that did not run because the class XmppNetworking did not compile
With the notation of build listeners and event handlers will come the ability to filter:
- unneeded build information in a "build-simple.log" file, for example
- add as much information as possible in a "build-complete.log" file, for example
Independent software vendors (i.e., ISVs) will be able to create their own handlers, listeners, and filters to create their own complete build solution. Python's large software collection will allow some vendors, to, for example, send each build.log to a database, or just send a message to an IRC channel. Combined with a tool like BuildBot, the possibilities are limitless.
Developers need a logging framework for SCons, similar to loggers and listeners in Ant. This is one area that SCons has been lacking in compared to the competition. Currently the only way to get results from SCons is to capture standard out and parse it. This is not very useful, however, as there is no:
- distinction which target is being built
- whether what's being built is a executable, shared library, etc.
- overall result document that provides a summary
Ideally, there would be multiple ways to output and format results. This lends itself to a concept similar to the Python logging module. Specifically, there will be a notion of formatters and handlers, interchangeable and joinable like LEGO parts:
ConsoleHandler: prints to stdout/stderror
FileHandler: prints to a File
- DBHandler: sends results to a DB
SimpleFormatter: just prints out everything
- XMLFormatter: print out an XML to some predefined schema
FancyFormatter: simply prints to stdout/stderr, but gives more information such as time, etc.
Using this vocabulary, the current SCons default would be a ConsoleHandler with a SimpleFormatter. The following UML diagram illustrates this relationship:
I would develop the following components using this framework:
ColorFormatter to colorize the output of gcc and other compiles
- XMLFormatter using an XML schema
PipeHandler which writes output to a named pipe
- GUI which would parse the log files and let you select--dynamically--how much output you want to see
Care will have to be taken to ensure that the logging framework does not use too many resources. Of course, the more complicated the handlers and formatters, the longer it will take to execute (e.g., XMLFormatter with a RotatingFileHandler will take much longer than SimpleFormatter with a ConsoleHandler).
Benchmarks will be taken and compared to the current SCons implemention to ensure that the new logging framework does not place have significant impacts on performance.
If there is enough time, I will also work on some architecture rework, as described on the main SCons Wiki for Summer of Code page. This will entail using the Configure context stuff to find tools, instead of the hard-coded generate() methods in each Tool module. This will relieve SCons of the burden of having to the availability of each Tool on the given machine.
I would like to point out that all of my ideas for this proposal are official SCons proposals. I have decided to perform a few large-size tasks:
Goal and/or Deliverable
May 23 - May 31
solid understanding what it would take implement the logging framework
June 1 - June 14
basic logging framework is in place
June 14 - June 30
adding formatters and handlers such as XMLFormatter, FileHandler, GccColorizer, etc.
July 1 - July 5
I feel it's best to reserve one week for any unforeseen issues, such as source control, integration, etc.