Please note:The SCons wiki is now restored from the attack in March 2013. All old passwords have been invalidated. Please reset your password if you have an account. If you note missing pages, please report them to webmaster@scons.org. Also, new account creation is currently disabled due to an ongoing spam flood (2013/08/27).

Sphinx as a documentation tool

techtonik: Historically, we use Docbook format to maintain documentation. Docbook is XML based format that should allow to convert source files into various formats, such as HTML, PDF, man pages and allow easy skinning and customizing. This was in theory. In practice it appeared that Docbook transformations in either XSLT or DSSSL languages are beautiful at micro level, but clumsy, hard to comprehend and maintain when you need to make a bigger thing done. The complexity in Docbook templates (and in customizations) grows significantly faster than with traditional template engines, and there are almost no free tools to debug XSLT templates. Docbook was good at the beginning, because there were no alternative, but today there is at least Sphinx.

We still maintain docs in Docbook, but we don't use XSLT for transformation. Transformation is made using even more "prehistoric" DSSSL templates and tool-chain that only runs on *nix like systems.

Greg: There are two issues with using Sphinx.

One issue is that Sphinx is a reStructuredText editor. Now, reStructuredText is a fine language for documentation, but we have many dozens of pages in XML that would have to be converted, and converted losslessly. I don't see an automated tool that would provide any assistance, and without such a tool, I can't see any way we could proceed.

Greg: The other, and far more important, issue is that I don't see how a workflow can be constructed to extract the examples, run them, verify that any changes are correct, and then proceed with the example output inserted in the documentation. Without such a workflow in hand, Sphinx is a non-starter.

DB: I have never worked with Sphinx (took a short look at the reST primer right now), but know Docbook a little. I don't think that reST is a good language for technical documentation at all, because it does not restrict the user to "structured authoring". You can freely assign attributes like italic or bold to text passages, which automatically leads to trouble when you try to get the output for, let's say, a guibutton consistent across all input files (and users!). This is where I feel that Docbook is still superior, you can also immediately check an input document against the DTD...and if it is valid, you are able to process it. There is (or: should be) a clear separation between the user that edits documents and one (or a few people) that actually process the XML to a HTML or PDF documentation. I don't like any support for fancy WYSIWYG to get in the way of this.

DB: It may be true that Docbook toolchains are difficult to set up and I would also vote for the step DSSSL->XSL asap, but with Sphinx you also need LaTeX for the conversion to PDF as a prerequisite. This could be another source for trouble because LaTeX distributions can differ a lot, when looking at which packages are installed and which not, for example. So, you don't necessarily get it all for free with Sphinx...

DB: Yes, Docbook stylesheets are pretty complex and need some care if you try to do special things. But if you stick to the default set of XSL for HTML and PDF (=FO), some basic customization like "displaying the SCons logo in page headers" is quite easy to accomplish.

Finally, I would like to point at a paragraph in the reST primer about sections. It talks about the adornments for section headers and requires the user (!) to "Be consistent, since all sections marked with the same adornment style are deemed to be at the same level".

Sorry, but this is not the right path to follow, or is it?

All together, our main concern should not be to make documentation (and its processing) easier for the core developer, but for the average user. If we can agree on this, then Docbook is still my choice. I will come up with an example workflow for Docbook, using XIncludes and PIs (processing instructions) in the next days and post it to this page.

Improving the Docbook workflow

Overall goals are:

Validation/Editing

For a start, I created a first draft of a special "SCons Docbook DTD".

dtd.tgz

It can be used to check the User's guide for validity, after changing the doctype in main.xml accordingly. I also prepared configuration packages for two visual XML editors, namely the Xmlmind editor

xxe4.tgz

and Syntext's Serna

scons.tgz

. They should enable you to edit the User's guide WYSIWYG style. For more infos about how to install these archives, please have a look at the "SCons Docbook DTD Reference manual"

docbook_dtd.pdf

Some issues to attack

I had another close look at the top-level SConstruct and the SConscript in the doc folder. The following points are worth further investigation, in my opinion:

doclifter_examples.tgz

scons-time.html

All together, I see good chances to trade a new dependency on xsltproc (or the libxml2/libxslt Python modules) for removing groff, man2html, tidy, fig2dev and (at least for HTML) jw. When another FOP renderer is installed on the user's system (fop,XEP,...), it could replace the whole jade/openjade/jadetex/pdfjadetex stuff for PDF output.

SCons titlepages

I started to work on a design for the SCons documentation titlepages. Here is a first version of the User Guide 2.0.0,

scons_manual.pdf

illustrating what the current customizations look like.

What we have, so far

Soon to come

I am currently working on restructuring the User Guide source to use XIncludes instead of entity references. My thoughts revolve around a processing chain that looks something like this:

  1. User edits some input file(s) for documentation.
  2. Ensure that all required include files are present, e.g. for the example outputs of the User Guide. If not, create dummy files where needed.
  3. Validate all single input files.
  4. (optional) Rerun all SCons examples in the User Guide. Compare new vs. old output and report differences. Also check for errors during the SCons processing, in order to detect flaws in the examples (like missing files).
  5. Preprocess the User Guide by resolving all XIncludes and replacing special SCons DTD elements, e.g. scons_example, with valid Docbook counterparts.
  6. Finally, create the different output formats with the SCons Docbook Tool.

A normal user or technical writer would usually stop with step 3 and then commit his changes. The points 4-6 should be reserved for members of the Release Team, when a new version is about to be published.

DeveloperGuide/Documentation/Discussion (last edited 2010-12-31 11:31:00 by DirkBaechle)