Doxygen Builder
The Doxygen builder will generate doxygen docs, given a Doxyfile. It will scan the Doxyfile and determine what directories will be created and what sources are used to generate the docs. This frees you up from writing special code to manage clean up and regeneration of the docs.
N.B. It seems there was a bug in scons versions before 0.97.0d20070918 which prevented dependencies from working for this builder. See this email to the mailing list for details.
Russel Winder has started a Bazaar branch on Launchpad https://code.launchpad.net/~russel/sconsaddons/doxygen to try and acrete all the work found on and from this page into a single good tool. You may want to use this version rather than trying to replicate the various changes needed to reconcile all the variations reported on this page. If you find any errors or improvements please contribute them back via the Bazaar branch rather than posting code to this page. Thanks.
Please use the Bazaar branch above and not any of the codes below, which are left here just to preserve the historical record.
The original instructions for use are given below.
Save the following script as file 'doxygen.py' and put its directory in the 'toolpath' list as shown in "Usage" below.
1 # vim: set et sw=3 tw=0 fo=awqorc ft=python:
2 #
3 # Astxx, the Asterisk C++ API and Utility Library.
4 # Copyright (C) 2005, 2006 Matthew A. Nicholson
5 # Copyright (C) 2006 Tim Blechmann
6 #
7 # This library is free software; you can redistribute it and/or
8 # modify it under the terms of the GNU Lesser General Public
9 # License version 2.1 as published by the Free Software Foundation.
10 #
11 # This library is distributed in the hope that it will be useful,
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
14 # Lesser General Public License for more details.
15 #
16 # You should have received a copy of the GNU Lesser General Public
17 # License along with this library; if not, write to the Free Software
18 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
19
20 import os
21 import os.path
22 import glob
23 from fnmatch import fnmatch
24
25 def DoxyfileParse(file_contents):
26 """
27 Parse a Doxygen source file and return a dictionary of all the values.
28 Values will be strings and lists of strings.
29 """
30 data = {}
31
32 import shlex
33 lex = shlex.shlex(instream = file_contents, posix = True)
34 lex.wordchars += "*+./-:"
35 lex.whitespace = lex.whitespace.replace("\n", "")
36 lex.escape = ""
37
38 lineno = lex.lineno
39 token = lex.get_token()
40 key = token # the first token should be a key
41 last_token = ""
42 key_token = False
43 next_key = False
44 new_data = True
45
46 def append_data(data, key, new_data, token):
47 if new_data or len(data[key]) == 0:
48 data[key].append(token)
49 else:
50 data[key][-1] += token
51
52 while token:
53 if token in ['\n']:
54 if last_token not in ['\\']:
55 key_token = True
56 elif token in ['\\']:
57 pass
58 elif key_token:
59 key = token
60 key_token = False
61 else:
62 if token == "+=":
63 if key not in data:
64 data[key] = []
65 elif token == "=":
66 data[key] = []
67 else:
68 append_data( data, key, new_data, token )
69 new_data = True
70
71 last_token = token
72 token = lex.get_token()
73
74 if last_token == '\\' and token != '\n':
75 new_data = False
76 append_data( data, key, new_data, '\\' )
77
78 # compress lists of len 1 into single strings
79 for k, v in data.items():
80 if len(v) == 0:
81 data.pop(k)
82
83 # items in the following list will be kept as lists and not converted to strings
84 if k in ["INPUT", "FILE_PATTERNS", "EXCLUDE_PATTERNS"]:
85 continue
86
87 if len(v) == 1:
88 data[k] = v[0]
89
90 return data
91
92 def DoxySourceScan(node, env, path):
93 """
94 Doxygen Doxyfile source scanner. This should scan the Doxygen file and add
95 any files used to generate docs to the list of source files.
96 """
97 default_file_patterns = [
98 '*.c', '*.cc', '*.cxx', '*.cpp', '*.c++', '*.java', '*.ii', '*.ixx',
99 '*.ipp', '*.i++', '*.inl', '*.h', '*.hh ', '*.hxx', '*.hpp', '*.h++',
100 '*.idl', '*.odl', '*.cs', '*.php', '*.php3', '*.inc', '*.m', '*.mm',
101 '*.py',
102 ]
103
104 default_exclude_patterns = [
105 '*~',
106 ]
107
108 sources = []
109
110 data = DoxyfileParse(node.get_contents())
111
112 recursive = data.get("RECURSIVE") == "YES"
113
114 file_patterns = data.get("FILE_PATTERNS", default_file_patterns)
115 exclude_patterns = data.get("EXCLUDE_PATTERNS", default_exclude_patterns)
116
117 for node in data.get("INPUT", []):
118 if os.path.isfile(node):
119 sources.append(node)
120 elif os.path.isdir(node):
121 if recursive:
122 for root, dirs, files in os.walk(node):
123 for f in files:
124 filename = os.path.join(root, f)
125
126 pattern_check = any(fnmatch(filename, y) for y in file_patterns)
127 exclude_check = any(fnmatch(filename, y) for y in exclude_patterns)
128
129 if pattern_check and not exclude_check:
130 sources.append(filename)
131 else:
132 for pattern in file_patterns:
133 sources.extend(glob.glob("/".join([node, pattern])))
134
135 sources = [env.File(path) for path in sources]
136 return sources
137
138
139 def DoxySourceScanCheck(node, env):
140 """Check if we should scan this file"""
141 return os.path.isfile(node.path)
142
143 def DoxyEmitter(source, target, env):
144 """Doxygen Doxyfile emitter"""
145 # possible output formats and their default values and output locations
146 output_formats = {
147 "HTML": ("YES", "html"),
148 "LATEX": ("YES", "latex"),
149 "RTF": ("NO", "rtf"),
150 "MAN": ("NO", "man"),
151 "XML": ("NO", "xml"),
152 }
153
154 data = DoxyfileParse(source[0].get_contents())
155
156 targets = []
157 out_dir = data.get("OUTPUT_DIRECTORY", ".")
158
159 # add our output locations
160 for k, v in output_formats.items():
161 if data.get("GENERATE_" + k, v[0]) == "YES":
162 targets.append(env.Dir( os.path.join(out_dir, data.get(k + "_OUTPUT", v[1]))) )
163
164 # don't clobber targets
165 for node in targets:
166 env.Precious(node)
167
168 # set up cleaning stuff
169 for node in targets:
170 env.Clean(node, node)
171
172 return (targets, source)
173
174 def generate(env):
175 """
176 Add builders and construction variables for the
177 Doxygen tool. This is currently for Doxygen 1.4.6.
178 """
179 doxyfile_scanner = env.Scanner(
180 DoxySourceScan,
181 "DoxySourceScan",
182 scan_check = DoxySourceScanCheck,
183 )
184
185 import SCons.Builder
186 doxyfile_builder = SCons.Builder.Builder(
187 action = "cd ${SOURCE.dir} && ${DOXYGEN} ${SOURCE.file}",
188 emitter = DoxyEmitter,
189 target_factory = env.fs.Entry,
190 single_source = True,
191 source_scanner = doxyfile_scanner,
192 )
193
194 env.Append(BUILDERS = {
195 'Doxygen': doxyfile_builder,
196 })
197
198 env.AppendUnique(
199 DOXYGEN = 'doxygen',
200 )
201
202 def exists(env):
203 """
204 Make sure doxygen exists.
205 """
206 return env.Detect("doxygen")
Usage
Note added by Robert Lupton, rhl@astro.princeton.edu
I had to make two changes to make this work.
1. I had to double the $ in the Action:
action = env.Action("cd $${SOURCE.dir} && $${DOXYGEN} $${SOURCE.file}"),
2. As written, the Builder runs from the top level directory TOP when it scans the doxyfile, but runs doxygen from the source directory. This means that it you set INPUT to e.g. "..", the scanner will set the dependencies to refer to all files found by searching TOP/.. --- which isn't what you want!
Here's a fix (around line 122):
#
# We're running in the top-level directory, but the doxygen
# configuration file is in the same directory as node; this means
# that relative pathnames in node must be adjusted before they can
# go onto the sources list
#
conf_dir = os.path.dirname(str(node))
for node in data.get("INPUT", []):
if not os.path.isabs(node):
node = os.path.join(conf_dir, node)
Note added by SK
The code above originally had the following initialization of the action = argument when creating the Builder:
action = env.Action("cd ${SOURCE.dir} && ${DOXYGEN} ${SOURCE.file}"),The env.Action() call explicitly asks for the string to be evaluated at call time, when the action is created, which is why Robert found it necessary to double the $ characters. (It probably did work in earlier versions, but variable substitution in construction environment methods has been "cleaned up" in some recent versions, and this may have been a casualty.)
Since there's nothing special about the action being created (no strfunction, for example), it's much simpler to just pass the command-line string to the Builder and let SCons create the Action object.
action = "cd ${SOURCE.dir} && ${DOXYGEN} ${SOURCE.file}",I updated the code above so that people who cut and paste without reading all the way to the bottom of the page shouldn't have this problem.
Additional update 6 March 2007: There was also a left-over env.Builder that had to be changed to the raw form of the call to avoid variable expansion earlier than we want. Code above changed.
Note added by Dirk Reiners, dirk@louisiana.edu
I added two (at least for me ;)) important features of doxygen: variable substituion and hierarchical doxygen files.
Variable substituion allows doxygen to reference variables from the scons environment using $(VARNAME). This is very useful for things like version numbers or for only having certain parts (as defined by scons) included in the documentation without having to mess with doxygen files.
Hierarchical doxygen files just interpret the @INCLUDE key as an include.
I also had trouble with files that started with a key, I fixed that.
The changes are a little longish for putting them in the text, so I attached the changed file doxygen.py_dr_070226. Note that I'm a python newbie, so there are probably more elegant ways to do some of the things I did. Feel free to change them.
Hope it helps.
Note added by anonymous
Replace the line token = env[token[2:-1]] by token = env[token[2:token.find(")")]] to suppress wrong warnings when using environment variables in Doxyfile as path. (Like in "$(MY_LIBRARY)/include")
Note added by Christoph Boehme, cxb632@bham.ac.uk
Robert Lupton noted that you have to change the source paths if you keep your Doxyfile in a subdirectory and use relative paths. I found that I had to do the same for the target path in the Doxyfile. Therefore, I added the following lines after line 160:
This is essentially the same code as Robert Lupton's.
Adding tagfile to targets and html templates to sources
The following code adds the tagfile to the target list. I added it in line 166:
To add the html templates from the Doxyfile to the list of sources, you need to apply Robert Lupton's change and add the following snippet in line 137:
1 # Add additional files to the list ouf source files:
2 def append_additional_source(option):
3 file = data.get(option, "")
4 if file != "":
5 if not os.path.isabs(file):
6 file = os.path.join(conf_dir, file)
7 if os.path.isfile(file):
8 sources.append(file)
9
10 append_additional_source("HTML_STYLESHEET")
11 append_additional_source("HTML_HEADER")
12 append_additional_source("HTML_FOOTER")
You can easily add dependencies on other output file templates by adding additional calls to append_additional_source().
Addendum 18 July 2007: I added some code to add tagfiles to the list of sources. Since the tagfiles-option allows for equal-signs in the value, I had to change the parsing code a bit. The new code is found in file doxygen.py . This file also includes the other changes I have made.
Note added by Reinderien
I believe that the line
"MAN": ("YES", "man"),should read
"MAN": ("NO", "man"),I was getting unnecessary doxygen runs, and scons --debug-explain showed that doxygen.py thinks the man target is on by default when it isn't.