Chapter 20. Extending SCons: Writing Your Own Scanners

SCons has built-in Scanners that know how to look in C/C++, Fortran, D, IDL, LaTeX, Python and SWIG source files for information about other files that targets built from those files depend on. For example, if you have a file format which uses #include to specify files which should be included into the source file when it is processed, you can use an existing scanner already included in SCons. You can use the same mechanisms that SCons uses to create its built-in Scanners to write Scanners of your own for file types that SCons does not know how to scan "out of the box."

20.1. A Simple Scanner Example

Suppose, for example, that we want to create a simple Scanner for .k files. A .k file contains some text that will be processed, and can include other files on lines that begin with include followed by a file name:

include filename.k
    

Scanning a file will be handled by a Python function that you must supply. Here is a function that will use the Python re module to scan for the include lines in our example:

import re

include_re = re.compile(r'^include\s+(\S+)$', re.M)

def kfile_scan(node, env, path, arg=None):
    contents = node.get_text_contents()
    return env.File(include_re.findall(contents))
    

It is important to note that you have to return a list of File nodes from the scanner function, simple strings for the file names won't do. As in the examples we are showing here, you can use the File function of your current construction environment in order to create nodes on the fly from a sequence of file names with relative paths.

The scanner function must accept the four specified arguments and return a list of implicit dependencies. Presumably, these would be dependencies found from examining the contents of the file, although the function can perform any manipulation at all to generate the list of dependencies.

node

An SCons node object representing the file being scanned. The path name to the file can be used by converting the node to a string using the str function, or an internal SCons get_text_contents object method can be used to fetch the contents.

env

The construction environment in effect for this scan. The scanner function may choose to use construction variables from this environment to affect its behavior.

path

A list of directories that form the search path for included files for this Scanner. This is how SCons handles the $CPPPATH and $LIBPATH variables.

arg

An optional argument that can be passed to this scanner function when it is called from a scanner instance. The argument is only supplied if it was given when the scanner instance is created (see the manpage section "Scanner Objects"). This can be useful, for example, to distinguish which scanner type called us, if the function might be bound to several scanner objects. Since the argument is only supplied in the function call if it was defined for that scanner, the function needs to be prepared to possibly be called in different ways if multiple scanners are expected to use this function - giving the parameter a default value as shown above is a good way to do this. If the function to scanner relationship will be 1:1, just make sure they match.

A scanner object is created using the Scanner function, which typically takes an skeys argument to associate a file suffix with this Scanner. The scanner object must then be associated with the $SCANNERS construction variable in the current construction environment, typically by using the Append method:

kscan = Scanner(function=kfile_scan, skeys=['.k'])
env.Append(SCANNERS=kscan)
    

Let's put this all together. Our new file type, with the .k suffix, will be processed by a command named kprocess, which lives in non-standard location /usr/local/bin, so we add that path to the execution environment so SCons can find it. Here's what it looks like:

import re

include_re = re.compile(r'^include\s+(\S+)$', re.M)

def kfile_scan(node, env, path):
    contents = node.get_text_contents()
    includes = include_re.findall(contents)
    return env.File(includes)

kscan = Scanner(function=kfile_scan, skeys=['.k'])
env = Environment()
env.AppendENVPath('PATH', '/usr/local/bin')
env.Append(SCANNERS=kscan)

env.Command('foo', 'foo.k', 'kprocess < $SOURCES > $TARGET')
      

Assume a foo.k file like this:

some initial text
include other_file
some other text
      

Now if we run scons we can see that the scanner works - it identified the dependency other_file via the detected include line, although we get an error message because we forgot to create that file!

% scons -Q
scons: *** [foo] Implicit dependency `other_file' not found, needed by target `foo'.