10.2. Command-Line variable=value Build Variables

10.2. Command-Line `variable`=`value` Build Variables
Prev	Chapter 10. Controlling a Build From the Command Line	Next

10.2. Command-Line `variable`=`value` Build Variables

You may want to control various aspects of your build by allowing the user to specify variable=value values on the command line. For example, suppose you want users to be able to build a debug version of a program by running SCons as follows:

% scons -Q debug=1

SCons provides an ARGUMENTS dictionary that stores all of the variable=value assignments from the command line. This allows you to modify aspects of your build in response to specifications on the command line. (Note that unless you want to require that users always specify a variable, you probably want to use the Python dictionary get method, which allows you to specify a default value to be used if there is no specification on the command line.)

The following code sets the $CCFLAGS construction variable in response to the debug flag being set in the ARGUMENTS dictionary:

env = Environment()
debug = ARGUMENTS.get('debug', 0)
if int(debug):
    env.Append(CCFLAGS='-g')
env.Program('prog.c')

This results in the -g compiler option being used when debug=1 is used on the command line:

% scons -Q debug=0
cc -o prog.o -c prog.c
cc -o prog prog.o
% scons -Q debug=0
scons: `.' is up to date.
% scons -Q debug=1
cc -o prog.o -c -g prog.c
cc -o prog prog.o
% scons -Q debug=1
scons: `.' is up to date.

Notice that SCons keeps track of the last values used to build the object files, and as a result correctly rebuilds the object and executable files only when the value of the debug argument has changed.

The ARGUMENTS dictionary has two minor drawbacks. First, because it is a dictionary, it can only store one value for each specified keyword, and thus only "remembers" the last setting for each keyword on the command line. This makes the ARGUMENTS dictionary inappropriate if users should be able to specify multiple values on the command line for a given keyword. Second, it does not preserve the order in which the variable settings were specified, which is a problem if you want the configuration to behave differently in response to the order in which the build variable settings were specified on the command line.

To accomodate these requirements, SCons provides an ARGLIST variable that gives you direct access to variable=value settings on the command line, in the exact order they were specified, and without removing any duplicate settings. Each element in the ARGLIST variable is itself a two-element list containing the keyword and the value of the setting, and you must loop through, or otherwise select from, the elements of ARGLIST to process the specific settings you want in whatever way is appropriate for your configuration. For example, the following code to let the user add to the CPPDEFINES construction variable by specifying multiple define= settings on the command line:

cppdefines = []
for key, value in ARGLIST:
    if key == 'define':
        cppdefines.append(value)
env = Environment(CPPDEFINES=cppdefines)
env.Object('prog.c')

Yields the following output:

% scons -Q define=FOO
cc -o prog.o -c -DFOO prog.c
% scons -Q define=FOO define=BAR
cc -o prog.o -c -DFOO -DBAR prog.c

Note that the ARGLIST and ARGUMENTS variables do not interfere with each other, but merely provide slightly different views into how the user specified variable=value settings on the command line. You can use both variables in the same SCons configuration. In general, the ARGUMENTS dictionary is more convenient to use, (since you can just fetch variable settings through a dictionary access), and the ARGLIST list is more flexible (since you can examine the specific order in which the user's command-line variabe settings).

10.2.1. Controlling Command-Line Build Variables

Being able to use a command-line build variable like debug=1 is handy, but it can be a chore to write specific Python code to recognize each such variable, check for errors and provide appropriate messages, and apply the values to a construction variable. To help with this, SCons supports a class to define such build variables easily, and a mechanism to apply the build variables to a construction environment. This allows you to control how the build variables affect construction environments.

For example, suppose that you want users to set a RELEASE construction variable on the command line whenever the time comes to build a program for release, and that the value of this variable should be added to the command line with the appropriate define to pass the value to the C compiler. Here's how you might do that by setting the appropriate value in a dictionary for the $CPPDEFINES construction variable:

vars = Variables(None, ARGUMENTS)
vars.Add('RELEASE', default=0)
env = Environment(variables=vars, CPPDEFINES={'RELEASE_BUILD': '${RELEASE}'})
env.Program(['foo.c', 'bar.c'])

This SConstruct file first creates a Variables object which uses the values from the command-line options dictionary ARGUMENTS (the vars = Variables(None, ARGUMENTS) call). It then uses the object's Add method to indicate that the RELEASE variable can be set on the command line, and that if not set the default value will be 0.

We then pass the created Variables object as a variables keyword argument to the Environment call used to create the construction environment. This then allows a user to set the RELEASE build variable on the command line and have the variable show up in the command line used to build each object from a C source file:

% scons -Q RELEASE=1
cc -o bar.o -c -DRELEASE_BUILD=1 bar.c
cc -o foo.o -c -DRELEASE_BUILD=1 foo.c
cc -o foo foo.o bar.o

Historical note: In old SCons (prior to 0.98.1), these build variables were known as "command-line build options." The class was named the Options class, and the predefined functions to construct options were named BoolOption, EnumOption, ListOption, PathOption, PackageOption and AddOptions (contrast with the current names in "Pre-Defined Build Variable Functions" below. You may encounter these names in older SConscript files, wiki pages, blog entries, StackExchange articles, etc. These old names no longer work, but a mental substitution of “Variable”" for “Option” will let the concepts transfer to current usage models.

10.2.2. Providing Help for Command-Line Build Variables

To make command-line build variables most useful, you ideally want to provide some help text that will describe the available variables when the user runs scons -h. You could write this text by hand, but SCons provides an easier way. Variables objects support a GenerateHelpText method that will, as its name suggests, generate text that describes the various variables that have been added to it. The default text includes the help string itself plus other information such as allowed values. (The generated text can also be customized by replacing the FormatVariableHelpText method). You then pass the output from this method to the Help function:

vars = Variables(None, ARGUMENTS)
vars.Add('RELEASE', help='Set to 1 to build for release', default=0)
env = Environment(variables=vars)
Help(vars.GenerateHelpText(env))

SCons will now display some useful text when the -h option is used:

% scons -Q -h

RELEASE: Set to 1 to build for release
    default: 0
    actual: 0

Use scons -H for help about command-line options.

Notice that the help output shows the default value, and the current actual value of the build variable.

10.2.3. Reading Build Variables From a File

Giving the user a way to specify the value of a build variable on the command line is useful, but can still be tedious if users must specify the variable every time they run SCons. We can let users provide customized build variable settings in a local file by providing a file name when we create the Variables object:

vars = Variables('custom.py')
vars.Add('RELEASE', help='Set to 1 to build for release', default=0)
env = Environment(variables=vars, CPPDEFINES={'RELEASE_BUILD': '${RELEASE}'})
env.Program(['foo.c', 'bar.c'])
Help(vars.GenerateHelpText(env))

This then allows the user to control the RELEASE variable by setting it in the custom.py file:

RELEASE = 1

Note that this file is actually executed like a Python script. Now when we run SCons:

% scons -Q
cc -o bar.o -c -DRELEASE_BUILD=1 bar.c
cc -o foo.o -c -DRELEASE_BUILD=1 foo.c
cc -o foo foo.o bar.o

And if we change the contents of custom.py to:

RELEASE = 0

The object files are rebuilt appropriately with the new variable:

% scons -Q
cc -o bar.o -c -DRELEASE_BUILD=0 bar.c
cc -o foo.o -c -DRELEASE_BUILD=0 foo.c
cc -o foo foo.o bar.o

Finally, you can combine both methods with:

vars = Variables('custom.py', ARGUMENTS)

where values in the option file custom.py get overwritten by the ones specified on the command line.

10.2.4. Pre-Defined Build Variable Functions

SCons provides a number of functions that provide ready-made behaviors for various types of command-line build variables. These functions all return a tuple which is ready to be passed to an Add call.

10.2.4.1. True/False Values: the `BoolVariable` Build Variable Function

It's often handy to be able to specify a variable that controls a simple Boolean variable with a true or false value. It would be even more handy to accomodate users who have different preferences for how to represent true or false values. The BoolVariable function makes it easy to accomodate these common representations of true or false.

The BoolVariable function takes three arguments: the name of the build variable, the default value of the build variable, and the help string for the variable. It then returns appropriate information for passing to the Add method of a Variables object, like so:

vars = Variables('custom.py')
vars.Add(BoolVariable('RELEASE', help='Set to build for release', default=0))
env = Environment(variables=vars, CPPDEFINES={'RELEASE_BUILD': '${RELEASE}'})
env.Program('foo.c')

With this build variable, the RELEASE variable can now be enabled by setting it to the value yes or t:

% scons -Q RELEASE=yes foo.o
cc -o foo.o -c -DRELEASE_BUILD=True foo.c

% scons -Q RELEASE=t foo.o
cc -o foo.o -c -DRELEASE_BUILD=True foo.c

Other values that equate to true include y, 1, on and all.

Conversely, RELEASE may now be given a false value by setting it to no or f:

% scons -Q RELEASE=no foo.o
cc -o foo.o -c -DRELEASE_BUILD=False foo.c

% scons -Q RELEASE=f foo.o
cc -o foo.o -c -DRELEASE_BUILD=False foo.c

Other values that equate to false include n, 0, off and none.

Lastly, if a user tries to specify any other value, SCons supplies an appropriate error message:

% scons -Q RELEASE=bad_value foo.o

scons: *** Error converting option: RELEASE
Invalid value for boolean option: bad_value
File "/home/my/project/SConstruct", line 3, in <module>

10.2.4.2. Single Value From a Selection: the `EnumVariable` Build Variable Function

Suppose that we want a user to be able to set a COLOR variable that selects a background color to be displayed by an application, but that we want to restrict the choices to a specific set of allowed colors. This can be set up quite easily using the EnumVariable, which takes a list of allowed_values in addition to the variable name, default value, and help text arguments:

vars = Variables('custom.py')
vars.Add(
    EnumVariable(
        'COLOR',
        help='Set background color',
        default='red',
        allowed_values=('red', 'green', 'blue'),
    )
)
env = Environment(variables=vars, CPPDEFINES={'COLOR': '"${COLOR}"'})
env.Program('foo.c')
Help(vars.GenerateHelpText(env))

The user can now explicity set the COLOR build variable to any of the specified allowed values:

% scons -Q COLOR=red foo.o
cc -o foo.o -c -DCOLOR="red" foo.c
% scons -Q COLOR=blue foo.o
cc -o foo.o -c -DCOLOR="blue" foo.c
% scons -Q COLOR=green foo.o
cc -o foo.o -c -DCOLOR="green" foo.c

But, almost more importantly, an attempt to set COLOR to a value that's not in the list generates an error message:

% scons -Q COLOR=magenta foo.o

scons: *** Invalid value for option COLOR: magenta.  Valid values are: ('red', 'green', 'blue')
File "/home/my/project/SConstruct", line 10, in <module>

This example can also serve to further illustrate help generation: the help message here picks up not only the help text, but augments it with information gathered from allowed_values and default:

% scons -Q -h

COLOR: Set background color (red|green|blue)
    default: red
    actual: red

Use scons -H for help about command-line options.

The EnumVariable function also supports a way to map alternate names to allowed values. Suppose, for example, that we want to allow the user to use the word navy as a synonym for blue. We do this by adding a map dictionary that will map its key values to the desired legal value:

vars = Variables('custom.py')
vars.Add(
    EnumVariable(
        'COLOR',
        help='Set background color',
        default='red',
        allowed_values=('red', 'green', 'blue'),
        map={'navy': 'blue'},
    )
)
env = Environment(variables=vars, CPPDEFINES={'COLOR': '"${COLOR}"'})
env.Program('foo.c')

As desired, the user can then use navy on the command line, and SCons will translate it into blue when it comes time to use the COLOR variable to build a target:

% scons -Q COLOR=navy foo.o
cc -o foo.o -c -DCOLOR="blue" foo.c

By default, when using the EnumVariable function, arguments that differ from the legal values only in case are treated as illegal values:

% scons -Q COLOR=Red foo.o

scons: *** Invalid value for option COLOR: Red.  Valid values are: ('red', 'green', 'blue')
File "/home/my/project/SConstruct", line 10, in <module>
% scons -Q COLOR=BLUE foo.o

scons: *** Invalid value for option COLOR: BLUE.  Valid values are: ('red', 'green', 'blue')
File "/home/my/project/SConstruct", line 10, in <module>
% scons -Q COLOR=nAvY foo.o

scons: *** Invalid value for option COLOR: nAvY.  Valid values are: ('red', 'green', 'blue')
File "/home/my/project/SConstruct", line 10, in <module>

The EnumVariable function can take an additional ignorecase keyword argument that, when set to 1, tells SCons to allow case differences when the values are specified:

vars = Variables('custom.py')
vars.Add(
    EnumVariable(
        'COLOR',
        help='Set background color',
        default='red',
        allowed_values=('red', 'green', 'blue'),
        map={'navy': 'blue'},
        ignorecase=1,
    )
)
env = Environment(variables=vars, CPPDEFINES={'COLOR': '"${COLOR}"'})
env.Program('foo.c')

Which yields the output:

% scons -Q COLOR=Red foo.o
cc -o foo.o -c -DCOLOR="Red" foo.c
% scons -Q COLOR=BLUE foo.o
cc -o foo.o -c -DCOLOR="BLUE" foo.c
% scons -Q COLOR=nAvY foo.o
cc -o foo.o -c -DCOLOR="blue" foo.c
% scons -Q COLOR=green foo.o
cc -o foo.o -c -DCOLOR="green" foo.c

Notice that an ignorecase value of 1 preserves the case-spelling that the user supplied. If you want SCons to translate the names into lower-case, regardless of the case used by the user, specify an ignorecase value of 2:

vars = Variables('custom.py')
vars.Add(
    EnumVariable(
        'COLOR',
        help='Set background color',
        default='red',
        allowed_values=('red', 'green', 'blue'),
        map={'navy': 'blue'},
        ignorecase=2,
    )
)
env = Environment(variables=vars, CPPDEFINES={'COLOR': '"${COLOR}"'})
env.Program('foo.c')

Now SCons will use values of red, green or blue regardless of how the user spells those values on the command line:

% scons -Q COLOR=Red foo.o
cc -o foo.o -c -DCOLOR="red" foo.c
% scons -Q COLOR=nAvY foo.o
cc -o foo.o -c -DCOLOR="blue" foo.c
% scons -Q COLOR=GREEN foo.o
cc -o foo.o -c -DCOLOR="green" foo.c

10.2.4.3. Multiple Values From a List: the `ListVariable` Build Variable Function

Another way in which you might want to allow users to control a build variable is to specify a list of one or more legal values. SCons supports this through the ListVariable function. If, for example, we want a user to be able to set a COLORS variable to one or more of the legal list of values:

vars = Variables('custom.py')
vars.Add(
    ListVariable(
        'COLORS', help='List of colors', default=0, names=['red', 'green', 'blue']
    )
)
env = Environment(variables=vars, CPPDEFINES={'COLORS': '"${COLORS}"'})
env.Program('foo.c')

A user can now specify a comma-separated list of legal values, which will get translated into a space-separated list for passing to the any build commands:

% scons -Q COLORS=red,blue foo.o
cc -o foo.o -c -DCOLORS="red -Dblue" foo.c
% scons -Q COLORS=blue,green,red foo.o
cc -o foo.o -c -DCOLORS="blue -Dgreen -Dred" foo.c

In addition, the ListVariable function allows the user to specify explicit keywords of all or none to select all of the legal values, or none of them, respectively:

% scons -Q COLORS=all foo.o
cc -o foo.o -c -DCOLORS="red -Dgreen -Dblue" foo.c
% scons -Q COLORS=none foo.o
cc -o foo.o -c -DCOLORS="" foo.c

And, of course, an illegal value still generates an error message:

% scons -Q COLORS=magenta foo.o

scons: *** Error converting option: COLORS
Invalid value(s) for option: magenta
File "/home/my/project/SConstruct", line 7, in <module>

You can use this last characteristic as a way to enforce at least one of your valid options being chosen by specifying the valid values with the names parameter and then giving a value not in that list as the default parameter - that way if no value is given on the command line, the default is chosen, and it will error out with an invalid value. The example is, in fact, set up that way by using 0 as the default:

% scons -Q foo.o

scons: *** Error converting option: COLORS
Invalid value(s) for option: 0
File "/home/my/project/SConstruct", line 7, in <module>

This technique works for EnumVariable as well.

10.2.4.4. Path Names: the `PathVariable` Build Variable Function

SCons supports a PathVariable function to make it easy to create a build variable to control an expected path name. If, for example, you need to define a variable in the preprocessor that controls the location of a configuration file:

vars = Variables('custom.py')
vars.Add(
    PathVariable(
        'CONFIG', help='Path to configuration file', default='/etc/my_config'
    )
)
env = Environment(variables=vars, CPPDEFINES={'CONFIG_FILE': '"$CONFIG"'})
env.Program('foo.c')

This allows you to override the CONFIG build variable on the command line as necessary:

% scons -Q foo.o
cc -o foo.o -c -DCONFIG_FILE="/etc/my_config" foo.c
% scons -Q CONFIG=/usr/local/etc/other_config foo.o
scons: `foo.o' is up to date.

By default, PathVariable checks to make sure that the specified path exists and generates an error if it doesn't:

% scons -Q CONFIG=/does/not/exist foo.o

scons: *** Path for option CONFIG does not exist: /does/not/exist
File "/home/my/project/SConstruct", line 7, in <module>

PathVariable provides a number of methods that you can use to change this behavior. If you want to ensure that any specified paths are, in fact, files and not directories, use the PathVariable.PathIsFile method as the validation function:

vars = Variables('custom.py')
vars.Add(
    PathVariable(
        'CONFIG',
        help='Path to configuration file',
        default='/etc/my_config',
        validator=PathVariable.PathIsFile,
    )
)
env = Environment(variables=vars, CPPDEFINES={'CONFIG_FILE': '"$CONFIG"'})
env.Program('foo.c')

Conversely, to ensure that any specified paths are directories and not files, use the PathVariable.PathIsDir method as the validation function:

vars = Variables('custom.py')
vars.Add(
    PathVariable(
        'DBDIR',
        help='Path to database directory',
        default='/var/my_dbdir',
        validator=PathVariable.PathIsDir,
    )
)
env = Environment(variables=vars, CPPDEFINES={'DBDIR': '"$DBDIR"'})
env.Program('foo.c')

If you want to make sure that any specified paths are directories, and you would like the directory created if it doesn't already exist, use the PathVariable.PathIsDirCreate method as the validation function:

vars = Variables('custom.py')
vars.Add(
    PathVariable(
        'DBDIR',
        help='Path to database directory',
        default='/var/my_dbdir',
        validator=PathVariable.PathIsDirCreate,
    )
)
env = Environment(variables=vars, CPPDEFINES={'DBDIR': '"$DBDIR"'})
env.Program('foo.c')

Lastly, if you don't care whether the path exists, is a file, or a directory, use the PathVariable.PathAccept method to accept any path that the user supplies:

vars = Variables('custom.py')
vars.Add(
    PathVariable(
        'OUTPUT',
        help='Path to output file or directory',
        default=None,
        validator=PathVariable.PathAccept,
    )
)
env = Environment(variables=vars, CPPDEFINES={'OUTPUT': '"$OUTPUT"'})
env.Program('foo.c')

10.2.4.5. Enabled/Disabled Path Names: the `PackageVariable` Build Variable Function

Sometimes you want to give users even more control over a path name variable, allowing them to explicitly enable or disable the path name by using yes or no keywords, in addition to allow them to supply an explicit path name. SCons supports the PackageVariable function to support this:

vars = Variables("custom.py")
vars.Add(
    PackageVariable("PACKAGE", help="Location package", default="/opt/location")
)
env = Environment(variables=vars, CPPDEFINES={"PACKAGE": '"$PACKAGE"'})
env.Program("foo.c")

When the SConscript file uses the PackageVariable function, the user can still use the default or supply an overriding path name, but can now explicitly set the specified variable to a value that indicates the package should be enabled (in which case the default should be used) or disabled:

% scons -Q foo.o
cc -o foo.o -c -DPACKAGE="/opt/location" foo.c
% scons -Q PACKAGE=/usr/local/location foo.o
cc -o foo.o -c -DPACKAGE="/usr/local/location" foo.c
% scons -Q PACKAGE=yes foo.o
cc -o foo.o -c -DPACKAGE="True" foo.c
% scons -Q PACKAGE=no foo.o
cc -o foo.o -c -DPACKAGE="False" foo.c

10.2.5. Adding Multiple Command-Line Build Variables at Once

Lastly, SCons provides a way to add multiple build variables to a Variables object at once. Instead of having to call the Add method multiple times, you can call the AddVariables method with the build variables to be added to the object. Each build variable is specified as either a tuple of arguments, or as a call to one of the pre-defined functions for pre-packaged command-line build variables, which returns such a tuple. Note that an individual tuple cannot take keyword arguments in the way that a call to Add or one of the build variable functions can. The order of variables given to AddVariables does not matter.

vars = Variables()
vars.AddVariables(
    ('RELEASE', 'Set to 1 to build for release', 0),
    ('CONFIG', 'Configuration file', '/etc/my_config'),
    BoolVariable('warnings',
                 help='compilation with -Wall and similiar',
                 default=1),
    EnumVariable('debug',
                 help='debug output and symbols',
                 default='no',
                 allowed_values=('yes', 'no', 'full'),
                 map={},
                 ignorecase=0),
    ListVariable('shared',
                 help='libraries to build as shared libraries',
                 default='all',
                 names = list_of_libs),
    PackageVariable('x11',
                    help='use X11 installed here (yes = search some places)',
                    default='yes'),
    PathVariable('qtdir',
                 help='where the root of Qt is installed',
                 default=qtdir),
)

10.2.6. Handling Unknown Command-Line Build Variables: the `UnknownVariables` Function

Users may, of course, occasionally misspell variable names in their command-line settings. SCons does not generate an error or warning for any unknown variables the users specifies on the command line. (This is in no small part because you may be processing the arguments directly using the ARGUMENTS dictionary, and therefore SCons can't know in the general case whether a given "misspelled" variable is really unknown and a potential problem, or something that your SConscript file will handle directly with some Python code.)

If, however, you're using a Variables object to define a specific set of command-line build variables that you expect users to be able to set, you may want to provide an error message or warning of your own if the user supplies a variable setting that is not among the defined list of variable names known to the Variables object. You can do this by calling the UnknownVariables method of the Variables object:

vars = Variables(None)
vars.Add('RELEASE', help='Set to 1 to build for release', default=0)
env = Environment(variables=vars, CPPDEFINES={'RELEASE_BUILD': '${RELEASE}'})
unknown = vars.UnknownVariables()
if unknown:
    print("Unknown variables: %s" % " ".join(unknown.keys()))
    Exit(1)
env.Program('foo.c')

The UnknownVariables method returns a dictionary containing the keywords and values of any variables the user specified on the command line that are not among the variables known to the Variables object (from having been specified using the Variables object's Add method). In the example above, we check for whether the dictionary returned by UnknownVariables is non-empty, and if so print the Python list containing the names of the unknown variables and then call the Exit function to terminate SCons:

% scons -Q NOT_KNOWN=foo
Unknown variables: NOT_KNOWN

Of course, you can process the items in the dictionary returned by the UnknownVariables function in any way appropriate to your build configuration, including just printing a warning message but not exiting, logging an error somewhere, etc.

Note that you must delay the call of UnknownVariables until after you have applied the Variables object to a construction environment with the variables= keyword argument of an Environment call: the variables in the object are not fully processed until this has happened.