INTRO

This aims to be a complete, usable, and fairly non-trivial SCons example for C/C++.

I set out with the goal of making a simple to use SCons-based build set up that would automatically build Release and Debug variants into separate directories. Additionally I sometimes like to have my build consolidate all the libs, executables, and/or headers into given directories after building. For instance creating, a source_tree/Lib dir and a source_tree/Include directory for the libs and public header files for utility library projects.

And when you build multiple variations of a lib, some Debug some Release, you often want to give them different names, like 'foo.lib' vs 'foo_dbg.lib' (or 'libfoo.a' vs 'libfoo_dbg.a'). The scripts below handle all that, while providing a few other niceties.

This is still a work in progress, but already it does a lot of things I find useful.

Naturally, you want your SConscript files to be as simple as possible and basically have them 'include' all your standard rules for building. I considered doing that via the Python execfile() function, and storing the default rules in a separate file at the build root, but in the end it seemed like I might as well just Export/Import a build object with all the smarts in methods. Then the global build object can just be defined in the main SConstruct. Originally I thought it would be good to put all the source and target data etc into a big dictionary structure in the SConscript files and pass that to my processing methods, but upon consideration it became clear that what I was really accomplishing with all that was decorating, or 'wrapping', the default Program() and Library() builder methods. The WrapperFunctions Wiki page describes exactly how to do this in a much more straightforward way. Part of the benefit of a wrappers approach is that to the SConscript writer, the syntax looks pretty much like the basic SConscript examples in the SCons User's Guide.

So basically the result is that the SConscript just calls some things that look pretty similar to Program() and Library() calls (specifically, I've named them dProgram() and dLibrary()), but all the extra things happen that I described above. For example

   1 Import('env')
   2 env.dLibrary(target='foo',source='foo.c *.cpp',CPPPATH=['#Include'])

Complete examples of the SConstruct files are included below.

OVERVIEW

So what's good about it:

What needs work:

Other possible negatives:

THE CODE

Here are the files.

An example SConscript for a dir with some libraries:

   1 Import('env')
   2 env.dLibrary(target='viewer',
   3              libsrc = '''
   4                viewer.cpp
   5                #camera/camdisplay.cpp
   6                #math/mat16fv.cpp
   7                #timer/stopwatch.cpp
   8              ''',
   9              CPPPATH = Split('#camera #math #math_objs #timer')
  10              )
  11 env.dHeaders('*.hpp #camera/*.hpp trackball.h')

There's only one lib there, but you can of course make multiple dLibrary calls. Also note that not all the sources for this lib are located in the directory with this SConscript. That's fine. The dHeaders call is used to install the headers listed in the '#Include' directory of the tree. Note that it can take a single string that include a combination of globs, build relative globs, and non-globs. Any non-list argument is automatically Split() and anything that looks like a glob is globbed on (and globbed properly, in the source directory, not in the build directory as is the default SCons behavior)

An example SConscript for a dir with several binaries:

   1 Import('env')
   2  
   3 for appname in Split('''
   4     test_basic
   5     test_basic_oo
   6     test_keymouse
   7     test_menu
   8 '''):
   9     env.dProgram(target=appname, source=[appname + '.cpp'],
  10                  CPPPATH=Split('#viewer #camera #math'),
  11                  LIBPATH=['#lib'],
  12                  LIBS=['viewer'])

The above example uses a loop, but you could certainly also just write each one, one-by-one, if you're afraid of Python and find that more intuitive.

Finally, the top-level Root SConscript just calls subdirs:

   1 Import('env')
   2  
   3 env.dSubdirs('subdir1 subdir2 librarysrc tests')

And now the main SConstruct file:

   1 # The goal here is to be able to build libs and programs using just
   2 # the list of sources.
   3 # I wish to 'wrap' or 'decorate' the base scons behavior in
   4 # a few ways:
   5 # * default variant builds ('Debug'/'Release')
   6 # * automatic build dirs (which are variant-sensitive)
   7 # - with work-arounds to make *everything* go in the build dir
   8 # * automatic no-fuss globs (that *also* work properly for build dirs)
   9 # * default platform-specific build env vars (also variant-sensitive)
  10 # (i.e. some extra compiler specific CCFLAGS and such)
  11 # * automatic 'installation' of libs, program, and headers into specific
  12 # directories (e.g. '#lib', '#bin', and '#include')
  13 import sys
  14  
  15 ############################################
  16 # Site-specific setup
  17  
  18 # Hmm... this should go somewhere else. SCustomize??
  19 # This is my own site-specific win32 setup...
  20 if sys.platform == 'win32':
  21     stdinc = [r'c:\usr\include']
  22     stdlibinc = [r'c:\usr\lib']
  23 else:
  24     stdinc = []
  25     stdlibinc = []
  26  
  27 ############################################
  28 # Generic boiler plate
  29 #-------------------------------------------
  30 import os
  31 import os.path
  32  
  33 opts = Options('SCustomize')
  34 opts.Add('debug', 'Build with debugging symbols', 0)
  35 opts.Add('CC', 'Set C compiler')
  36  
  37 env = Environment(options=opts)
  38 Help(opts.GenerateHelpText(env))
  39  
  40 debug = env.get('debug',0)
  41 build_base = 'build'
  42  
  43 if debug:
  44     env.Append(CPPDEFINES = ['DEBUG', '_DEBUG'])
  45     variant = 'Debug'
  46 else:
  47     env.Append(CPPDEFINES = ['NDEBUG'])
  48     variant = 'Release'
  49  
  50 ############################################
  51 # PLATFORM SPECIFIC CONFIGS
  52 ############################################
  53 #-------------- win32 MSVC -----------------
  54 if env['CC'] == 'cl':
  55     def freeMSVCHack(env, vclibs):
  56         # SCons automatically finds full versions of msvc via the registry, so
  57         # if it can't find 'cl', it may be because we're trying to use the
  58         # free version
  59         def isMicrosoftSDKDir(dir):
  60             return os.path.exists(os.path.join(dir, 'Include', 'Windows.h')) and os.path.exists(os.path.join(dir, 'Lib', 'WinMM.lib'))
  61  
  62         def findMicrosoftSDK():
  63             import SCons.Platform.win32
  64             import SCons.Util
  65             import re
  66             if not SCons.Util.can_read_reg:
  67                 return None
  68             HLM = SCons.Util.HKEY_LOCAL_MACHINE
  69             K = r'Software\Microsoft\.NETFramework\AssemblyFolders\PSDK Assemblies'
  70             try:
  71                 k = SCons.Util.RegOpenKeyEx(HLM, K)
  72                 p = SCons.Util.RegQueryValueEx(k,'')[0]
  73                 # this should have \include at the end, so chop that off
  74                 p = re.sub(r'(?i)\\+Include\\*$','',p)
  75                 if isMicrosoftSDKDir(p): return p
  76             except SCons.Util.RegError:
  77                 pass
  78  
  79             K = r'SOFTWARE\Microsoft\MicrosoftSDK\InstalledSDKs'
  80             try:
  81                 k = SCons.Util.RegOpenKeyEx(HLM, K)
  82                 i=0
  83                 while 1:
  84                     p = SCons.Util.RegEnumKey(k,i)
  85                     i+=1
  86                     subk = SCons.Util.RegOpenKeyEx(k, p)
  87                     try:
  88                         p = SCons.Util.RegQueryValueEx(subk,'Install Dir')[0]
  89                         # trim trailing backslashes
  90                         p = re.sub(r'\\*$','',p)
  91                         if isMicrosoftSDKDir(p): return p
  92                     except SCons.Util.RegError:
  93                         pass
  94             except SCons.Util.RegError:
  95                 pass
  96  
  97             return None
  98  
  99         # End of local defs. Actual freeMSVCHack begins here
 100         if not env['MSVS'].get('VCINSTALLDIR'):
 101             if os.environ.get('VCToolkitInstallDir'):
 102                 vcdir=os.environ['VCToolkitInstallDir']
 103                 env.PrependENVPath('INCLUDE', os.path.join(vcdir, 'Include'))
 104                 env.PrependENVPath('LIB', os.path.join(vcdir, 'Lib'))
 105                 env.PrependENVPath('PATH', os.path.join(vcdir, 'Bin'))
 106                 env['MSVS']['VERSION'] = '7.1'
 107                 env['MSVS']['VERSIONS'] = ['7.1']
 108             if not env['MSVS'].get('PLATFORMSDKDIR'):
 109                 sdkdir = findMicrosoftSDK()
 110                 if sdkdir:
 111                     env.PrependENVPath('INCLUDE', os.path.join(sdkdir, 'Include'))
 112                     env.PrependENVPath('LIB', os.path.join(sdkdir, 'Lib'))
 113                     env.PrependENVPath('PATH', os.path.join(sdkdir, 'Bin'))
 114                     env['MSVS']['PLATFORMSDKDIR']=sdkdir
 115             # FREE MSVC7 only allows
 116             # /ML(libc) /MT(libcmt) or /MLd(libcd)
 117             # Full IDE versions also have
 118             # /MD(msvcrtd) /MTd(libcmtd) and /MDd(msvcrtd)
 119             # So if you want to debug with the freever, the only option is
 120             # the single-threaded lib, /MLd
 121             vclibs['Debug']='/MLd'
 122             vclibs['Release']='/MT'
 123  
 124     # MSVC SETUP
 125     # MDd is for multithreaded debug dll CRT (msvcrtd)
 126     # MD is for multithreaded dll CRT (msvcrt)
 127     # These are just my preferences
 128     vclibs = {'Debug':'/MDd','Release':'/MD'}
 129     freeMSVCHack(env, vclibs)
 130  
 131     env.Append(CCFLAGS=[vclibs[variant]])
 132     if debug:
 133         env.Append(CCFLAGS=Split('/Zi /Fd${TARGET}.pdb'))
 134         env.Append(LINKFLAGS = ['/DEBUG'])
 135         # env.Clean('.', '${TARGET}.pdb')
 136         # Need to clean .pdbs somehow! The above line doesn't work!
 137     else:
 138         env.Append(CCFLAGS=Split('/Og /Ot /Ob1 /Op /G6'))
 139  
 140     env.Append(CCFLAGS=Split('/EHsc /J /W3 /Gd'))
 141     env.Append(CPPDEFINES=Split('WIN32 _WINDOWS'))
 142  
 143 #-------------- gcc-like (default) ---------
 144 else: # generic posix-like
 145     if debug:
 146         env.Append(CPPFLAGS = ['-g'])
 147     else:
 148         env.Append(CPPFLAGS = ['-O3'])
 149 #-------------------------------------------
 150  
 151  
 152 # Put all the little .sconsign files into one big file.
 153 # (Does this slow down parallel builds?)
 154 # Need to create the build dir before we put the signatures db in there
 155 fullbuildpath = Dir(build_base).abspath
 156 if not os.path.exists(fullbuildpath): os.makedirs(fullbuildpath)
 157 import dbhash
 158 env.SConsignFile(os.path.join(build_base, 'sconsignatures'), dbhash)
 159  
 160  
 161 # Make a singleton global object for keeping track of all the extra data
 162 # and methods that are being added
 163 class Globals:
 164     def __init__(self):
 165         self.env = env
 166         self.stdinc = stdinc
 167         self.stdlibinc = stdlibinc
 168         self.variant = variant
 169         self.build_base = os.path.join(build_base, variant)
 170         self.libname_decorators = { 'Debug' : '_dbg' }
 171         self.appname_decorators = { 'Debug' : '_dbg' }
 172         self.incinstdir = '#include'
 173         self.libinstdir = '#lib'
 174         self.appinstdir = '#bin'
 175         self.objcache = {}
 176  
 177     def Glob(self, pat):
 178         ## GLOB IN THE REAL SOURCE DIRECTORY (NOT BUILD DIR)
 179         import glob
 180         prevdir = os.getcwd();
 181         if pat[0] != '#':
 182             os.chdir(self.env.Dir('.').srcnode().abspath)
 183             ret = glob.glob(pat)
 184         else:
 185             pat = pat[1:]
 186             base = os.path.dirname(pat)
 187             searchdir = self.env.Dir('#').srcnode().abspath
 188             os.chdir(searchdir)
 189             ret = ['#'+x for x in glob.glob(pat)]
 190         os.chdir(prevdir)
 191         return ret
 192  
 193     def GlobExpand(self, list):
 194         ## look for pattern-like things and glob on those
 195         ret = []
 196         for item in list:
 197             if item.find('*') or item.find('?') or item.find('['):
 198                 ret += self.Glob(item)
 199             else:
 200                 ret += [item]
 201         return ret
 202  
 203     def IsALocalLib(self, lib):
 204         # This is rather heuristic determining if a lib is local or not
 205         return lib.find('/') or lib.find(os.sep) or lib[0]=='#'
 206  
 207     def MyHeaderMethod(self, env, source, **dict):
 208         if type(source)==type(''): source = Split(source)
 209         source = self.GlobExpand(source)
 210         nodes = []
 211         if hasattr(self,'incinstdir') and self.incinstdir:
 212             for i in source:
 213                 nodes.append( env.Install(self.incinstdir, i) )
 214         return nodes
 215  
 216     def MyLibraryMethod(self, env, **dict):
 217         relincs = dict.get('CPPPATH',[])
 218         dict['CPPPATH'] = ['.'] + relincs + self.stdinc
 219  
 220         # These shenanigans are necessary to get SCons to build non-local
 221         # sources in the BuildDir instead of their own local directories
 222         target = dict.pop('target')
 223         source = dict.pop('source')
 224         if type(source)==type(''): source = Split(source)
 225         allsrc = []
 226         for x in source:
 227             objbase = os.path.basename(x)
 228             if self.objcache.get(objbase):
 229                 #print 'Reusing node', objbase
 230                 #NOTE: We should check that defines etc are all the same!!
 231                 allsrc += self.objcache[objbase]
 232             else:
 233                 onode = env.SharedObject(
 234                     os.path.splitext(objbase)[0], x,
 235                     **dict)
 236                 allsrc += onode
 237                 self.objcache[objbase]=onode
 238  
 239         targpath = '#' + os.path.join(self.build_dir, target)
 240         # decorate libname with e.g. '_dbg'
 241         if hasattr(self,'libname_decorators'):
 242             targpath += self.libname_decorators.get(self.variant,'')
 243  
 244         dict['source'] = allsrc
 245         dict['target'] = targpath
 246         node = env.Library(**dict)
 247         if hasattr(self,'libinstdir') and self.libinstdir:
 248             env.Install(self.libinstdir, node)
 249         return node
 250  
 251     def MyProgramMethod(self, env, **dict):
 252         # Enhance CPPPATH,LIBPATH
 253         dict['CPPPATH'] = ['.'] + dict.get('CPPPATH',[]) + self.stdinc
 254         dict['LIBPATH'] = dict.get('LIBPATH',[]) + self.stdlibinc
 255  
 256         # These shenanigans are necessary to get SCons to build non-local
 257         # sources in the BuildDir instead of their own local directories
 258         target = dict.pop('target')
 259         source = dict.pop('source',[])
 260         if type(source)==type(''): source = Split(source)
 261         allsrc = []
 262         for x in source:
 263             objbase = os.path.basename(x)
 264             if self.objcache.get(objbase):
 265                 print 'Reusing node', objbase
 266                 allsrc += self.objcache[objbase]
 267             else:
 268                 onode = self.env.SharedObject(
 269                     os.path.splitext(objbase)[0], x,
 270                     **dict)
 271                 allsrc += onode
 272                 self.objcache[objbase]=onode
 273  
 274         targpath = '#' + os.path.join(self.build_dir, target)
 275         # decorate app name with e.g. '_dbg'
 276         if hasattr(self,'appname_decorators'):
 277             deco = self.appname_decorators.get(self.variant,'')
 278             targpath += deco
 279         # decorate local lib names with e.g. '_dbg'
 280         if hasattr(self,'libname_decorators'):
 281             deco = self.libname_decorators.get(self.variant,'')
 282             # decorate source lib names with e.g. '_dbg'
 283             LIBS = []
 284             for l in dict.pop('LIBS',[]):
 285                 if self.IsALocalLib(l):
 286                     #print "Decorating lib", l, '->', l+deco
 287                     LIBS += [l+deco]
 288                 else:
 289                     LIBS += [l]
 290  
 291         dict['target']=targpath
 292         dict['source']=allsrc
 293         dict['LIBS']=LIBS
 294         node = env.Program(**dict)
 295  
 296         if hasattr(self,'appinstdir') and self.appinstdir:
 297             self.env.Install(self.appinstdir, node)
 298  
 299         return node
 300  
 301     def MySubdirsMethod(self, env, subdirs, **dict):
 302         # Build sub-directories
 303         if type(subdirs)==type(''): subdirs=Split(subdirs)
 304         for d in subdirs:
 305             savedir = self.build_dir
 306             self.build_dir = os.path.join(self.build_dir, d)
 307             env.SConscript(os.path.join(d, 'SConscript'),
 308                            exports=['G','env'])
 309             self.build_dir = savedir
 310  
 311  
 312 G = Globals()
 313  
 314 # Wrap the methods of G into method objects, and then add them
 315 # as methods to Environment.
 316 # See http://www.scons.org/cgi-bin/wiki/WrapperFunctions
 317 def MyLibraryMethod(env, **dict):
 318     G.MyLibraryMethod(env, **dict)
 319 def MyHeaderMethod(env, source, **dict):
 320     G.MyHeaderMethod(env, source, **dict)
 321 def MyProgramMethod(env, **dict):
 322     G.MyProgramMethod(env, **dict)
 323 def MySubdirsMethod(env, subdirs, **dict):
 324     G.MySubdirsMethod(env, subdirs, **dict)
 325 from SCons.Script.SConscript import SConsEnvironment # just do this once
 326 SConsEnvironment.dLibrary = MyLibraryMethod
 327 SConsEnvironment.dProgram = MyProgramMethod
 328 SConsEnvironment.dHeaders = MyHeaderMethod
 329 SConsEnvironment.dSubdirs = MySubdirsMethod
 330  
 331 G.build_dir = G.build_base
 332  
 333 # Call the SConscript in the top-level directory
 334 env.SConscript('SConscript',build_dir=G.build_dir,exports=['env','G'])

Note that a big chunk of that code has to do with trying to find out if the free VC++ toolkit is installed. SCons doesn't currently (ver 0.96.90) detect this even if it is the only compiler available. I just happened to be using the free toolkit while I waited for my actual copy of Visual Studio to arrive, when I started playing with SCons. Consequently I spent a lot of my first day or so with SCons struggling to figure out why on earth the simplest compile with VC required so much code to get the environment right. Well since I struggled through it, I'm including the results of that struggle here in the form of the freeMSVCHack() function above. Hopefully something similar will be built into future releases of SCons.

That's it for now. This is still a work in progress as of 2/18/05, JST. I'll keep updating this page as I make progress. Feedback from the experts is much appreciated.

One last thing, why did I name all my functions like dProgram or dLibrary? What's with the d?

The answer is just I wanted to use names similar to the existing builders I was wrapping, and d seemed like a decent prefix. Maybe it stands for different or decorated or decent. Or maybe dorky :-)

Bill Baxter

ACKNOWLEDGEMENTS

Thanks to Steven Knight, Dobes Vandermeer, Gary Oberbrunner and the other folks on the SCons users mailing list for helpful suggestions, pointers, and advice. And of course a big thanks to Steven Knight and the other developers of SCons.

AllInSConstruct (last edited 2010-09-12 06:19:48 by adsl-71-141-96-112)