SCons.Action

1 """SCons.Action 2 3 This encapsulates information about executing any sort of action that 4 can build one or more target Nodes (typically files) from one or more 5 source Nodes (also typically files) given a specific Environment. 6 7 The base class here is ActionBase. The base class supplies just a few 8 OO utility methods and some generic methods for displaying information 9 about an Action in response to the various commands that control printing. 10 11 A second-level base class is _ActionAction. This extends ActionBase 12 by providing the methods that can be used to show and perform an 13 action. True Action objects will subclass _ActionAction; Action 14 factory class objects will subclass ActionBase. 15 16 The heavy lifting is handled by subclasses for the different types of 17 actions we might execute: 18 19 CommandAction 20 CommandGeneratorAction 21 FunctionAction 22 ListAction 23 24 The subclasses supply the following public interface methods used by 25 other modules: 26 27 __call__() 28 THE public interface, "calling" an Action object executes the 29 command or Python function. This also takes care of printing 30 a pre-substitution command for debugging purposes. 31 32 get_contents() 33 Fetches the "contents" of an Action for signature calculation 34 plus the varlist. This is what gets MD5 checksummed to decide 35 if a target needs to be rebuilt because its action changed. 36 37 genstring() 38 Returns a string representation of the Action *without* 39 command substitution, but allows a CommandGeneratorAction to 40 generate the right action based on the specified target, 41 source and env. This is used by the Signature subsystem 42 (through the Executor) to obtain an (imprecise) representation 43 of the Action operation for informative purposes. 44 45 46 Subclasses also supply the following methods for internal use within 47 this module: 48 49 __str__() 50 Returns a string approximation of the Action; no variable 51 substitution is performed. 52 53 execute() 54 The internal method that really, truly, actually handles the 55 execution of a command or Python function. This is used so 56 that the __call__() methods can take care of displaying any 57 pre-substitution representations, and *then* execute an action 58 without worrying about the specific Actions involved. 59 60 get_presig() 61 Fetches the "contents" of a subclass for signature calculation. 62 The varlist is added to this to produce the Action's contents. 63 TODO(?): Change this to always return ascii/bytes and not unicode (or py3 strings) 64 65 strfunction() 66 Returns a substituted string representation of the Action. 67 This is used by the _ActionAction.show() command to display the 68 command/function that will be executed to generate the target(s). 69 70 There is a related independent ActionCaller class that looks like a 71 regular Action, and which serves as a wrapper for arbitrary functions 72 that we want to let the user specify the arguments to now, but actually 73 execute later (when an out-of-date check determines that it's needed to 74 be executed, for example). Objects of this class are returned by an 75 ActionFactory class that provides a __call__() method as a convenient 76 way for wrapping up the functions. 77 78 """ 79 80 # Copyright (c) 2001 - 2019 The SCons Foundation 81 # 82 # Permission is hereby granted, free of charge, to any person obtaining 83 # a copy of this software and associated documentation files (the 84 # "Software"), to deal in the Software without restriction, including 85 # without limitation the rights to use, copy, modify, merge, publish, 86 # distribute, sublicense, and/or sell copies of the Software, and to 87 # permit persons to whom the Software is furnished to do so, subject to 88 # the following conditions: 89 # 90 # The above copyright notice and this permission notice shall be included 91 # in all copies or substantial portions of the Software. 92 # 93 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY 94 # KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE 95 # WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 96 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 97 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 98 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 99 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 100 101 __revision__ = "src/engine/SCons/Action.py e724ae812eb96f4858a132f5b8c769724744faf6 2019-07-21 00:04:47 bdeegan" 102 103 import os 104 import pickle 105 import re 106 import sys 107 import subprocess 108 import itertools 109 import inspect 110 from collections import OrderedDict 111 112 import SCons.Debug 113 from SCons.Debug import logInstanceCreation 114 import SCons.Errors 115 import SCons.Util 116 import SCons.Subst 117 118 # we use these a lot, so try to optimize them 119 from SCons.Util import is_String, is_List 120

121 -class _null(object):

122 pass

123 124 print_actions = 1 125 execute_actions = 1 126 print_actions_presub = 0 127 128 # Use pickle protocol 1 when pickling functions for signature 129 # otherwise python3 and python2 will yield different pickles 130 # for the same object. 131 # This is due to default being 1 for python 2.7, and 3 for 3.x 132 # TODO: We can roll this forward to 2 (if it has value), but not 133 # before a deprecation cycle as the sconsigns will change 134 ACTION_SIGNATURE_PICKLE_PROTOCOL = 1 135 136

137 -def rfile(n):

138 try: 139 return n.rfile() 140 except AttributeError: 141 return n

142 143

144 -def default_exitstatfunc(s):

145 return s

146 147 strip_quotes = re.compile('^[\'"](.*)[\'"]$') 148 149

150 -def _callable_contents(obj):

151 """Return the signature contents of a callable Python object. 152 """ 153 try: 154 # Test if obj is a method. 155 return _function_contents(obj.__func__) 156 157 except AttributeError: 158 try: 159 # Test if obj is a callable object. 160 return _function_contents(obj.__call__.__func__) 161 162 except AttributeError: 163 try: 164 # Test if obj is a code object. 165 return _code_contents(obj) 166 167 except AttributeError: 168 # Test if obj is a function object. 169 return _function_contents(obj)

170 171

172 -def _object_contents(obj):

173 """Return the signature contents of any Python object. 174 175 We have to handle the case where object contains a code object 176 since it can be pickled directly. 177 """ 178 try: 179 # Test if obj is a method. 180 return _function_contents(obj.__func__) 181 182 except AttributeError: 183 try: 184 # Test if obj is a callable object. 185 return _function_contents(obj.__call__.__func__) 186 187 except AttributeError: 188 try: 189 # Test if obj is a code object. 190 return _code_contents(obj) 191 192 except AttributeError: 193 try: 194 # Test if obj is a function object. 195 return _function_contents(obj) 196 197 except AttributeError as ae: 198 # Should be a pickle-able Python object. 199 try: 200 return _object_instance_content(obj) 201 # pickling an Action instance or object doesn't yield a stable 202 # content as instance property may be dumped in different orders 203 # return pickle.dumps(obj, ACTION_SIGNATURE_PICKLE_PROTOCOL) 204 except (pickle.PicklingError, TypeError, AttributeError) as ex: 205 # This is weird, but it seems that nested classes 206 # are unpickable. The Python docs say it should 207 # always be a PicklingError, but some Python 208 # versions seem to return TypeError. Just do 209 # the best we can. 210 return bytearray(repr(obj), 'utf-8')

211 212

213 -def _code_contents(code, docstring=None):

214 r"""Return the signature contents of a code object. 215 216 By providing direct access to the code object of the 217 function, Python makes this extremely easy. Hooray! 218 219 Unfortunately, older versions of Python include line 220 number indications in the compiled byte code. Boo! 221 So we remove the line number byte codes to prevent 222 recompilations from moving a Python function. 223 224 See: 225 - https://docs.python.org/2/library/inspect.html 226 - http://python-reference.readthedocs.io/en/latest/docs/code/index.html 227 228 For info on what each co\_ variable provides 229 230 The signature is as follows (should be byte/chars): 231 co_argcount, len(co_varnames), len(co_cellvars), len(co_freevars), 232 ( comma separated signature for each object in co_consts ), 233 ( comma separated signature for each object in co_names ), 234 ( The bytecode with line number bytecodes removed from co_code ) 235 236 co_argcount - Returns the number of positional arguments (including arguments with default values). 237 co_varnames - Returns a tuple containing the names of the local variables (starting with the argument names). 238 co_cellvars - Returns a tuple containing the names of local variables that are referenced by nested functions. 239 co_freevars - Returns a tuple containing the names of free variables. (?) 240 co_consts - Returns a tuple containing the literals used by the bytecode. 241 co_names - Returns a tuple containing the names used by the bytecode. 242 co_code - Returns a string representing the sequence of bytecode instructions. 243 244 """ 245 246 # contents = [] 247 248 # The code contents depends on the number of local variables 249 # but not their actual names. 250 contents = bytearray("{}, {}".format(code.co_argcount, len(code.co_varnames)), 'utf-8') 251 252 contents.extend(b", ") 253 contents.extend(bytearray(str(len(code.co_cellvars)), 'utf-8')) 254 contents.extend(b", ") 255 contents.extend(bytearray(str(len(code.co_freevars)), 'utf-8')) 256 257 # The code contents depends on any constants accessed by the 258 # function. Note that we have to call _object_contents on each 259 # constants because the code object of nested functions can 260 # show-up among the constants. 261 z = [_object_contents(cc) for cc in code.co_consts if cc != docstring] 262 contents.extend(b',(') 263 contents.extend(bytearray(',', 'utf-8').join(z)) 264 contents.extend(b')') 265 266 # The code contents depends on the variable names used to 267 # accessed global variable, as changing the variable name changes 268 # the variable actually accessed and therefore changes the 269 # function result. 270 z= [bytearray(_object_contents(cc)) for cc in code.co_names] 271 contents.extend(b',(') 272 contents.extend(bytearray(',','utf-8').join(z)) 273 contents.extend(b')') 274 275 # The code contents depends on its actual code!!! 276 contents.extend(b',(') 277 contents.extend(code.co_code) 278 contents.extend(b')') 279 280 return contents

281 282

283 -def _function_contents(func):

284 """ 285 The signature is as follows (should be byte/chars): 286 < _code_contents (see above) from func.__code__ > 287 ,( comma separated _object_contents for function argument defaults) 288 ,( comma separated _object_contents for any closure contents ) 289 290 291 See also: https://docs.python.org/3/reference/datamodel.html 292 - func.__code__ - The code object representing the compiled function body. 293 - func.__defaults__ - A tuple containing default argument values for those arguments that have defaults, or None if no arguments have a default value 294 - func.__closure__ - None or a tuple of cells that contain bindings for the function's free variables. 295 296 :Returns: 297 Signature contents of a function. (in bytes) 298 """ 299 300 contents = [_code_contents(func.__code__, func.__doc__)] 301 302 # The function contents depends on the value of defaults arguments 303 if func.__defaults__: 304 305 function_defaults_contents = [_object_contents(cc) for cc in func.__defaults__] 306 307 defaults = bytearray(b',(') 308 defaults.extend(bytearray(b',').join(function_defaults_contents)) 309 defaults.extend(b')') 310 311 contents.append(defaults) 312 else: 313 contents.append(b',()') 314 315 # The function contents depends on the closure captured cell values. 316 closure = func.__closure__ or [] 317 318 try: 319 closure_contents = [_object_contents(x.cell_contents) for x in closure] 320 except AttributeError: 321 closure_contents = [] 322 323 contents.append(b',(') 324 contents.append(bytearray(b',').join(closure_contents)) 325 contents.append(b')') 326 327 retval = bytearray(b'').join(contents) 328 return retval

329 330

331 -def _object_instance_content(obj):

332 """ 333 Returns consistant content for a action class or an instance thereof 334 335 :Parameters: 336 - `obj` Should be either and action class or an instance thereof 337 338 :Returns: 339 bytearray or bytes representing the obj suitable for generating a signature from. 340 """ 341 retval = bytearray() 342 343 if obj is None: 344 return b'N.' 345 346 if isinstance(obj, SCons.Util.BaseStringTypes): 347 return SCons.Util.to_bytes(obj) 348 349 inst_class = obj.__class__ 350 inst_class_name = bytearray(obj.__class__.__name__,'utf-8') 351 inst_class_module = bytearray(obj.__class__.__module__,'utf-8') 352 inst_class_hierarchy = bytearray(repr(inspect.getclasstree([obj.__class__,])),'utf-8') 353 # print("ICH:%s : %s"%(inst_class_hierarchy, repr(obj))) 354 355 properties = [(p, getattr(obj, p, "None")) for p in dir(obj) if not (p[:2] == '__' or inspect.ismethod(getattr(obj, p)) or inspect.isbuiltin(getattr(obj,p))) ] 356 properties.sort() 357 properties_str = ','.join(["%s=%s"%(p[0],p[1]) for p in properties]) 358 properties_bytes = bytearray(properties_str,'utf-8') 359 360 methods = [p for p in dir(obj) if inspect.ismethod(getattr(obj, p))] 361 methods.sort() 362 363 method_contents = [] 364 for m in methods: 365 # print("Method:%s"%m) 366 v = _function_contents(getattr(obj, m)) 367 # print("[%s->]V:%s [%s]"%(m,v,type(v))) 368 method_contents.append(v) 369 370 retval = bytearray(b'{') 371 retval.extend(inst_class_name) 372 retval.extend(b":") 373 retval.extend(inst_class_module) 374 retval.extend(b'}[[') 375 retval.extend(inst_class_hierarchy) 376 retval.extend(b']]{{') 377 retval.extend(bytearray(b",").join(method_contents)) 378 retval.extend(b"}}{{{") 379 retval.extend(properties_bytes) 380 retval.extend(b'}}}') 381 return retval

382 383 # print("class :%s"%inst_class) 384 # print("class_name :%s"%inst_class_name) 385 # print("class_module :%s"%inst_class_module) 386 # print("Class hier :\n%s"%pp.pformat(inst_class_hierarchy)) 387 # print("Inst Properties:\n%s"%pp.pformat(properties)) 388 # print("Inst Methods :\n%s"%pp.pformat(methods)) 389

390 -def _actionAppend(act1, act2):

391 # This function knows how to slap two actions together. 392 # Mainly, it handles ListActions by concatenating into 393 # a single ListAction. 394 a1 = Action(act1) 395 a2 = Action(act2) 396 if a1 is None: 397 return a2 398 if a2 is None: 399 return a1 400 if isinstance(a1, ListAction): 401 if isinstance(a2, ListAction): 402 return ListAction(a1.list + a2.list) 403 else: 404 return ListAction(a1.list + [ a2 ]) 405 else: 406 if isinstance(a2, ListAction): 407 return ListAction([ a1 ] + a2.list) 408 else: 409 return ListAction([ a1, a2 ])

410 411

412 -def _do_create_keywords(args, kw):

413 """This converts any arguments after the action argument into 414 their equivalent keywords and adds them to the kw argument. 415 """ 416 v = kw.get('varlist', ()) 417 # prevent varlist="FOO" from being interpreted as ['F', 'O', 'O'] 418 if is_String(v): v = (v,) 419 kw['varlist'] = tuple(v) 420 if args: 421 # turn positional args into equivalent keywords 422 cmdstrfunc = args[0] 423 if cmdstrfunc is None or is_String(cmdstrfunc): 424 kw['cmdstr'] = cmdstrfunc 425 elif callable(cmdstrfunc): 426 kw['strfunction'] = cmdstrfunc 427 else: 428 raise SCons.Errors.UserError( 429 'Invalid command display variable type. ' 430 'You must either pass a string or a callback which ' 431 'accepts (target, source, env) as parameters.') 432 if len(args) > 1: 433 kw['varlist'] = tuple(SCons.Util.flatten(args[1:])) + kw['varlist'] 434 if kw.get('strfunction', _null) is not _null \ 435 and kw.get('cmdstr', _null) is not _null: 436 raise SCons.Errors.UserError( 437 'Cannot have both strfunction and cmdstr args to Action()')

438 439

440 -def _do_create_action(act, kw):

441 """This is the actual "implementation" for the 442 Action factory method, below. This handles the 443 fact that passing lists to Action() itself has 444 different semantics than passing lists as elements 445 of lists. 446 447 The former will create a ListAction, the latter 448 will create a CommandAction by converting the inner 449 list elements to strings.""" 450 451 if isinstance(act, ActionBase): 452 return act 453 454 if is_String(act): 455 var=SCons.Util.get_environment_var(act) 456 if var: 457 # This looks like a string that is purely an Environment 458 # variable reference, like "$FOO" or "${FOO}". We do 459 # something special here...we lazily evaluate the contents 460 # of that Environment variable, so a user could put something 461 # like a function or a CommandGenerator in that variable 462 # instead of a string. 463 return LazyAction(var, kw) 464 commands = str(act).split('\n') 465 if len(commands) == 1: 466 return CommandAction(commands[0], **kw) 467 # The list of string commands may include a LazyAction, so we 468 # reprocess them via _do_create_list_action. 469 return _do_create_list_action(commands, kw) 470 471 if is_List(act): 472 return CommandAction(act, **kw) 473 474 if callable(act): 475 try: 476 gen = kw['generator'] 477 del kw['generator'] 478 except KeyError: 479 gen = 0 480 if gen: 481 action_type = CommandGeneratorAction 482 else: 483 action_type = FunctionAction 484 return action_type(act, kw) 485 486 # Catch a common error case with a nice message: 487 if isinstance(act, int) or isinstance(act, float): 488 raise TypeError("Don't know how to create an Action from a number (%s)"%act) 489 # Else fail silently (???) 490 return None

491 492

493 -def _do_create_list_action(act, kw):

494 """A factory for list actions. Convert the input list into Actions 495 and then wrap them in a ListAction.""" 496 acts = [] 497 for a in act: 498 aa = _do_create_action(a, kw) 499 if aa is not None: acts.append(aa) 500 if not acts: 501 return ListAction([]) 502 elif len(acts) == 1: 503 return acts[0] 504 else: 505 return ListAction(acts)

506 507

508 -def Action(act, *args, **kw):

509 """A factory for action objects.""" 510 # Really simple: the _do_create_* routines do the heavy lifting. 511 _do_create_keywords(args, kw) 512 if is_List(act): 513 return _do_create_list_action(act, kw) 514 return _do_create_action(act, kw)

515 516

517 -class ActionBase(object):

518 """Base class for all types of action objects that can be held by 519 other objects (Builders, Executors, etc.) This provides the 520 common methods for manipulating and combining those actions.""" 521

522 - def __eq__(self, other):

523 return self.__dict__ == other

524

525 - def no_batch_key(self, env, target, source):

526 return None

527 528 batch_key = no_batch_key 529

530 - def genstring(self, target, source, env):

531 return str(self)

532

533 - def get_contents(self, target, source, env):

534 result = self.get_presig(target, source, env) 535 536 if not isinstance(result,(bytes, bytearray)): 537 result = bytearray(result, 'utf-8') 538 else: 539 # Make a copy and put in bytearray, without this the contents returned by get_presig 540 # can be changed by the logic below, appending with each call and causing very 541 # hard to track down issues... 542 result = bytearray(result) 543 544 # At this point everything should be a bytearray 545 546 # This should never happen, as the Action() factory should wrap 547 # the varlist, but just in case an action is created directly, 548 # we duplicate this check here. 549 vl = self.get_varlist(target, source, env) 550 if is_String(vl): vl = (vl,) 551 for v in vl: 552 # do the subst this way to ignore $(...$) parts: 553 if isinstance(result, bytearray): 554 result.extend(SCons.Util.to_bytes(env.subst_target_source('${'+v+'}', SCons.Subst.SUBST_SIG, target, source))) 555 else: 556 raise Exception("WE SHOULD NEVER GET HERE result should be bytearray not:%s"%type(result)) 557 # result.append(SCons.Util.to_bytes(env.subst_target_source('${'+v+'}', SCons.Subst.SUBST_SIG, target, source))) 558 559 560 if isinstance(result, (bytes,bytearray)): 561 return result 562 else: 563 raise Exception("WE SHOULD NEVER GET HERE - #2 result should be bytearray not:%s" % type(result))

564 # return b''.join(result) 565

566 - def __add__(self, other):

567 return _actionAppend(self, other)

568

569 - def __radd__(self, other):

570 return _actionAppend(other, self)

571

572 - def presub_lines(self, env):

573 # CommandGeneratorAction needs a real environment 574 # in order to return the proper string here, since 575 # it may call LazyAction, which looks up a key 576 # in that env. So we temporarily remember the env here, 577 # and CommandGeneratorAction will use this env 578 # when it calls its _generate method. 579 self.presub_env = env 580 lines = str(self).split('\n') 581 self.presub_env = None # don't need this any more 582 return lines

583

584 - def get_varlist(self, target, source, env, executor=None):

585 return self.varlist

586

587 - def get_targets(self, env, executor):

588 """ 589 Returns the type of targets ($TARGETS, $CHANGED_TARGETS) used 590 by this action. 591 """ 592 return self.targets

593 594

595 -class _ActionAction(ActionBase):

596 """Base class for actions that create output objects."""

597 - def __init__(self, cmdstr=_null, strfunction=_null, varlist=(), 598 presub=_null, chdir=None, exitstatfunc=None, 599 batch_key=None, targets='$TARGETS', 600 **kw):

601 self.cmdstr = cmdstr 602 if strfunction is not _null: 603 if strfunction is None: 604 self.cmdstr = None 605 else: 606 self.strfunction = strfunction 607 self.varlist = varlist 608 self.presub = presub 609 self.chdir = chdir 610 if not exitstatfunc: 611 exitstatfunc = default_exitstatfunc 612 self.exitstatfunc = exitstatfunc 613 614 self.targets = targets 615 616 if batch_key: 617 if not callable(batch_key): 618 # They have set batch_key, but not to their own 619 # callable. The default behavior here will batch 620 # *all* targets+sources using this action, separated 621 # for each construction environment. 622 def default_batch_key(self, env, target, source): 623 return (id(self), id(env))

624 batch_key = default_batch_key 625 SCons.Util.AddMethod(self, batch_key, 'batch_key')

626

627 - def print_cmd_line(self, s, target, source, env):

628 """ 629 In python 3, and in some of our tests, sys.stdout is 630 a String io object, and it takes unicode strings only 631 In other cases it's a regular Python 2.x file object 632 which takes strings (bytes), and if you pass those a 633 unicode object they try to decode with 'ascii' codec 634 which fails if the cmd line has any hi-bit-set chars. 635 This code assumes s is a regular string, but should 636 work if it's unicode too. 637 """ 638 try: 639 sys.stdout.write(s + u"\n") 640 except UnicodeDecodeError: 641 sys.stdout.write(s + "\n")

642

643 - def __call__(self, target, source, env, 644 exitstatfunc=_null, 645 presub=_null, 646 show=_null, 647 execute=_null, 648 chdir=_null, 649 executor=None):

650 if not is_List(target): 651 target = [target] 652 if not is_List(source): 653 source = [source] 654 655 if presub is _null: 656 presub = self.presub 657 if presub is _null: 658 presub = print_actions_presub 659 if exitstatfunc is _null: exitstatfunc = self.exitstatfunc 660 if show is _null: show = print_actions 661 if execute is _null: execute = execute_actions 662 if chdir is _null: chdir = self.chdir 663 save_cwd = None 664 if chdir: 665 save_cwd = os.getcwd() 666 try: 667 chdir = str(chdir.get_abspath()) 668 except AttributeError: 669 if not is_String(chdir): 670 if executor: 671 chdir = str(executor.batches[0].targets[0].dir) 672 else: 673 chdir = str(target[0].dir) 674 if presub: 675 if executor: 676 target = executor.get_all_targets() 677 source = executor.get_all_sources() 678 t = ' and '.join(map(str, target)) 679 l = '\n '.join(self.presub_lines(env)) 680 out = u"Building %s with action:\n %s\n" % (t, l) 681 sys.stdout.write(out) 682 cmd = None 683 if show and self.strfunction: 684 if executor: 685 target = executor.get_all_targets() 686 source = executor.get_all_sources() 687 try: 688 cmd = self.strfunction(target, source, env, executor) 689 except TypeError: 690 cmd = self.strfunction(target, source, env) 691 if cmd: 692 if chdir: 693 cmd = ('os.chdir(%s)\n' % repr(chdir)) + cmd 694 try: 695 get = env.get 696 except AttributeError: 697 print_func = self.print_cmd_line 698 else: 699 print_func = get('PRINT_CMD_LINE_FUNC') 700 if not print_func: 701 print_func = self.print_cmd_line 702 print_func(cmd, target, source, env) 703 stat = 0 704 if execute: 705 if chdir: 706 os.chdir(chdir) 707 try: 708 stat = self.execute(target, source, env, executor=executor) 709 if isinstance(stat, SCons.Errors.BuildError): 710 s = exitstatfunc(stat.status) 711 if s: 712 stat.status = s 713 else: 714 stat = s 715 else: 716 stat = exitstatfunc(stat) 717 finally: 718 if save_cwd: 719 os.chdir(save_cwd) 720 if cmd and save_cwd: 721 print_func('os.chdir(%s)' % repr(save_cwd), target, source, env) 722 723 return stat

724 725

726 -def _string_from_cmd_list(cmd_list):

727 """Takes a list of command line arguments and returns a pretty 728 representation for printing.""" 729 cl = [] 730 for arg in map(str, cmd_list): 731 if ' ' in arg or '\t' in arg: 732 arg = '"' + arg + '"' 733 cl.append(arg) 734 return ' '.join(cl)

735 736 default_ENV = None 737 738

739 -def get_default_ENV(env):

740 """ 741 A fiddlin' little function that has an 'import SCons.Environment' which 742 can't be moved to the top level without creating an import loop. Since 743 this import creates a local variable named 'SCons', it blocks access to 744 the global variable, so we move it here to prevent complaints about local 745 variables being used uninitialized. 746 """ 747 global default_ENV 748 try: 749 return env['ENV'] 750 except KeyError: 751 if not default_ENV: 752 import SCons.Environment 753 # This is a hideously expensive way to get a default shell 754 # environment. What it really should do is run the platform 755 # setup to get the default ENV. Fortunately, it's incredibly 756 # rare for an Environment not to have a shell environment, so 757 # we're not going to worry about it overmuch. 758 default_ENV = SCons.Environment.Environment()['ENV'] 759 return default_ENV

760 761

762 -def _subproc(scons_env, cmd, error = 'ignore', **kw):

763 """Do common setup for a subprocess.Popen() call 764 765 This function is still in draft mode. We're going to need something like 766 it in the long run as more and more places use subprocess, but I'm sure 767 it'll have to be tweaked to get the full desired functionality. 768 one special arg (so far?), 'error', to tell what to do with exceptions. 769 """ 770 # allow std{in,out,err} to be "'devnull'". This is like 771 # subprocess.DEVNULL, which does not exist for Py2. Use the 772 # subprocess one if possible. 773 # Clean this up when Py2 support is dropped 774 try: 775 from subprocess import DEVNULL 776 except ImportError: 777 DEVNULL = None 778 779 for stream in 'stdin', 'stdout', 'stderr': 780 io = kw.get(stream) 781 if is_String(io) and io == 'devnull': 782 if DEVNULL: 783 kw[stream] = DEVNULL 784 else: 785 kw[stream] = open(os.devnull, "r+") 786 787 # Figure out what shell environment to use 788 ENV = kw.get('env', None) 789 if ENV is None: ENV = get_default_ENV(scons_env) 790 791 # Ensure that the ENV values are all strings: 792 new_env = {} 793 for key, value in ENV.items(): 794 if is_List(value): 795 # If the value is a list, then we assume it is a path list, 796 # because that's a pretty common list-like value to stick 797 # in an environment variable: 798 value = SCons.Util.flatten_sequence(value) 799 new_env[key] = os.pathsep.join(map(str, value)) 800 else: 801 # It's either a string or something else. If it's a string, 802 # we still want to call str() because it might be a *Unicode* 803 # string, which makes subprocess.Popen() gag. If it isn't a 804 # string or a list, then we just coerce it to a string, which 805 # is the proper way to handle Dir and File instances and will 806 # produce something reasonable for just about everything else: 807 new_env[key] = str(value) 808 kw['env'] = new_env 809 810 try: 811 pobj = subprocess.Popen(cmd, **kw) 812 except EnvironmentError as e: 813 if error == 'raise': raise 814 # return a dummy Popen instance that only returns error 815 class dummyPopen(object): 816 def __init__(self, e): self.exception = e 817 def communicate(self, input=None): return ('', '') 818 def wait(self): return -self.exception.errno 819 stdin = None 820 class f(object): 821 def read(self): return '' 822 def readline(self): return '' 823 def __iter__(self): return iter(())

824 stdout = stderr = f() 825 pobj = dummyPopen(e) 826 finally: 827 # clean up open file handles stored in parent's kw 828 for k, v in kw.items(): 829 if inspect.ismethod(getattr(v, 'close', None)): 830 v.close() 831 832 return pobj 833 834

835 -class CommandAction(_ActionAction):

836 """Class for command-execution actions."""

837 - def __init__(self, cmd, **kw):

838 # Cmd can actually be a list or a single item; if it's a 839 # single item it should be the command string to execute; if a 840 # list then it should be the words of the command string to 841 # execute. Only a single command should be executed by this 842 # object; lists of commands should be handled by embedding 843 # these objects in a ListAction object (which the Action() 844 # factory above does). cmd will be passed to 845 # Environment.subst_list() for substituting environment 846 # variables. 847 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.CommandAction') 848 849 _ActionAction.__init__(self, **kw) 850 if is_List(cmd): 851 if [c for c in cmd if is_List(c)]: 852 raise TypeError("CommandAction should be given only " 853 "a single command") 854 self.cmd_list = cmd

855

856 - def __str__(self):

857 if is_List(self.cmd_list): 858 return ' '.join(map(str, self.cmd_list)) 859 return str(self.cmd_list)

860

861 - def process(self, target, source, env, executor=None):

862 if executor: 863 result = env.subst_list(self.cmd_list, 0, executor=executor) 864 else: 865 result = env.subst_list(self.cmd_list, 0, target, source) 866 silent = None 867 ignore = None 868 while True: 869 try: c = result[0][0][0] 870 except IndexError: c = None 871 if c == '@': silent = 1 872 elif c == '-': ignore = 1 873 else: break 874 result[0][0] = result[0][0][1:] 875 try: 876 if not result[0][0]: 877 result[0] = result[0][1:] 878 except IndexError: 879 pass 880 return result, ignore, silent

881

882 - def strfunction(self, target, source, env, executor=None):

883 if self.cmdstr is None: 884 return None 885 if self.cmdstr is not _null: 886 from SCons.Subst import SUBST_RAW 887 if executor: 888 c = env.subst(self.cmdstr, SUBST_RAW, executor=executor) 889 else: 890 c = env.subst(self.cmdstr, SUBST_RAW, target, source) 891 if c: 892 return c 893 cmd_list, ignore, silent = self.process(target, source, env, executor) 894 if silent: 895 return '' 896 return _string_from_cmd_list(cmd_list[0])

897

898 - def execute(self, target, source, env, executor=None):

899 """Execute a command action. 900 901 This will handle lists of commands as well as individual commands, 902 because construction variable substitution may turn a single 903 "command" into a list. This means that this class can actually 904 handle lists of commands, even though that's not how we use it 905 externally. 906 """ 907 escape_list = SCons.Subst.escape_list 908 flatten_sequence = SCons.Util.flatten_sequence 909 910 try: 911 shell = env['SHELL'] 912 except KeyError: 913 raise SCons.Errors.UserError('Missing SHELL construction variable.') 914 915 try: 916 spawn = env['SPAWN'] 917 except KeyError: 918 raise SCons.Errors.UserError('Missing SPAWN construction variable.') 919 else: 920 if is_String(spawn): 921 spawn = env.subst(spawn, raw=1, conv=lambda x: x) 922 923 escape = env.get('ESCAPE', lambda x: x) 924 925 ENV = get_default_ENV(env) 926 927 # Ensure that the ENV values are all strings: 928 for key, value in ENV.items(): 929 if not is_String(value): 930 if is_List(value): 931 # If the value is a list, then we assume it is a 932 # path list, because that's a pretty common list-like 933 # value to stick in an environment variable: 934 value = flatten_sequence(value) 935 ENV[key] = os.pathsep.join(map(str, value)) 936 else: 937 # If it isn't a string or a list, then we just coerce 938 # it to a string, which is the proper way to handle 939 # Dir and File instances and will produce something 940 # reasonable for just about everything else: 941 ENV[key] = str(value) 942 943 if executor: 944 target = executor.get_all_targets() 945 source = executor.get_all_sources() 946 cmd_list, ignore, silent = self.process(target, list(map(rfile, source)), env, executor) 947 948 # Use len() to filter out any "command" that's zero-length. 949 for cmd_line in filter(len, cmd_list): 950 # Escape the command line for the interpreter we are using. 951 cmd_line = escape_list(cmd_line, escape) 952 result = spawn(shell, escape, cmd_line[0], cmd_line, ENV) 953 if not ignore and result: 954 msg = "Error %s" % result 955 return SCons.Errors.BuildError(errstr=msg, 956 status=result, 957 action=self, 958 command=cmd_line) 959 return 0

960

961 - def get_presig(self, target, source, env, executor=None):

962 """Return the signature contents of this action's command line. 963 964 This strips $(-$) and everything in between the string, 965 since those parts don't affect signatures. 966 """ 967 from SCons.Subst import SUBST_SIG 968 cmd = self.cmd_list 969 if is_List(cmd): 970 cmd = ' '.join(map(str, cmd)) 971 else: 972 cmd = str(cmd) 973 if executor: 974 return env.subst_target_source(cmd, SUBST_SIG, executor=executor) 975 else: 976 return env.subst_target_source(cmd, SUBST_SIG, target, source)

977

978 - def get_implicit_deps(self, target, source, env, executor=None):

979 icd = env.get('IMPLICIT_COMMAND_DEPENDENCIES', True) 980 if is_String(icd) and icd[:1] == '$': 981 icd = env.subst(icd) 982 if not icd or icd in ('0', 'None'): 983 return [] 984 from SCons.Subst import SUBST_SIG 985 if executor: 986 cmd_list = env.subst_list(self.cmd_list, SUBST_SIG, executor=executor) 987 else: 988 cmd_list = env.subst_list(self.cmd_list, SUBST_SIG, target, source) 989 res = [] 990 for cmd_line in cmd_list: 991 if cmd_line: 992 d = str(cmd_line[0]) 993 m = strip_quotes.match(d) 994 if m: 995 d = m.group(1) 996 d = env.WhereIs(d) 997 if d: 998 res.append(env.fs.File(d)) 999 return res

1000 1001

1002 -class CommandGeneratorAction(ActionBase):

1003 """Class for command-generator actions."""

1004 - def __init__(self, generator, kw):

1005 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.CommandGeneratorAction') 1006 self.generator = generator 1007 self.gen_kw = kw 1008 self.varlist = kw.get('varlist', ()) 1009 self.targets = kw.get('targets', '$TARGETS')

1010

1011 - def _generate(self, target, source, env, for_signature, executor=None):

1012 # ensure that target is a list, to make it easier to write 1013 # generator functions: 1014 if not is_List(target): 1015 target = [target] 1016 1017 if executor: 1018 target = executor.get_all_targets() 1019 source = executor.get_all_sources() 1020 ret = self.generator(target=target, 1021 source=source, 1022 env=env, 1023 for_signature=for_signature) 1024 gen_cmd = Action(ret, **self.gen_kw) 1025 if not gen_cmd: 1026 raise SCons.Errors.UserError("Object returned from command generator: %s cannot be used to create an Action." % repr(ret)) 1027 return gen_cmd

1028

1029 - def __str__(self):

1030 try: 1031 env = self.presub_env 1032 except AttributeError: 1033 env = None 1034 if env is None: 1035 env = SCons.Defaults.DefaultEnvironment() 1036 act = self._generate([], [], env, 1) 1037 return str(act)

1038

1039 - def batch_key(self, env, target, source):

1040 return self._generate(target, source, env, 1).batch_key(env, target, source)

1041

1042 - def genstring(self, target, source, env, executor=None):

1043 return self._generate(target, source, env, 1, executor).genstring(target, source, env)

1044

1045 - def __call__(self, target, source, env, exitstatfunc=_null, presub=_null, 1046 show=_null, execute=_null, chdir=_null, executor=None):

1047 act = self._generate(target, source, env, 0, executor) 1048 if act is None: 1049 raise SCons.Errors.UserError("While building `%s': " 1050 "Cannot deduce file extension from source files: %s" 1051 % (repr(list(map(str, target))), repr(list(map(str, source))))) 1052 return act(target, source, env, exitstatfunc, presub, 1053 show, execute, chdir, executor)

1054

1055 - def get_presig(self, target, source, env, executor=None):

1056 """Return the signature contents of this action's command line. 1057 1058 This strips $(-$) and everything in between the string, 1059 since those parts don't affect signatures. 1060 """ 1061 return self._generate(target, source, env, 1, executor).get_presig(target, source, env)

1062

1063 - def get_implicit_deps(self, target, source, env, executor=None):

1064 return self._generate(target, source, env, 1, executor).get_implicit_deps(target, source, env)

1065

1066 - def get_varlist(self, target, source, env, executor=None):

1067 return self._generate(target, source, env, 1, executor).get_varlist(target, source, env, executor)

1068

1069 - def get_targets(self, env, executor):

1070 return self._generate(None, None, env, 1, executor).get_targets(env, executor)

1071 1072

1073 -class LazyAction(CommandGeneratorAction, CommandAction):

1074 """ 1075 A LazyAction is a kind of hybrid generator and command action for 1076 strings of the form "$VAR". These strings normally expand to other 1077 strings (think "$CCCOM" to "$CC -c -o $TARGET $SOURCE"), but we also 1078 want to be able to replace them with functions in the construction 1079 environment. Consequently, we want lazy evaluation and creation of 1080 an Action in the case of the function, but that's overkill in the more 1081 normal case of expansion to other strings. 1082 1083 So we do this with a subclass that's both a generator *and* 1084 a command action. The overridden methods all do a quick check 1085 of the construction variable, and if it's a string we just call 1086 the corresponding CommandAction method to do the heavy lifting. 1087 If not, then we call the same-named CommandGeneratorAction method. 1088 The CommandGeneratorAction methods work by using the overridden 1089 _generate() method, that is, our own way of handling "generation" of 1090 an action based on what's in the construction variable. 1091 """ 1092

1093 - def __init__(self, var, kw):

1094 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.LazyAction') 1095 CommandAction.__init__(self, '${'+var+'}', **kw) 1096 self.var = SCons.Util.to_String(var) 1097 self.gen_kw = kw

1098

1099 - def get_parent_class(self, env):

1100 c = env.get(self.var) 1101 if is_String(c) and '\n' not in c: 1102 return CommandAction 1103 return CommandGeneratorAction

1104

1105 - def _generate_cache(self, env):

1106 if env: 1107 c = env.get(self.var, '') 1108 else: 1109 c = '' 1110 gen_cmd = Action(c, **self.gen_kw) 1111 if not gen_cmd: 1112 raise SCons.Errors.UserError("$%s value %s cannot be used to create an Action." % (self.var, repr(c))) 1113 return gen_cmd

1114

1115 - def _generate(self, target, source, env, for_signature, executor=None):

1116 return self._generate_cache(env)

1117

1118 - def __call__(self, target, source, env, *args, **kw):

1119 c = self.get_parent_class(env) 1120 return c.__call__(self, target, source, env, *args, **kw)

1121

1122 - def get_presig(self, target, source, env):

1123 c = self.get_parent_class(env) 1124 return c.get_presig(self, target, source, env)

1125

1126 - def get_varlist(self, target, source, env, executor=None):

1127 c = self.get_parent_class(env) 1128 return c.get_varlist(self, target, source, env, executor)

1129 1130

1131 -class FunctionAction(_ActionAction):

1132 """Class for Python function actions.""" 1133

1134 - def __init__(self, execfunction, kw):

1135 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.FunctionAction') 1136 1137 self.execfunction = execfunction 1138 try: 1139 self.funccontents = _callable_contents(execfunction) 1140 except AttributeError: 1141 try: 1142 # See if execfunction will do the heavy lifting for us. 1143 self.gc = execfunction.get_contents 1144 except AttributeError: 1145 # This is weird, just do the best we can. 1146 self.funccontents = _object_contents(execfunction) 1147 1148 _ActionAction.__init__(self, **kw)

1149

1150 - def function_name(self):

1151 try: 1152 return self.execfunction.__name__ 1153 except AttributeError: 1154 try: 1155 return self.execfunction.__class__.__name__ 1156 except AttributeError: 1157 return "unknown_python_function"

1158

1159 - def strfunction(self, target, source, env, executor=None):

1160 if self.cmdstr is None: 1161 return None 1162 if self.cmdstr is not _null: 1163 from SCons.Subst import SUBST_RAW 1164 if executor: 1165 c = env.subst(self.cmdstr, SUBST_RAW, executor=executor) 1166 else: 1167 c = env.subst(self.cmdstr, SUBST_RAW, target, source) 1168 if c: 1169 return c 1170 1171 def array(a): 1172 def quote(s): 1173 try: 1174 str_for_display = s.str_for_display 1175 except AttributeError: 1176 s = repr(s) 1177 else: 1178 s = str_for_display() 1179 return s

1180 return '[' + ", ".join(map(quote, a)) + ']'

1181 try: 1182 strfunc = self.execfunction.strfunction 1183 except AttributeError: 1184 pass 1185 else: 1186 if strfunc is None: 1187 return None 1188 if callable(strfunc): 1189 return strfunc(target, source, env) 1190 name = self.function_name() 1191 tstr = array(target) 1192 sstr = array(source) 1193 return "%s(%s, %s)" % (name, tstr, sstr) 1194

1195 - def __str__(self):

1196 name = self.function_name() 1197 if name == 'ActionCaller': 1198 return str(self.execfunction) 1199 return "%s(target, source, env)" % name

1200

1201 - def execute(self, target, source, env, executor=None):

1202 exc_info = (None,None,None) 1203 try: 1204 if executor: 1205 target = executor.get_all_targets() 1206 source = executor.get_all_sources() 1207 rsources = list(map(rfile, source)) 1208 try: 1209 result = self.execfunction(target=target, source=rsources, env=env) 1210 except KeyboardInterrupt as e: 1211 raise 1212 except SystemExit as e: 1213 raise 1214 except Exception as e: 1215 result = e 1216 exc_info = sys.exc_info() 1217 1218 if result: 1219 result = SCons.Errors.convert_to_BuildError(result, exc_info) 1220 result.node=target 1221 result.action=self 1222 try: 1223 result.command=self.strfunction(target, source, env, executor) 1224 except TypeError: 1225 result.command=self.strfunction(target, source, env) 1226 1227 # FIXME: This maintains backward compatibility with respect to 1228 # which type of exceptions were returned by raising an 1229 # exception and which ones were returned by value. It would 1230 # probably be best to always return them by value here, but 1231 # some codes do not check the return value of Actions and I do 1232 # not have the time to modify them at this point. 1233 if (exc_info[1] and 1234 not isinstance(exc_info[1],EnvironmentError)): 1235 raise result 1236 1237 return result 1238 finally: 1239 # Break the cycle between the traceback object and this 1240 # function stack frame. See the sys.exc_info() doc info for 1241 # more information about this issue. 1242 del exc_info

1243

1244 - def get_presig(self, target, source, env):

1245 """Return the signature contents of this callable action.""" 1246 try: 1247 return self.gc(target, source, env) 1248 except AttributeError: 1249 return self.funccontents

1250

1251 - def get_implicit_deps(self, target, source, env):

1252 return []

1253

1254 -class ListAction(ActionBase):

1255 """Class for lists of other actions."""

1256 - def __init__(self, actionlist):

1257 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.ListAction') 1258 def list_of_actions(x): 1259 if isinstance(x, ActionBase): 1260 return x 1261 return Action(x)

1262 self.list = list(map(list_of_actions, actionlist)) 1263 # our children will have had any varlist 1264 # applied; we don't need to do it again 1265 self.varlist = () 1266 self.targets = '$TARGETS'

1267

1268 - def genstring(self, target, source, env):

1269 return '\n'.join([a.genstring(target, source, env) for a in self.list])

1270

1271 - def __str__(self):

1272 return '\n'.join(map(str, self.list))

1273

1274 - def presub_lines(self, env):

1275 return SCons.Util.flatten_sequence( 1276 [a.presub_lines(env) for a in self.list])

1277

1278 - def get_presig(self, target, source, env):

1279 """Return the signature contents of this action list. 1280 1281 Simple concatenation of the signatures of the elements. 1282 """ 1283 return b"".join([bytes(x.get_contents(target, source, env)) for x in self.list])

1284

1285 - def __call__(self, target, source, env, exitstatfunc=_null, presub=_null, 1286 show=_null, execute=_null, chdir=_null, executor=None):

1287 if executor: 1288 target = executor.get_all_targets() 1289 source = executor.get_all_sources() 1290 for act in self.list: 1291 stat = act(target, source, env, exitstatfunc, presub, 1292 show, execute, chdir, executor) 1293 if stat: 1294 return stat 1295 return 0

1296

1297 - def get_implicit_deps(self, target, source, env):

1298 result = [] 1299 for act in self.list: 1300 result.extend(act.get_implicit_deps(target, source, env)) 1301 return result

1302

1303 - def get_varlist(self, target, source, env, executor=None):

1304 result = OrderedDict() 1305 for act in self.list: 1306 for var in act.get_varlist(target, source, env, executor): 1307 result[var] = True 1308 return list(result.keys())

1309 1310

1311 -class ActionCaller(object):

1312 """A class for delaying calling an Action function with specific 1313 (positional and keyword) arguments until the Action is actually 1314 executed. 1315 1316 This class looks to the rest of the world like a normal Action object, 1317 but what it's really doing is hanging on to the arguments until we 1318 have a target, source and env to use for the expansion. 1319 """

1320 - def __init__(self, parent, args, kw):

1321 self.parent = parent 1322 self.args = args 1323 self.kw = kw

1324

1325 - def get_contents(self, target, source, env):

1326 actfunc = self.parent.actfunc 1327 try: 1328 # "self.actfunc" is a function. 1329 contents = actfunc.__code__.co_code 1330 except AttributeError: 1331 # "self.actfunc" is a callable object. 1332 try: 1333 contents = actfunc.__call__.__func__.__code__.co_code 1334 except AttributeError: 1335 # No __call__() method, so it might be a builtin 1336 # or something like that. Do the best we can. 1337 contents = repr(actfunc) 1338 1339 return contents

1340

1341 - def subst(self, s, target, source, env):

1342 # If s is a list, recursively apply subst() 1343 # to every element in the list 1344 if is_List(s): 1345 result = [] 1346 for elem in s: 1347 result.append(self.subst(elem, target, source, env)) 1348 return self.parent.convert(result) 1349 1350 # Special-case hack: Let a custom function wrapped in an 1351 # ActionCaller get at the environment through which the action 1352 # was called by using this hard-coded value as a special return. 1353 if s == '$__env__': 1354 return env 1355 elif is_String(s): 1356 return env.subst(s, 1, target, source) 1357 return self.parent.convert(s)

1358

1359 - def subst_args(self, target, source, env):

1360 return [self.subst(x, target, source, env) for x in self.args]

1361

1362 - def subst_kw(self, target, source, env):

1363 kw = {} 1364 for key in list(self.kw.keys()): 1365 kw[key] = self.subst(self.kw[key], target, source, env) 1366 return kw

1367

1368 - def __call__(self, target, source, env, executor=None):

1369 args = self.subst_args(target, source, env) 1370 kw = self.subst_kw(target, source, env) 1371 return self.parent.actfunc(*args, **kw)

1372

1373 - def strfunction(self, target, source, env):

1374 args = self.subst_args(target, source, env) 1375 kw = self.subst_kw(target, source, env) 1376 return self.parent.strfunc(*args, **kw)

1377

1378 - def __str__(self):

1379 return self.parent.strfunc(*self.args, **self.kw)

1380 1381

1382 -class ActionFactory(object):

1383 """A factory class that will wrap up an arbitrary function 1384 as an SCons-executable Action object. 1385 1386 The real heavy lifting here is done by the ActionCaller class. 1387 We just collect the (positional and keyword) arguments that we're 1388 called with and give them to the ActionCaller object we create, 1389 so it can hang onto them until it needs them. 1390 """

1391 - def __init__(self, actfunc, strfunc, convert=lambda x: x):

1392 self.actfunc = actfunc 1393 self.strfunc = strfunc 1394 self.convert = convert

1395

1396 - def __call__(self, *args, **kw):

1397 ac = ActionCaller(self, args, kw) 1398 action = Action(ac, strfunction=ac.strfunction) 1399 return action

1400 1401 # Local Variables: 1402 # tab-width:4 1403 # indent-tabs-mode:nil 1404 # End: 1405 # vim: set expandtab tabstop=4 shiftwidth=4: 1406

Source Code for Module SCons.Action