SCons.Action

1 """SCons.Action 2 3 This encapsulates information about executing any sort of action that 4 can build one or more target Nodes (typically files) from one or more 5 source Nodes (also typically files) given a specific Environment. 6 7 The base class here is ActionBase. The base class supplies just a few 8 OO utility methods and some generic methods for displaying information 9 about an Action in response to the various commands that control printing. 10 11 A second-level base class is _ActionAction. This extends ActionBase 12 by providing the methods that can be used to show and perform an 13 action. True Action objects will subclass _ActionAction; Action 14 factory class objects will subclass ActionBase. 15 16 The heavy lifting is handled by subclasses for the different types of 17 actions we might execute: 18 19 CommandAction 20 CommandGeneratorAction 21 FunctionAction 22 ListAction 23 24 The subclasses supply the following public interface methods used by 25 other modules: 26 27 __call__() 28 THE public interface, "calling" an Action object executes the 29 command or Python function. This also takes care of printing 30 a pre-substitution command for debugging purposes. 31 32 get_contents() 33 Fetches the "contents" of an Action for signature calculation 34 plus the varlist. This is what gets MD5 checksummed to decide 35 if a target needs to be rebuilt because its action changed. 36 37 genstring() 38 Returns a string representation of the Action *without* 39 command substitution, but allows a CommandGeneratorAction to 40 generate the right action based on the specified target, 41 source and env. This is used by the Signature subsystem 42 (through the Executor) to obtain an (imprecise) representation 43 of the Action operation for informative purposes. 44 45 46 Subclasses also supply the following methods for internal use within 47 this module: 48 49 __str__() 50 Returns a string approximation of the Action; no variable 51 substitution is performed. 52 53 execute() 54 The internal method that really, truly, actually handles the 55 execution of a command or Python function. This is used so 56 that the __call__() methods can take care of displaying any 57 pre-substitution representations, and *then* execute an action 58 without worrying about the specific Actions involved. 59 60 get_presig() 61 Fetches the "contents" of a subclass for signature calculation. 62 The varlist is added to this to produce the Action's contents. 63 TODO(?): Change this to always return ascii/bytes and not unicode (or py3 strings) 64 65 strfunction() 66 Returns a substituted string representation of the Action. 67 This is used by the _ActionAction.show() command to display the 68 command/function that will be executed to generate the target(s). 69 70 There is a related independent ActionCaller class that looks like a 71 regular Action, and which serves as a wrapper for arbitrary functions 72 that we want to let the user specify the arguments to now, but actually 73 execute later (when an out-of-date check determines that it's needed to 74 be executed, for example). Objects of this class are returned by an 75 ActionFactory class that provides a __call__() method as a convenient 76 way for wrapping up the functions. 77 78 """ 79 80 # Copyright (c) 2001 - 2019 The SCons Foundation 81 # 82 # Permission is hereby granted, free of charge, to any person obtaining 83 # a copy of this software and associated documentation files (the 84 # "Software"), to deal in the Software without restriction, including 85 # without limitation the rights to use, copy, modify, merge, publish, 86 # distribute, sublicense, and/or sell copies of the Software, and to 87 # permit persons to whom the Software is furnished to do so, subject to 88 # the following conditions: 89 # 90 # The above copyright notice and this permission notice shall be included 91 # in all copies or substantial portions of the Software. 92 # 93 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY 94 # KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE 95 # WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 96 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 97 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 98 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 99 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 100 101 __revision__ = "src/engine/SCons/Action.py 3a41ed6b288cee8d085373ad7fa02894e1903864 2019-01-23 17:30:35 bdeegan" 102 103 import os 104 import pickle 105 import re 106 import sys 107 import subprocess 108 import itertools 109 import inspect 110 from collections import OrderedDict 111 112 import SCons.Debug 113 from SCons.Debug import logInstanceCreation 114 import SCons.Errors 115 import SCons.Util 116 import SCons.Subst 117 118 # we use these a lot, so try to optimize them 119 from SCons.Util import is_String, is_List 120

121 -class _null(object):

122 pass

123 124 print_actions = 1 125 execute_actions = 1 126 print_actions_presub = 0 127 128 # Use pickle protocol 1 when pickling functions for signature 129 # otherwise python3 and python2 will yield different pickles 130 # for the same object. 131 # This is due to default being 1 for python 2.7, and 3 for 3.x 132 # TODO: We can roll this forward to 2 (if it has value), but not 133 # before a deprecation cycle as the sconsigns will change 134 ACTION_SIGNATURE_PICKLE_PROTOCOL = 1 135 136

137 -def rfile(n):

138 try: 139 return n.rfile() 140 except AttributeError: 141 return n

142 143

144 -def default_exitstatfunc(s):

145 return s

146 147 strip_quotes = re.compile('^[\'"](.*)[\'"]$') 148 149

150 -def _callable_contents(obj):

151 """Return the signature contents of a callable Python object. 152 """ 153 try: 154 # Test if obj is a method. 155 return _function_contents(obj.__func__) 156 157 except AttributeError: 158 try: 159 # Test if obj is a callable object. 160 return _function_contents(obj.__call__.__func__) 161 162 except AttributeError: 163 try: 164 # Test if obj is a code object. 165 return _code_contents(obj) 166 167 except AttributeError: 168 # Test if obj is a function object. 169 return _function_contents(obj)

170 171

172 -def _object_contents(obj):

173 """Return the signature contents of any Python object. 174 175 We have to handle the case where object contains a code object 176 since it can be pickled directly. 177 """ 178 try: 179 # Test if obj is a method. 180 return _function_contents(obj.__func__) 181 182 except AttributeError: 183 try: 184 # Test if obj is a callable object. 185 return _function_contents(obj.__call__.__func__) 186 187 except AttributeError: 188 try: 189 # Test if obj is a code object. 190 return _code_contents(obj) 191 192 except AttributeError: 193 try: 194 # Test if obj is a function object. 195 return _function_contents(obj) 196 197 except AttributeError as ae: 198 # Should be a pickle-able Python object. 199 try: 200 return _object_instance_content(obj) 201 # pickling an Action instance or object doesn't yield a stable 202 # content as instance property may be dumped in different orders 203 # return pickle.dumps(obj, ACTION_SIGNATURE_PICKLE_PROTOCOL) 204 except (pickle.PicklingError, TypeError, AttributeError) as ex: 205 # This is weird, but it seems that nested classes 206 # are unpickable. The Python docs say it should 207 # always be a PicklingError, but some Python 208 # versions seem to return TypeError. Just do 209 # the best we can. 210 return bytearray(repr(obj), 'utf-8')

211 212

213 -def _code_contents(code, docstring=None):

214 """Return the signature contents of a code object. 215 216 By providing direct access to the code object of the 217 function, Python makes this extremely easy. Hooray! 218 219 Unfortunately, older versions of Python include line 220 number indications in the compiled byte code. Boo! 221 So we remove the line number byte codes to prevent 222 recompilations from moving a Python function. 223 224 See: 225 - https://docs.python.org/2/library/inspect.html 226 - http://python-reference.readthedocs.io/en/latest/docs/code/index.html 227 228 For info on what each co\_ variable provides 229 230 The signature is as follows (should be byte/chars): 231 co_argcount, len(co_varnames), len(co_cellvars), len(co_freevars), 232 ( comma separated signature for each object in co_consts ), 233 ( comma separated signature for each object in co_names ), 234 ( The bytecode with line number bytecodes removed from co_code ) 235 236 co_argcount - Returns the number of positional arguments (including arguments with default values). 237 co_varnames - Returns a tuple containing the names of the local variables (starting with the argument names). 238 co_cellvars - Returns a tuple containing the names of local variables that are referenced by nested functions. 239 co_freevars - Returns a tuple containing the names of free variables. (?) 240 co_consts - Returns a tuple containing the literals used by the bytecode. 241 co_names - Returns a tuple containing the names used by the bytecode. 242 co_code - Returns a string representing the sequence of bytecode instructions. 243 244 """ 245 246 # contents = [] 247 248 # The code contents depends on the number of local variables 249 # but not their actual names. 250 contents = bytearray("{}, {}".format(code.co_argcount, len(code.co_varnames)), 'utf-8') 251 252 contents.extend(b", ") 253 contents.extend(bytearray(str(len(code.co_cellvars)), 'utf-8')) 254 contents.extend(b", ") 255 contents.extend(bytearray(str(len(code.co_freevars)), 'utf-8')) 256 257 # The code contents depends on any constants accessed by the 258 # function. Note that we have to call _object_contents on each 259 # constants because the code object of nested functions can 260 # show-up among the constants. 261 z = [_object_contents(cc) for cc in code.co_consts if cc != docstring] 262 contents.extend(b',(') 263 contents.extend(bytearray(',', 'utf-8').join(z)) 264 contents.extend(b')') 265 266 # The code contents depends on the variable names used to 267 # accessed global variable, as changing the variable name changes 268 # the variable actually accessed and therefore changes the 269 # function result. 270 z= [bytearray(_object_contents(cc)) for cc in code.co_names] 271 contents.extend(b',(') 272 contents.extend(bytearray(',','utf-8').join(z)) 273 contents.extend(b')') 274 275 # The code contents depends on its actual code!!! 276 contents.extend(b',(') 277 contents.extend(code.co_code) 278 contents.extend(b')') 279 280 return contents

281 282

283 -def _function_contents(func):

284 """ 285 The signature is as follows (should be byte/chars): 286 < _code_contents (see above) from func.__code__ > 287 ,( comma separated _object_contents for function argument defaults) 288 ,( comma separated _object_contents for any closure contents ) 289 290 291 See also: https://docs.python.org/3/reference/datamodel.html 292 - func.__code__ - The code object representing the compiled function body. 293 - func.__defaults__ - A tuple containing default argument values for those arguments that have defaults, or None if no arguments have a default value 294 - func.__closure__ - None or a tuple of cells that contain bindings for the function's free variables. 295 296 :Returns: 297 Signature contents of a function. (in bytes) 298 """ 299 300 contents = [_code_contents(func.__code__, func.__doc__)] 301 302 # The function contents depends on the value of defaults arguments 303 if func.__defaults__: 304 305 function_defaults_contents = [_object_contents(cc) for cc in func.__defaults__] 306 307 defaults = bytearray(b',(') 308 defaults.extend(bytearray(b',').join(function_defaults_contents)) 309 defaults.extend(b')') 310 311 contents.append(defaults) 312 else: 313 contents.append(b',()') 314 315 # The function contents depends on the closure captured cell values. 316 closure = func.__closure__ or [] 317 318 try: 319 closure_contents = [_object_contents(x.cell_contents) for x in closure] 320 except AttributeError: 321 closure_contents = [] 322 323 contents.append(b',(') 324 contents.append(bytearray(b',').join(closure_contents)) 325 contents.append(b')') 326 327 retval = bytearray(b'').join(contents) 328 return retval

329 330

331 -def _object_instance_content(obj):

332 """ 333 Returns consistant content for a action class or an instance thereof 334 335 :Parameters: 336 - `obj` Should be either and action class or an instance thereof 337 338 :Returns: 339 bytearray or bytes representing the obj suitable for generating a signature from. 340 """ 341 retval = bytearray() 342 343 if obj is None: 344 return b'N.' 345 346 if isinstance(obj, SCons.Util.BaseStringTypes): 347 return SCons.Util.to_bytes(obj) 348 349 inst_class = obj.__class__ 350 inst_class_name = bytearray(obj.__class__.__name__,'utf-8') 351 inst_class_module = bytearray(obj.__class__.__module__,'utf-8') 352 inst_class_hierarchy = bytearray(repr(inspect.getclasstree([obj.__class__,])),'utf-8') 353 # print("ICH:%s : %s"%(inst_class_hierarchy, repr(obj))) 354 355 properties = [(p, getattr(obj, p, "None")) for p in dir(obj) if not (p[:2] == '__' or inspect.ismethod(getattr(obj, p)) or inspect.isbuiltin(getattr(obj,p))) ] 356 properties.sort() 357 properties_str = ','.join(["%s=%s"%(p[0],p[1]) for p in properties]) 358 properties_bytes = bytearray(properties_str,'utf-8') 359 360 methods = [p for p in dir(obj) if inspect.ismethod(getattr(obj, p))] 361 methods.sort() 362 363 method_contents = [] 364 for m in methods: 365 # print("Method:%s"%m) 366 v = _function_contents(getattr(obj, m)) 367 # print("[%s->]V:%s [%s]"%(m,v,type(v))) 368 method_contents.append(v) 369 370 retval = bytearray(b'{') 371 retval.extend(inst_class_name) 372 retval.extend(b":") 373 retval.extend(inst_class_module) 374 retval.extend(b'}[[') 375 retval.extend(inst_class_hierarchy) 376 retval.extend(b']]{{') 377 retval.extend(bytearray(b",").join(method_contents)) 378 retval.extend(b"}}{{{") 379 retval.extend(properties_bytes) 380 retval.extend(b'}}}') 381 return retval

382 383 # print("class :%s"%inst_class) 384 # print("class_name :%s"%inst_class_name) 385 # print("class_module :%s"%inst_class_module) 386 # print("Class hier :\n%s"%pp.pformat(inst_class_hierarchy)) 387 # print("Inst Properties:\n%s"%pp.pformat(properties)) 388 # print("Inst Methods :\n%s"%pp.pformat(methods)) 389

390 -def _actionAppend(act1, act2):

391 # This function knows how to slap two actions together. 392 # Mainly, it handles ListActions by concatenating into 393 # a single ListAction. 394 a1 = Action(act1) 395 a2 = Action(act2) 396 if a1 is None: 397 return a2 398 if a2 is None: 399 return a1 400 if isinstance(a1, ListAction): 401 if isinstance(a2, ListAction): 402 return ListAction(a1.list + a2.list) 403 else: 404 return ListAction(a1.list + [ a2 ]) 405 else: 406 if isinstance(a2, ListAction): 407 return ListAction([ a1 ] + a2.list) 408 else: 409 return ListAction([ a1, a2 ])

410 411

412 -def _do_create_keywords(args, kw):

413 """This converts any arguments after the action argument into 414 their equivalent keywords and adds them to the kw argument. 415 """ 416 v = kw.get('varlist', ()) 417 # prevent varlist="FOO" from being interpreted as ['F', 'O', 'O'] 418 if is_String(v): v = (v,) 419 kw['varlist'] = tuple(v) 420 if args: 421 # turn positional args into equivalent keywords 422 cmdstrfunc = args[0] 423 if cmdstrfunc is None or is_String(cmdstrfunc): 424 kw['cmdstr'] = cmdstrfunc 425 elif callable(cmdstrfunc): 426 kw['strfunction'] = cmdstrfunc 427 else: 428 raise SCons.Errors.UserError( 429 'Invalid command display variable type. ' 430 'You must either pass a string or a callback which ' 431 'accepts (target, source, env) as parameters.') 432 if len(args) > 1: 433 kw['varlist'] = tuple(SCons.Util.flatten(args[1:])) + kw['varlist'] 434 if kw.get('strfunction', _null) is not _null \ 435 and kw.get('cmdstr', _null) is not _null: 436 raise SCons.Errors.UserError( 437 'Cannot have both strfunction and cmdstr args to Action()')

438 439

440 -def _do_create_action(act, kw):

441 """This is the actual "implementation" for the 442 Action factory method, below. This handles the 443 fact that passing lists to Action() itself has 444 different semantics than passing lists as elements 445 of lists. 446 447 The former will create a ListAction, the latter 448 will create a CommandAction by converting the inner 449 list elements to strings.""" 450 451 if isinstance(act, ActionBase): 452 return act 453 454 if is_String(act): 455 var=SCons.Util.get_environment_var(act) 456 if var: 457 # This looks like a string that is purely an Environment 458 # variable reference, like "$FOO" or "${FOO}". We do 459 # something special here...we lazily evaluate the contents 460 # of that Environment variable, so a user could put something 461 # like a function or a CommandGenerator in that variable 462 # instead of a string. 463 return LazyAction(var, kw) 464 commands = str(act).split('\n') 465 if len(commands) == 1: 466 return CommandAction(commands[0], **kw) 467 # The list of string commands may include a LazyAction, so we 468 # reprocess them via _do_create_list_action. 469 return _do_create_list_action(commands, kw) 470 471 if is_List(act): 472 return CommandAction(act, **kw) 473 474 if callable(act): 475 try: 476 gen = kw['generator'] 477 del kw['generator'] 478 except KeyError: 479 gen = 0 480 if gen: 481 action_type = CommandGeneratorAction 482 else: 483 action_type = FunctionAction 484 return action_type(act, kw) 485 486 # Catch a common error case with a nice message: 487 if isinstance(act, int) or isinstance(act, float): 488 raise TypeError("Don't know how to create an Action from a number (%s)"%act) 489 # Else fail silently (???) 490 return None

491 492

493 -def _do_create_list_action(act, kw):

494 """A factory for list actions. Convert the input list into Actions 495 and then wrap them in a ListAction.""" 496 acts = [] 497 for a in act: 498 aa = _do_create_action(a, kw) 499 if aa is not None: acts.append(aa) 500 if not acts: 501 return ListAction([]) 502 elif len(acts) == 1: 503 return acts[0] 504 else: 505 return ListAction(acts)

506 507

508 -def Action(act, *args, **kw):

509 """A factory for action objects.""" 510 # Really simple: the _do_create_* routines do the heavy lifting. 511 _do_create_keywords(args, kw) 512 if is_List(act): 513 return _do_create_list_action(act, kw) 514 return _do_create_action(act, kw)

515 516

517 -class ActionBase(object):

518 """Base class for all types of action objects that can be held by 519 other objects (Builders, Executors, etc.) This provides the 520 common methods for manipulating and combining those actions.""" 521

522 - def __eq__(self, other):

523 return self.__dict__ == other

524

525 - def no_batch_key(self, env, target, source):

526 return None

527 528 batch_key = no_batch_key 529

530 - def genstring(self, target, source, env):

531 return str(self)

532

533 - def get_contents(self, target, source, env):

534 result = self.get_presig(target, source, env) 535 536 if not isinstance(result,(bytes, bytearray)): 537 result = bytearray("",'utf-8').join([ SCons.Util.to_bytes(r) for r in result ]) 538 else: 539 # Make a copy and put in bytearray, without this the contents returned by get_presig 540 # can be changed by the logic below, appending with each call and causing very 541 # hard to track down issues... 542 result = bytearray(result) 543 544 # At this point everything should be a bytearray 545 546 # This should never happen, as the Action() factory should wrap 547 # the varlist, but just in case an action is created directly, 548 # we duplicate this check here. 549 vl = self.get_varlist(target, source, env) 550 if is_String(vl): vl = (vl,) 551 for v in vl: 552 # do the subst this way to ignore $(...$) parts: 553 if isinstance(result, bytearray): 554 result.extend(SCons.Util.to_bytes(env.subst_target_source('${'+v+'}', SCons.Subst.SUBST_SIG, target, source))) 555 else: 556 raise Exception("WE SHOULD NEVER GET HERE result should be bytearray not:%s"%type(result)) 557 # result.append(SCons.Util.to_bytes(env.subst_target_source('${'+v+'}', SCons.Subst.SUBST_SIG, target, source))) 558 559 560 if isinstance(result, (bytes,bytearray)): 561 return result 562 else: 563 raise Exception("WE SHOULD NEVER GET HERE - #2 result should be bytearray not:%s" % type(result))

564 # return b''.join(result) 565

566 - def __add__(self, other):

567 return _actionAppend(self, other)

568

569 - def __radd__(self, other):

570 return _actionAppend(other, self)

571

572 - def presub_lines(self, env):

573 # CommandGeneratorAction needs a real environment 574 # in order to return the proper string here, since 575 # it may call LazyAction, which looks up a key 576 # in that env. So we temporarily remember the env here, 577 # and CommandGeneratorAction will use this env 578 # when it calls its _generate method. 579 self.presub_env = env 580 lines = str(self).split('\n') 581 self.presub_env = None # don't need this any more 582 return lines

583

584 - def get_varlist(self, target, source, env, executor=None):

585 return self.varlist

586

587 - def get_targets(self, env, executor):

588 """ 589 Returns the type of targets ($TARGETS, $CHANGED_TARGETS) used 590 by this action. 591 """ 592 return self.targets

593 594

595 -class _ActionAction(ActionBase):

596 """Base class for actions that create output objects."""

597 - def __init__(self, cmdstr=_null, strfunction=_null, varlist=(), 598 presub=_null, chdir=None, exitstatfunc=None, 599 batch_key=None, targets='$TARGETS', 600 **kw):

601 self.cmdstr = cmdstr 602 if strfunction is not _null: 603 if strfunction is None: 604 self.cmdstr = None 605 else: 606 self.strfunction = strfunction 607 self.varlist = varlist 608 self.presub = presub 609 self.chdir = chdir 610 if not exitstatfunc: 611 exitstatfunc = default_exitstatfunc 612 self.exitstatfunc = exitstatfunc 613 614 self.targets = targets 615 616 if batch_key: 617 if not callable(batch_key): 618 # They have set batch_key, but not to their own 619 # callable. The default behavior here will batch 620 # *all* targets+sources using this action, separated 621 # for each construction environment. 622 def default_batch_key(self, env, target, source): 623 return (id(self), id(env))

624 batch_key = default_batch_key 625 SCons.Util.AddMethod(self, batch_key, 'batch_key')

626

627 - def print_cmd_line(self, s, target, source, env):

628 """ 629 In python 3, and in some of our tests, sys.stdout is 630 a String io object, and it takes unicode strings only 631 In other cases it's a regular Python 2.x file object 632 which takes strings (bytes), and if you pass those a 633 unicode object they try to decode with 'ascii' codec 634 which fails if the cmd line has any hi-bit-set chars. 635 This code assumes s is a regular string, but should 636 work if it's unicode too. 637 """ 638 try: 639 sys.stdout.write(s + u"\n") 640 except UnicodeDecodeError: 641 sys.stdout.write(s + "\n")

642

643 - def __call__(self, target, source, env, 644 exitstatfunc=_null, 645 presub=_null, 646 show=_null, 647 execute=_null, 648 chdir=_null, 649 executor=None):

650 if not is_List(target): 651 target = [target] 652 if not is_List(source): 653 source = [source] 654 655 if presub is _null: 656 presub = self.presub 657 if presub is _null: 658 presub = print_actions_presub 659 if exitstatfunc is _null: exitstatfunc = self.exitstatfunc 660 if show is _null: show = print_actions 661 if execute is _null: execute = execute_actions 662 if chdir is _null: chdir = self.chdir 663 save_cwd = None 664 if chdir: 665 save_cwd = os.getcwd() 666 try: 667 chdir = str(chdir.get_abspath()) 668 except AttributeError: 669 if not is_String(chdir): 670 if executor: 671 chdir = str(executor.batches[0].targets[0].dir) 672 else: 673 chdir = str(target[0].dir) 674 if presub: 675 if executor: 676 target = executor.get_all_targets() 677 source = executor.get_all_sources() 678 t = ' and '.join(map(str, target)) 679 l = '\n '.join(self.presub_lines(env)) 680 out = u"Building %s with action:\n %s\n" % (t, l) 681 sys.stdout.write(out) 682 cmd = None 683 if show and self.strfunction: 684 if executor: 685 target = executor.get_all_targets() 686 source = executor.get_all_sources() 687 try: 688 cmd = self.strfunction(target, source, env, executor) 689 except TypeError: 690 cmd = self.strfunction(target, source, env) 691 if cmd: 692 if chdir: 693 cmd = ('os.chdir(%s)\n' % repr(chdir)) + cmd 694 try: 695 get = env.get 696 except AttributeError: 697 print_func = self.print_cmd_line 698 else: 699 print_func = get('PRINT_CMD_LINE_FUNC') 700 if not print_func: 701 print_func = self.print_cmd_line 702 print_func(cmd, target, source, env) 703 stat = 0 704 if execute: 705 if chdir: 706 os.chdir(chdir) 707 try: 708 stat = self.execute(target, source, env, executor=executor) 709 if isinstance(stat, SCons.Errors.BuildError): 710 s = exitstatfunc(stat.status) 711 if s: 712 stat.status = s 713 else: 714 stat = s 715 else: 716 stat = exitstatfunc(stat) 717 finally: 718 if save_cwd: 719 os.chdir(save_cwd) 720 if cmd and save_cwd: 721 print_func('os.chdir(%s)' % repr(save_cwd), target, source, env) 722 723 return stat

724 725

726 -def _string_from_cmd_list(cmd_list):

727 """Takes a list of command line arguments and returns a pretty 728 representation for printing.""" 729 cl = [] 730 for arg in map(str, cmd_list): 731 if ' ' in arg or '\t' in arg: 732 arg = '"' + arg + '"' 733 cl.append(arg) 734 return ' '.join(cl)

735 736 default_ENV = None 737 738

739 -def get_default_ENV(env):

740 """ 741 A fiddlin' little function that has an 'import SCons.Environment' which 742 can't be moved to the top level without creating an import loop. Since 743 this import creates a local variable named 'SCons', it blocks access to 744 the global variable, so we move it here to prevent complaints about local 745 variables being used uninitialized. 746 """ 747 global default_ENV 748 try: 749 return env['ENV'] 750 except KeyError: 751 if not default_ENV: 752 import SCons.Environment 753 # This is a hideously expensive way to get a default shell 754 # environment. What it really should do is run the platform 755 # setup to get the default ENV. Fortunately, it's incredibly 756 # rare for an Environment not to have a shell environment, so 757 # we're not going to worry about it overmuch. 758 default_ENV = SCons.Environment.Environment()['ENV'] 759 return default_ENV

760 761

762 -def _subproc(scons_env, cmd, error = 'ignore', **kw):

763 """Do common setup for a subprocess.Popen() call 764 765 This function is still in draft mode. We're going to need something like 766 it in the long run as more and more places use subprocess, but I'm sure 767 it'll have to be tweaked to get the full desired functionality. 768 one special arg (so far?), 'error', to tell what to do with exceptions. 769 """ 770 # allow std{in,out,err} to be "'devnull'" 771 io = kw.get('stdin') 772 if is_String(io) and io == 'devnull': 773 kw['stdin'] = open(os.devnull) 774 io = kw.get('stdout') 775 if is_String(io) and io == 'devnull': 776 kw['stdout'] = open(os.devnull, 'w') 777 io = kw.get('stderr') 778 if is_String(io) and io == 'devnull': 779 kw['stderr'] = open(os.devnull, 'w') 780 781 # Figure out what shell environment to use 782 ENV = kw.get('env', None) 783 if ENV is None: ENV = get_default_ENV(scons_env) 784 785 # Ensure that the ENV values are all strings: 786 new_env = {} 787 for key, value in ENV.items(): 788 if is_List(value): 789 # If the value is a list, then we assume it is a path list, 790 # because that's a pretty common list-like value to stick 791 # in an environment variable: 792 value = SCons.Util.flatten_sequence(value) 793 new_env[key] = os.pathsep.join(map(str, value)) 794 else: 795 # It's either a string or something else. If it's a string, 796 # we still want to call str() because it might be a *Unicode* 797 # string, which makes subprocess.Popen() gag. If it isn't a 798 # string or a list, then we just coerce it to a string, which 799 # is the proper way to handle Dir and File instances and will 800 # produce something reasonable for just about everything else: 801 new_env[key] = str(value) 802 kw['env'] = new_env 803 804 try: 805 pobj = subprocess.Popen(cmd, **kw) 806 except EnvironmentError as e: 807 if error == 'raise': raise 808 # return a dummy Popen instance that only returns error 809 class dummyPopen(object): 810 def __init__(self, e): self.exception = e 811 def communicate(self, input=None): return ('', '') 812 def wait(self): return -self.exception.errno 813 stdin = None 814 class f(object): 815 def read(self): return '' 816 def readline(self): return '' 817 def __iter__(self): return iter(())

818 stdout = stderr = f() 819 pobj = dummyPopen(e) 820 finally: 821 # clean up open file handles stored in parent's kw 822 for k, v in kw.items(): 823 if hasattr(v, 'close'): 824 v.close() 825 return pobj 826 827

828 -class CommandAction(_ActionAction):

829 """Class for command-execution actions."""

830 - def __init__(self, cmd, **kw):

831 # Cmd can actually be a list or a single item; if it's a 832 # single item it should be the command string to execute; if a 833 # list then it should be the words of the command string to 834 # execute. Only a single command should be executed by this 835 # object; lists of commands should be handled by embedding 836 # these objects in a ListAction object (which the Action() 837 # factory above does). cmd will be passed to 838 # Environment.subst_list() for substituting environment 839 # variables. 840 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.CommandAction') 841 842 _ActionAction.__init__(self, **kw) 843 if is_List(cmd): 844 if [c for c in cmd if is_List(c)]: 845 raise TypeError("CommandAction should be given only " 846 "a single command") 847 self.cmd_list = cmd

848

849 - def __str__(self):

850 if is_List(self.cmd_list): 851 return ' '.join(map(str, self.cmd_list)) 852 return str(self.cmd_list)

853

854 - def process(self, target, source, env, executor=None):

855 if executor: 856 result = env.subst_list(self.cmd_list, 0, executor=executor) 857 else: 858 result = env.subst_list(self.cmd_list, 0, target, source) 859 silent = None 860 ignore = None 861 while True: 862 try: c = result[0][0][0] 863 except IndexError: c = None 864 if c == '@': silent = 1 865 elif c == '-': ignore = 1 866 else: break 867 result[0][0] = result[0][0][1:] 868 try: 869 if not result[0][0]: 870 result[0] = result[0][1:] 871 except IndexError: 872 pass 873 return result, ignore, silent

874

875 - def strfunction(self, target, source, env, executor=None):

876 if self.cmdstr is None: 877 return None 878 if self.cmdstr is not _null: 879 from SCons.Subst import SUBST_RAW 880 if executor: 881 c = env.subst(self.cmdstr, SUBST_RAW, executor=executor) 882 else: 883 c = env.subst(self.cmdstr, SUBST_RAW, target, source) 884 if c: 885 return c 886 cmd_list, ignore, silent = self.process(target, source, env, executor) 887 if silent: 888 return '' 889 return _string_from_cmd_list(cmd_list[0])

890

891 - def execute(self, target, source, env, executor=None):

892 """Execute a command action. 893 894 This will handle lists of commands as well as individual commands, 895 because construction variable substitution may turn a single 896 "command" into a list. This means that this class can actually 897 handle lists of commands, even though that's not how we use it 898 externally. 899 """ 900 escape_list = SCons.Subst.escape_list 901 flatten_sequence = SCons.Util.flatten_sequence 902 903 try: 904 shell = env['SHELL'] 905 except KeyError: 906 raise SCons.Errors.UserError('Missing SHELL construction variable.') 907 908 try: 909 spawn = env['SPAWN'] 910 except KeyError: 911 raise SCons.Errors.UserError('Missing SPAWN construction variable.') 912 else: 913 if is_String(spawn): 914 spawn = env.subst(spawn, raw=1, conv=lambda x: x) 915 916 escape = env.get('ESCAPE', lambda x: x) 917 918 ENV = get_default_ENV(env) 919 920 # Ensure that the ENV values are all strings: 921 for key, value in ENV.items(): 922 if not is_String(value): 923 if is_List(value): 924 # If the value is a list, then we assume it is a 925 # path list, because that's a pretty common list-like 926 # value to stick in an environment variable: 927 value = flatten_sequence(value) 928 ENV[key] = os.pathsep.join(map(str, value)) 929 else: 930 # If it isn't a string or a list, then we just coerce 931 # it to a string, which is the proper way to handle 932 # Dir and File instances and will produce something 933 # reasonable for just about everything else: 934 ENV[key] = str(value) 935 936 if executor: 937 target = executor.get_all_targets() 938 source = executor.get_all_sources() 939 cmd_list, ignore, silent = self.process(target, list(map(rfile, source)), env, executor) 940 941 # Use len() to filter out any "command" that's zero-length. 942 for cmd_line in filter(len, cmd_list): 943 # Escape the command line for the interpreter we are using. 944 cmd_line = escape_list(cmd_line, escape) 945 result = spawn(shell, escape, cmd_line[0], cmd_line, ENV) 946 if not ignore and result: 947 msg = "Error %s" % result 948 return SCons.Errors.BuildError(errstr=msg, 949 status=result, 950 action=self, 951 command=cmd_line) 952 return 0

953

954 - def get_presig(self, target, source, env, executor=None):

955 """Return the signature contents of this action's command line. 956 957 This strips $(-$) and everything in between the string, 958 since those parts don't affect signatures. 959 """ 960 from SCons.Subst import SUBST_SIG 961 cmd = self.cmd_list 962 if is_List(cmd): 963 cmd = ' '.join(map(str, cmd)) 964 else: 965 cmd = str(cmd) 966 if executor: 967 return env.subst_target_source(cmd, SUBST_SIG, executor=executor) 968 else: 969 return env.subst_target_source(cmd, SUBST_SIG, target, source)

970

971 - def get_implicit_deps(self, target, source, env, executor=None):

972 icd = env.get('IMPLICIT_COMMAND_DEPENDENCIES', True) 973 if is_String(icd) and icd[:1] == '$': 974 icd = env.subst(icd) 975 if not icd or icd in ('0', 'None'): 976 return [] 977 from SCons.Subst import SUBST_SIG 978 if executor: 979 cmd_list = env.subst_list(self.cmd_list, SUBST_SIG, executor=executor) 980 else: 981 cmd_list = env.subst_list(self.cmd_list, SUBST_SIG, target, source) 982 res = [] 983 for cmd_line in cmd_list: 984 if cmd_line: 985 d = str(cmd_line[0]) 986 m = strip_quotes.match(d) 987 if m: 988 d = m.group(1) 989 d = env.WhereIs(d) 990 if d: 991 res.append(env.fs.File(d)) 992 return res

993 994

995 -class CommandGeneratorAction(ActionBase):

996 """Class for command-generator actions."""

997 - def __init__(self, generator, kw):

998 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.CommandGeneratorAction') 999 self.generator = generator 1000 self.gen_kw = kw 1001 self.varlist = kw.get('varlist', ()) 1002 self.targets = kw.get('targets', '$TARGETS')

1003

1004 - def _generate(self, target, source, env, for_signature, executor=None):

1005 # ensure that target is a list, to make it easier to write 1006 # generator functions: 1007 if not is_List(target): 1008 target = [target] 1009 1010 if executor: 1011 target = executor.get_all_targets() 1012 source = executor.get_all_sources() 1013 ret = self.generator(target=target, 1014 source=source, 1015 env=env, 1016 for_signature=for_signature) 1017 gen_cmd = Action(ret, **self.gen_kw) 1018 if not gen_cmd: 1019 raise SCons.Errors.UserError("Object returned from command generator: %s cannot be used to create an Action." % repr(ret)) 1020 return gen_cmd

1021

1022 - def __str__(self):

1023 try: 1024 env = self.presub_env 1025 except AttributeError: 1026 env = None 1027 if env is None: 1028 env = SCons.Defaults.DefaultEnvironment() 1029 act = self._generate([], [], env, 1) 1030 return str(act)

1031

1032 - def batch_key(self, env, target, source):

1033 return self._generate(target, source, env, 1).batch_key(env, target, source)

1034

1035 - def genstring(self, target, source, env, executor=None):

1036 return self._generate(target, source, env, 1, executor).genstring(target, source, env)

1037

1038 - def __call__(self, target, source, env, exitstatfunc=_null, presub=_null, 1039 show=_null, execute=_null, chdir=_null, executor=None):

1040 act = self._generate(target, source, env, 0, executor) 1041 if act is None: 1042 raise SCons.Errors.UserError("While building `%s': " 1043 "Cannot deduce file extension from source files: %s" 1044 % (repr(list(map(str, target))), repr(list(map(str, source))))) 1045 return act(target, source, env, exitstatfunc, presub, 1046 show, execute, chdir, executor)

1047

1048 - def get_presig(self, target, source, env, executor=None):

1049 """Return the signature contents of this action's command line. 1050 1051 This strips $(-$) and everything in between the string, 1052 since those parts don't affect signatures. 1053 """ 1054 return self._generate(target, source, env, 1, executor).get_presig(target, source, env)

1055

1056 - def get_implicit_deps(self, target, source, env, executor=None):

1057 return self._generate(target, source, env, 1, executor).get_implicit_deps(target, source, env)

1058

1059 - def get_varlist(self, target, source, env, executor=None):

1060 return self._generate(target, source, env, 1, executor).get_varlist(target, source, env, executor)

1061

1062 - def get_targets(self, env, executor):

1063 return self._generate(None, None, env, 1, executor).get_targets(env, executor)

1064 1065

1066 -class LazyAction(CommandGeneratorAction, CommandAction):

1067 """ 1068 A LazyAction is a kind of hybrid generator and command action for 1069 strings of the form "$VAR". These strings normally expand to other 1070 strings (think "$CCCOM" to "$CC -c -o $TARGET $SOURCE"), but we also 1071 want to be able to replace them with functions in the construction 1072 environment. Consequently, we want lazy evaluation and creation of 1073 an Action in the case of the function, but that's overkill in the more 1074 normal case of expansion to other strings. 1075 1076 So we do this with a subclass that's both a generator *and* 1077 a command action. The overridden methods all do a quick check 1078 of the construction variable, and if it's a string we just call 1079 the corresponding CommandAction method to do the heavy lifting. 1080 If not, then we call the same-named CommandGeneratorAction method. 1081 The CommandGeneratorAction methods work by using the overridden 1082 _generate() method, that is, our own way of handling "generation" of 1083 an action based on what's in the construction variable. 1084 """ 1085

1086 - def __init__(self, var, kw):

1087 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.LazyAction') 1088 CommandAction.__init__(self, '${'+var+'}', **kw) 1089 self.var = SCons.Util.to_String(var) 1090 self.gen_kw = kw

1091

1092 - def get_parent_class(self, env):

1093 c = env.get(self.var) 1094 if is_String(c) and not '\n' in c: 1095 return CommandAction 1096 return CommandGeneratorAction

1097

1098 - def _generate_cache(self, env):

1099 if env: 1100 c = env.get(self.var, '') 1101 else: 1102 c = '' 1103 gen_cmd = Action(c, **self.gen_kw) 1104 if not gen_cmd: 1105 raise SCons.Errors.UserError("$%s value %s cannot be used to create an Action." % (self.var, repr(c))) 1106 return gen_cmd

1107

1108 - def _generate(self, target, source, env, for_signature, executor=None):

1109 return self._generate_cache(env)

1110

1111 - def __call__(self, target, source, env, *args, **kw):

1112 c = self.get_parent_class(env) 1113 return c.__call__(self, target, source, env, *args, **kw)

1114

1115 - def get_presig(self, target, source, env):

1116 c = self.get_parent_class(env) 1117 return c.get_presig(self, target, source, env)

1118

1119 - def get_varlist(self, target, source, env, executor=None):

1120 c = self.get_parent_class(env) 1121 return c.get_varlist(self, target, source, env, executor)

1122 1123

1124 -class FunctionAction(_ActionAction):

1125 """Class for Python function actions.""" 1126

1127 - def __init__(self, execfunction, kw):

1128 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.FunctionAction') 1129 1130 self.execfunction = execfunction 1131 try: 1132 self.funccontents = _callable_contents(execfunction) 1133 except AttributeError: 1134 try: 1135 # See if execfunction will do the heavy lifting for us. 1136 self.gc = execfunction.get_contents 1137 except AttributeError: 1138 # This is weird, just do the best we can. 1139 self.funccontents = _object_contents(execfunction) 1140 1141 _ActionAction.__init__(self, **kw)

1142

1143 - def function_name(self):

1144 try: 1145 return self.execfunction.__name__ 1146 except AttributeError: 1147 try: 1148 return self.execfunction.__class__.__name__ 1149 except AttributeError: 1150 return "unknown_python_function"

1151

1152 - def strfunction(self, target, source, env, executor=None):

1153 if self.cmdstr is None: 1154 return None 1155 if self.cmdstr is not _null: 1156 from SCons.Subst import SUBST_RAW 1157 if executor: 1158 c = env.subst(self.cmdstr, SUBST_RAW, executor=executor) 1159 else: 1160 c = env.subst(self.cmdstr, SUBST_RAW, target, source) 1161 if c: 1162 return c 1163 1164 def array(a): 1165 def quote(s): 1166 try: 1167 str_for_display = s.str_for_display 1168 except AttributeError: 1169 s = repr(s) 1170 else: 1171 s = str_for_display() 1172 return s

1173 return '[' + ", ".join(map(quote, a)) + ']'

1174 try: 1175 strfunc = self.execfunction.strfunction 1176 except AttributeError: 1177 pass 1178 else: 1179 if strfunc is None: 1180 return None 1181 if callable(strfunc): 1182 return strfunc(target, source, env) 1183 name = self.function_name() 1184 tstr = array(target) 1185 sstr = array(source) 1186 return "%s(%s, %s)" % (name, tstr, sstr) 1187

1188 - def __str__(self):

1189 name = self.function_name() 1190 if name == 'ActionCaller': 1191 return str(self.execfunction) 1192 return "%s(target, source, env)" % name

1193

1194 - def execute(self, target, source, env, executor=None):

1195 exc_info = (None,None,None) 1196 try: 1197 if executor: 1198 target = executor.get_all_targets() 1199 source = executor.get_all_sources() 1200 rsources = list(map(rfile, source)) 1201 try: 1202 result = self.execfunction(target=target, source=rsources, env=env) 1203 except KeyboardInterrupt as e: 1204 raise 1205 except SystemExit as e: 1206 raise 1207 except Exception as e: 1208 result = e 1209 exc_info = sys.exc_info() 1210 1211 if result: 1212 result = SCons.Errors.convert_to_BuildError(result, exc_info) 1213 result.node=target 1214 result.action=self 1215 try: 1216 result.command=self.strfunction(target, source, env, executor) 1217 except TypeError: 1218 result.command=self.strfunction(target, source, env) 1219 1220 # FIXME: This maintains backward compatibility with respect to 1221 # which type of exceptions were returned by raising an 1222 # exception and which ones were returned by value. It would 1223 # probably be best to always return them by value here, but 1224 # some codes do not check the return value of Actions and I do 1225 # not have the time to modify them at this point. 1226 if (exc_info[1] and 1227 not isinstance(exc_info[1],EnvironmentError)): 1228 raise result 1229 1230 return result 1231 finally: 1232 # Break the cycle between the traceback object and this 1233 # function stack frame. See the sys.exc_info() doc info for 1234 # more information about this issue. 1235 del exc_info

1236

1237 - def get_presig(self, target, source, env):

1238 """Return the signature contents of this callable action.""" 1239 try: 1240 return self.gc(target, source, env) 1241 except AttributeError: 1242 return self.funccontents

1243

1244 - def get_implicit_deps(self, target, source, env):

1245 return []

1246

1247 -class ListAction(ActionBase):

1248 """Class for lists of other actions."""

1249 - def __init__(self, actionlist):

1250 if SCons.Debug.track_instances: logInstanceCreation(self, 'Action.ListAction') 1251 def list_of_actions(x): 1252 if isinstance(x, ActionBase): 1253 return x 1254 return Action(x)

1255 self.list = list(map(list_of_actions, actionlist)) 1256 # our children will have had any varlist 1257 # applied; we don't need to do it again 1258 self.varlist = () 1259 self.targets = '$TARGETS'

1260

1261 - def genstring(self, target, source, env):

1262 return '\n'.join([a.genstring(target, source, env) for a in self.list])

1263

1264 - def __str__(self):

1265 return '\n'.join(map(str, self.list))

1266

1267 - def presub_lines(self, env):

1268 return SCons.Util.flatten_sequence( 1269 [a.presub_lines(env) for a in self.list])

1270

1271 - def get_presig(self, target, source, env):

1272 """Return the signature contents of this action list. 1273 1274 Simple concatenation of the signatures of the elements. 1275 """ 1276 return b"".join([bytes(x.get_contents(target, source, env)) for x in self.list])

1277

1278 - def __call__(self, target, source, env, exitstatfunc=_null, presub=_null, 1279 show=_null, execute=_null, chdir=_null, executor=None):

1280 if executor: 1281 target = executor.get_all_targets() 1282 source = executor.get_all_sources() 1283 for act in self.list: 1284 stat = act(target, source, env, exitstatfunc, presub, 1285 show, execute, chdir, executor) 1286 if stat: 1287 return stat 1288 return 0

1289

1290 - def get_implicit_deps(self, target, source, env):

1291 result = [] 1292 for act in self.list: 1293 result.extend(act.get_implicit_deps(target, source, env)) 1294 return result

1295

1296 - def get_varlist(self, target, source, env, executor=None):

1297 result = OrderedDict() 1298 for act in self.list: 1299 for var in act.get_varlist(target, source, env, executor): 1300 result[var] = True 1301 return list(result.keys())

1302 1303

1304 -class ActionCaller(object):

1305 """A class for delaying calling an Action function with specific 1306 (positional and keyword) arguments until the Action is actually 1307 executed. 1308 1309 This class looks to the rest of the world like a normal Action object, 1310 but what it's really doing is hanging on to the arguments until we 1311 have a target, source and env to use for the expansion. 1312 """

1313 - def __init__(self, parent, args, kw):

1314 self.parent = parent 1315 self.args = args 1316 self.kw = kw

1317

1318 - def get_contents(self, target, source, env):

1319 actfunc = self.parent.actfunc 1320 try: 1321 # "self.actfunc" is a function. 1322 contents = actfunc.__code__.co_code 1323 except AttributeError: 1324 # "self.actfunc" is a callable object. 1325 try: 1326 contents = actfunc.__call__.__func__.__code__.co_code 1327 except AttributeError: 1328 # No __call__() method, so it might be a builtin 1329 # or something like that. Do the best we can. 1330 contents = repr(actfunc) 1331 1332 return contents

1333

1334 - def subst(self, s, target, source, env):

1335 # If s is a list, recursively apply subst() 1336 # to every element in the list 1337 if is_List(s): 1338 result = [] 1339 for elem in s: 1340 result.append(self.subst(elem, target, source, env)) 1341 return self.parent.convert(result) 1342 1343 # Special-case hack: Let a custom function wrapped in an 1344 # ActionCaller get at the environment through which the action 1345 # was called by using this hard-coded value as a special return. 1346 if s == '$__env__': 1347 return env 1348 elif is_String(s): 1349 return env.subst(s, 1, target, source) 1350 return self.parent.convert(s)

1351

1352 - def subst_args(self, target, source, env):

1353 return [self.subst(x, target, source, env) for x in self.args]

1354

1355 - def subst_kw(self, target, source, env):

1356 kw = {} 1357 for key in list(self.kw.keys()): 1358 kw[key] = self.subst(self.kw[key], target, source, env) 1359 return kw

1360

1361 - def __call__(self, target, source, env, executor=None):

1362 args = self.subst_args(target, source, env) 1363 kw = self.subst_kw(target, source, env) 1364 return self.parent.actfunc(*args, **kw)

1365

1366 - def strfunction(self, target, source, env):

1367 args = self.subst_args(target, source, env) 1368 kw = self.subst_kw(target, source, env) 1369 return self.parent.strfunc(*args, **kw)

1370

1371 - def __str__(self):

1372 return self.parent.strfunc(*self.args, **self.kw)

1373 1374

1375 -class ActionFactory(object):

1376 """A factory class that will wrap up an arbitrary function 1377 as an SCons-executable Action object. 1378 1379 The real heavy lifting here is done by the ActionCaller class. 1380 We just collect the (positional and keyword) arguments that we're 1381 called with and give them to the ActionCaller object we create, 1382 so it can hang onto them until it needs them. 1383 """

1384 - def __init__(self, actfunc, strfunc, convert=lambda x: x):

1385 self.actfunc = actfunc 1386 self.strfunc = strfunc 1387 self.convert = convert

1388

1389 - def __call__(self, *args, **kw):

1390 ac = ActionCaller(self, args, kw) 1391 action = Action(ac, strfunction=ac.strfunction) 1392 return action

1393 1394 # Local Variables: 1395 # tab-width:4 1396 # indent-tabs-mode:nil 1397 # End: 1398 # vim: set expandtab tabstop=4 shiftwidth=4: 1399

Source Code for Module SCons.Action