ReleaseHOWTO/TipCherryPickBody

Assumed Setup

These initializations are the conventions that will be used in the rest of this flow.

The location of the SCons SVN archive:

  $ export HG=https://bitbucket.org/scons/scons

The version string, which should look something like this:

From within your base directory, execute this command:

  $ svn co $SVN/checkpoint

FIXME This segment is overkill for anyone who actually uses SVN. Cut it or drop it.

A conflict occurs when SVN tries to apply changes from the archive that overlap with other changes made locally. These are rare but can happen when cherry-picking applies changes out of order. In general, the correct approach is to use the text from the archive rather than the local version, but use your judgment when making the changes as there will be some cases where the original text (or even a merger of the two texts) is the correct choice as SVN is not always accurate when marking the conflicted regions.

Finding conflicts

To determine if there're conflicts, run this command:

  $ svn status
  ?      test/example.py.r4941
  ?      test/example.py.mine
  ?      test/example.py.r4649
  C      test/example.py
  ?      src/engine/SCons/CoreFile.py.r4941
  ?      src/engine/SCons/CoreFile.py.mine
  ?      src/engine/SCons/CoreFile.py.r4649
  C      src/engine/SCons/CoreFile.py
  $

Lines that begin with "C" identify a file that's conflicted. There are also a number of related files provided by SVN that can be used in the process of resolution, identified by lines that begin with "?". This is a simple resolution, so we won't use them.

Removing Conflicts

To remove the conflicts from a file, follow these steps:

• Locate a conflict marker in the text. The center of each conflicted region is marked with "=======" so searching for a line with only that on it will get you to the right place.

• The start of the region is marked with "<<<<<<< .mine" and contains the local text. Delete this text as well as the start and center markers.

• The end of the region is marked with something that looks like ">>>>>>> .r54321" and contains the text from the SVN archive. Delete the end marker.

• Repeat until there are no more conflict markers.

To see how your updated file compares with the archive, run this command:

  $ svn diff test/example.py

If the changes aren't satisfactory, repeat the editing step.

Once the editing is done, use this command to tell SVN that the conflict has been resolved:

  $ svn resolved test/example.py

As a side-effect, resolving the conflict will remove the related files provided by SVN.

Repeat this section for each conflicted file.

$ python bin/update-release-info.py release

The ReleaseConfig file is where the "official" version number ($VERSION), the Python version floors, and other information about the release is recorded. This command takes the information in ReleaseConfig and inserts it in the necessary files.

  $ svn commit -F log.file
  Sending        some/file
  Sending        some/other/file
  Transmitting file data ..
  Committed revision 54321.
  $

Build Candidate Packages

Provided you have all of the necessary utilities installed, this should be a simple matter of:

  $ rm -rf build bootstrap
  $ python bootstrap.py

While you're waiting for the candidate packages to be built, open another shell window and proceed with preparing the test script. Make sure that $SVN and $VERSION are set up as shell variables as described above.

Test

Prepare test script

The script to run the tests is in the source in bin/fill_me_in. Make a copy and edit it according to the instructions below.

  # report skipped tests as successful
  export TESTCOMMON_PASS_SKIPS=yes
  set -e       # stop on error
  #
  python runtest.py -a
  python runtest.py -a -p tar-gz
  python runtest.py -a -p zip
  python runtest.py -a -p local-tar-gz
  python runtest.py -a -p local-zip
  python runtest.py -a -p src-tar-gz
  python runtest.py -a -p src-zip
  python runtest.py -a -p rpm

The regression suite exercises a wide range of functionality. There are around a thousand tests in the suite, so each test run can take a while.

Regression tests of source tree

Normally, the SCons BuildBot monitors this branch, so it probably started running regression tests as soon as you checked in the changes above. If your machine is covered by one of the BuildBot machines (particularly if your machine is one of the BuildBot machines), you can remove the first runtest in the script.

Regression tests of candidate packages

The build step above not only builds the packages but also unpacks all of them into subdirectories so that you can run the test suite against the packaged goods. This is intended to catch packaging problems such as not adding a new module to the packaging MANIFEST list(s). The runtest.py script supports a -p option and arguments that run the SCons tests against the different unpacked directories.

Edit the script to include the tests you want to include. If you want to be complete, test all of the packages.

To be quicker but still reasonably thorough, test tar-gz and zip, one each of local- and src- (probably do -tar-gz for one and -zip for the other), and rpm.

For a quick-n-dirty test, just test tar-gz or zip, and maybe rpm. Since all of the different packages use the same lists as input, it's pretty unlikely that the tests will pass on one package and fail for another.

Once the script is prepared, save it and return to the window creating the packages.

Check the Build

The build creates the packages in the build/dist subdirectory. If everything built correctly, you should see files that look like the following:

  scons-$VERSION-1.noarch.rpm
  scons-$VERSION-1.src.rpm
  scons-$VERSION.linux-x86_64.tar.gz
  scons-$VERSION.linux-x86_64.zip
  scons-$VERSION.tar.gz
  scons-$VERSION.win32.exe
  scons-$VERSION.zip
  scons-doc-$VERSION.tar.gz
  scons-local-$VERSION.tar.gz
  scons-local-$VERSION.zip
  scons-src-$VERSION.tar.gz
  scons-src-$VERSION.zip

Note that the linux-x86_64 strings in some of the file names above may be different, depending on your local system architecture. That doesn't matter; those files don't go public. Go ahead and remove them.

Run Test Script

FIXME

If the build succeeded, run the test script:

  $ sh path/to/copy-of-script

The tests can run a very long time. While you're waiting, go back to your other shell window and prepare what you can on the assumption that the tests will succeed.

Download SCons Home

From within your base directory, run these commands:

  $ svn co $SVN/scons.org
  $ cd scons.org

The scons.org checkout is pretty large. If you don't want to wait, open another shell window and proceed. Make sure the new shell is set up with the SVN and VERSION shell variables as described above.

Prepare Blurb

Prepare the blurb by starting with a copy of build/scons/RELEASE.txt. It should be pretty much good-to-go, but look over it and see if there are any changes that should be made to make it more suitable as a blurb rather than release notes in a distribution. In general, use your own judgment as to what to keep and what to cut.

FIXME HTML copy of blurb by inserting <br/><br/><br/> instead of double blank lines, <br/><br/> instead of single blank lines, and <br/> in front of items in a list. Not perfect, but more readable.

FIXME Source of beta and final blurbs are different.

Prepare Tigris Announcement

Note that these files must be checked in to the trunk for them to show up on our tigris.org project pages; you can not update and commit these files from the directory in which you built the packages.

From within the www subdirectory of your trunk directory, edit these files:

Prepare scons.org

If you opened another window to do the prep above, move back to the window downloading scons.org; it should be done by now. If you didn't, change back to the scons.org subdirectory of your base directory.

Web site

Make the following changes to the following files: FIXME The destination names aren't CHANGES.txt or RELEASE.txt and the explanation needs to be tweaked. And the source for RELEASE.txt could be either RELEASE.txt or Announce.txt.

Install documentation

To unpack the release documentation into where it can be used by the web pages, just run this script:

FIXME When this script is checked in, it will allow arguments on the command line
  $ sh bin/FillMeIn [ $VERSION [ $SVN ] ]
or something similar. In the meantime, copy the script and run it.

  mkdir doc/$VERSION
  (cd doc/$VERSION && tar zxf build/dist/scons-doc-$RELEASE.tar.gz)
  svn add doc/$VERSION
  (cd doc && rm -f latest && ln -s $VERSION latest)
  case $VERSION in *.0.final.*)
      # This is a 'final' release on the main branch
      (cd doc && rm -f production && ln -s $VERSION production)
  esac

FIXME If any of the candidate packages fail, fix the problem in trunk following your usual development practices and go and do another dry run.

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run this command:

  $ svn cp . $SVN/tags/$VERSION -m"Tagging $VERSION"

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/FillMeIn script:

  set -e   # stop on error
  rm -rf ARCHIVE
  svn co --depth=files $SVN/ARCHIVE
  mkdir ARCHIVE/scons-$VERSION
  cp build/dist/* ARCHIVE/scons-$VERSION
  svn add ARCHIVE/scons-$VERSION
  svn commit -m"Saving copy of $VERSION in archive" ARCHIVE
  rm -rf ARCHIVE

Upload packages

Verify that you have SVN and VERSION in your shell environment as described in the setup section above, then run the bin/scp-sourceforge script:

  $ sh bin/scp-sourceforge sf_username 

Mark packages with release notes

Navigate to the SourceForge File Manager page for SCons. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

• Log in to your SourceForge account

• Click on Develop just under Forge in the SourceForge banner.

• Under My Projects, click on Develop next to the SCons project.

• Click on Project Admin -> File Manager

Open the scons, scons-src, and scons-local directories. Within each one, open the $VERSION folder. In the scons/$VERSION directory, select the RELEASE.txt file and mark mark it as the release notes in the popup that appears, then save.

If this is a final release of the production branch, select scons-$VERSION.win32.exe and mark it as the default selection for Windows, then select scons-$VERSION.tar.gz and mark it as the default for all other OSes.

In turn, select each of the files (other than the release notes file itself, which is automatically set) in all three folders (and the folders themselves) and specify that RELEASE.txt is to be the release note for that file. Because of the way the page refreshes after clicking Save, it seems to be easier to start at the bottom and work your way up.

FIXME: TEST THIS: You can do multiple releases quickly by opening the File Manager page multiple times in a separate tabs, but if you do, make sure to wait for the page to reload completely in one tab before clicking Save in another. Trying to update multiple releases at once doesn't work, presumably because the session can only handle one update at a time.

Hide previous releases

• For a checkpoint release, hide all checkpoints in the same series that are older than this checkpoint.
• For a minor or micro release, hide all the checkpoint releases leading up to this release.
• For a major release, hide all the checkpoint releases leading up this release AND hide all major and minor releases older than this release.

For each release name you wish to hide, you have to do the following in scons, scons-local, and scons-src:

• Click on the gear icon to the left of the package name and select Cut from the popup.

• Click on the gear icon to the left of old checkpoints and select Paste from the popup.

Test downloading

Navigate to the SCons download page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

• FIXME Add instructions if can't click.

For each of the scons, scons-src, and scons-local directories:

• Open the directory.
• Open the $VERSION folder within the directory.
• For each package in the folder, click on the link to download it.
• Verify that the files arrived and look reasonable.

Add news item

Navigate to the Project News page. If you're reading this from a printed copy rather than the wiki page and can't click on the link, follow these instructions:

• FIXME These aren't complete.

• Go to the Project Admin -> Feature Settings page

• In Project News, click Submit.

Once you get to the Project News page:

• Fill in the Subject: box with something like "Release $VERSION now available"

• Cut-and-paste the blurb you prepared above into the Details: box

• Click Submit