Administrative Tools for use by the Code Librarian

A substantial set of tools has been developed and refined to make administering the ACC Software Archive as painless as possible. The administrative duties of the librarian include:

• Maintaining reliable release diagnostic builds known as NIGHTLIES
• Ensuring that successfully built and tested nightly builds are promoted to DEVELopment status at regular intervals.
• Ensuring that once a particular DEVELopment build has been tested thoroughly in day-to-day use, that it is promoted to CURRENT (stable) status.
• Answering general queries about difficulties users or developers may have when interacting with the repository, build system, or the software applications.

In general, all but the last point in the above list has been engineered to be handled automatically by the tools described below. The list of those tools and a brief description of their function follows. All of these tools are kept under revision control in the /trunk/util area of the ACC Subversion repository and have historically been given a home in /home/cesrulib/bin/util for daily use.

Front-Line Programs

Name	Function	Usage
build_master	Controls all release build scenarios. This is responsible for producing the nightlies, bmad distributions, packages releases, and fixes builds	Running with no arguments gives a detailed summary of all options available.
rotate_release	Used to promote a given release to a higher development tier. i.e. NIGHTLY --> DEVEL or DEVEL --> CURRENT	Runing with no arguments prints help
DELETE_release	Deletes an entire release from all supported platforms along with log files.	It has several interlocks and safety checks to prevent accidental deletion of a release. Even so, please think before you act.
tag_release	Creates a new directory under the /tags area of the ACC repository with an appropriate name, copies a snapshot of all files used in the creation of a release build there and applies a descriptive log entry.	.

do_release_vms.com

OpenVMS DCL script used to build a release for the control system. Much more primitive and less generalized than the build_master script above due to limitations of DCL and various other shortcomings of the VMS environment and of the time that would be required to generalize this mechanism.

Script header contains usage notes.

Others

These tools may be run on their own, but are typically used as a behind-the-scenes component of the build system to guide its operation or provide extra information in status reports.

Name	Function	Usage
CESRbuild_test_suite	Runs basic functionality testing on the standard applications produced as components of a release build.
extract_test_summary	Prints a PASS/FAIL summary of the results of the test suite program mentioned above.
rel_disk_total	Calculates the disk space consumed by the release specified.
cesr_to_vms	Produces a .tar.gz archive of a specified source release directory and sends it via FTP to the VMS cluster to facilitate building a release there. Primarily used from within the main build system scripts.

Config Files

Name	Function	Usage
BUILD_SYSTEM.conf	Master configuration file used to customize the operation of many of the scripts mentioned here. Controls the DEVEL rotation attempt schedule.	See file for syntax.
TEST_SUITE.conf	Configuration file that specifies the names of all the standard applications that will be tested during a test suite run and operational parameters that control the execution of each test.	See file for additional documentation and syntax.

Behind-the-scenes Programs

These never need to be run from the command line and serve as background machinery to facilitate the operation of other programs listed on this page. Some of these are not strictly administrative programs, but are under the direct purview of the librarian and their use should be understood to an equal or greater extent than the other programs mentioned on this page.

Name	Function	Usage
build_serf	Support script invoked by build_master to marshal the build process on remote machines.	This is not meant to be invoked by the user.
cesrmakedepend	The custom dependency-generation engine that facilitates the achievement of the design behavior of the ACC build system. It is called from within M.tail and M.release when building user code or when building an entire release. It is unfortunately very poorly documented and is written in Perl consisting of some non-obvious bits. It should be documented in much more detail.
create_fdb.pl	Perl script used to create a "database" file of standardized syntax that serves as a catalog of all the functions callable from within the various ACC object code and the files within which they are defined.
fixbuilder	Used to produce a FIXES area module rebuild at developer request.	Runs once a minute on lnx209 to check the status of the "control" directory in the fixes area.
mkstatreports	Generates HTML statistics reports on the repository's history data.	Runs automatically for all-time stats and stats-since-CURRENT as part of the build_master process.
getcfg	Python script that allows several flexible ways to extract useful information from the BUILD_SYSTEM.conf configuration file mentioned in the section above.	Usage: getcfg <section name> -- Lists all variables in that section. getcfg <section name> <variable name> -- Prints the value of a specific variable in a given section. Primarily used inside other scripts that need to automatically configure themselves from a central location.

Third-Party Support Scripts / Modules

The directory /util/3rd-party contains at the moment a couple of third-party Python modules that are used in the administrative programs to provide necessary functionality.

Name	Description
pexpect-2.1	A pure Python expect-like module for advanced subprocess communication and control.
standout	Python module that allows for highly-configurable logging, output redirection and formatting options within scripts.

---

-- MattRendina - 12 Sep 2007