*** This Page is Outdated and is Slatted for Retirement... ***

Building and Running with the Accelerator Physics Code Libraries at CLASSE

An Overview

The Accelerator Physics code that is used to build AP libraries and executables is kept under version control in a source code repository. There are selected individuals and groups of developers that are responsible for the primary development and maintenance of their respective libraries and/or applications. The developers that are directly responsible for AP library development will be called primary developers. Specific affiliations will be detailed here in the future.

Each morning at around 3:45, an automated build system checks out the latest versions of the code files for all the libraries and applications that comprise an 'accelerator libraries build' and compiles them all to produce various binary entities native to the supported selection of computing platforms on-site.

• The products of a build process are:
  • Lots of static library files (.a extension)
  • Lots of binary object files (.o extension)
  • Lots of fortran-90 module files (.mod extension)
  • Executables meant for data collection, analysis, and simulation
  • A world readable/writable archive of important standard accelerator lattice and layout files.

Those libraries and apps are thus meant to be linked against and/or used "out-of-the-box", i.e. without having to deal with source code or compilation.

• The supported computing platforms are, as of 7-Feb-2007:
  • Linux on i686
  • OSF1 Unix on DEC-alpha
  • ( Slated for eventual inclusion ) Windows (via the CygWin environment)

Periodically, when a reasonable number of changes have been comitted to the repository in the course of normal development and maintenance work on the part of the developers, a particular nightly build will be promoted to a DEVELopment build. This process varies in frequency from once a month or so, up to several times a week depending on how heavy development activity is.

Once a DEVEL release has proven its stability it may be promoted to CURRENT. The CURRENT release is considered stable and is what all critical secondary developent uses for building applications. The tradeoff is that CURRENT will perhaps lag behind subsequent DEVEL releases in terms of features and polish, but should be nominally bug-free. These promotions to DEVEL and CURRENT are called rotations and upon each rotation a notification is sent out to the acc-code-lepp lab-managed Hypernews mailing list.

Additionally, to facilitate the recreation of a given release if necessary and to aid primary developers in indexing snapshots of their code over time, upon each rotation the collection of files used to build a given release will receive a tag in the repository. This simply means that a meaningful bit of text can be used to obtain the snapshot of files used to produce a release from that point on instead of a cryptic version number or numbers. This feature is useful mainly for primary developers.

These pre-compiled releases are archived for a period of time, typically more than six months, and the libraries, objects, and modules contained within can be linked against by programmers wanting to incorporate the functionality they provide into their own work. These developers will be called secondary developers. A subclass of secondary developers might include individuals that do not work under the direct purview of the lab and will be called off-site developers. Convenient support for those wishing to obtain code from off-site is still pending.

Quick Start Guide to Using the AP Code Libraries

Environment Setup

For any development work, primary and secondary, that requires linkages to precompiled binaries a mechanism exists allowing you to select a specific version of the AP libraries to link against, in addition to specifying a compiler compatible with the version you choose, so that your code will properly link against the libraries. Please consult the information in DevelopmentEnvironmentSetup to see how you may use this mechanism to select various versions

Supported Languages

The languages supported for automatic dependency generation by the ACC Build System during library and application development:

• C
• C++
• FORTRAN 77/90/95

The CLASSE Hierarchical Build Scheme

Build Overview

The ACC build system attempts to automate a number of key functions for the user:

• Provide standard compilation rules for compiling production and debug versions of routines
• Provide a set of standard linking rules for the creation of production and debug versions of executables
• Provide automatic code dependency generation to help insure that complicated sets of codes are built consistently and are fully up-to-date
• Provide automatic search paths for compiling against ACC include files and F90/F95 modules
• Provide automatic search paths for linking against ACC libraries
• Provide simple and flexible methods to configure what versions of ACC code a user builds against
  • Standard builds using the current ACC release which contains codes that we feel to be fairly stable and robust
  • Developer builds using the devel ACC release which contains the most recent codes needed by developers
  • Experimental builds using locally customized versions of ACC codes
• Provide a set of standard ACC executables that are directly available to users
• Support for the above functionality over multiple platforms and operating systems

These capabilities are provided by a set of tools based on GNU Make makefiles and PERL scripts.

Build Hierarchy

The ACC build tools implement a precedence order when searching for header files and F90/F95 module files during compilation, and for libraries during the linking phase. The diagram below provides a quick overview of this hierarchy. In short, the highest precedence is given to versions in the local user area, then to an optional group area which may be maintained by multiple developers working on a single project, then to the officially maintained ACC and 3rd-party Packages areas, and finally to the compiler and system areas. Specific details of the search path structure are described below.

---

Set Up a Build Area

The ACC build tools provide support for users to build their local codes and an executable from a job directory. Relative to the job directory, binary outputs will be placed in a set of directories at the same level (executables in ../bin, F90/F95 modules in ../modules, and library output in ../lib). Thus you may want to create a code development area in which you place the job directory. Thus your area might look like:

                     development
                          |
          +----------+----------+----------+
          |          |          |          |  
         job        bin      modules      lib

In order to set up your job directory, you need to do the following:

• Create a directory called job.
• Copy your source code to this directory.
• Copy ${ACC_GMAKE}/M.job to this directory and rename it as Makefile.
• Edit Makefileso that any ACC libraries your codes require are in the library search list:
  • Near the end of Makefile you will find a variable called CESR_LIBS which contains a space-separated listof CESR libraries which you want to link against. The default M.job makefile, which is intended to link a basic BMAD executable, has the following libraries in the list:
    • bmad
    • dcslib
    • recipes_f-90_LEPP
    • cesr_utils
  • You may need these libraries or other libraries that reside in the ${ACC_LIB} area.
  • On the next line you will find a variable called PKG_LIBS which contains a list of 3rd party libraries (maintained as part of the ACC PACKAGES area) that are needed to build a simple BMAD executable (the default example places the forest library in this variable). You may need other libraries from the ${PKG_LIBS} area or none at all.
  • You should replace the libraries on the CESR_LIBS and the PKG_LIBS lines with whichever CESR and PACKAGES libraries that your executable requires.
  • 2 additional library lists are also present, LOCAL_LIBS and CERN_LIBS. The LOCAL_LIBS variable is intended to contain a list of your special libraries while CERN_LIBS allows you to link against the standard CERN software libraries as maintained on LEPP computers.
• At this point you are ready to start building an application.

Build Your Code

The Standard Job Makefile

In order to build an executable from the code in your job area, you simply need to type the command gmake to run GNU Make. The CESR job makefile will automatically search for all local fortran, C and C++ routines, build them, and link an executable which it will, by default, place in a bin directory which is at the same level as the job directory. By default, linking takes place with the fortran compiler. If you have a C or C++ main and need to link with the C/C++ compiler, some further customization is required (please send email to the CESR Librarian to obtain help).

Search Paths and Precedence

Include Files

The search hierarchy for include files that are included by means of the C preprocessor #include are as follows (please note the special search rules described in the following section):

1. Current directory
2. Directories in the local user area (starting one directory above the current directory).
3. An area specified by use of the EXTRA_INC_DIRS variable.
4. Specified directories that are part of the ACC Release
5. Specified directories that are part of the 3rd-party PACKAGES
6. Specified directories that are part of the CERN areas
7. Compiler and system include areas

F90/F95 Module Files

The search hierarchy for F90/F95 modules is as follows:

1. Current directory
2. Local modules directory (typically ../modules)
3. An area specified by the use of the EXTRA_MOD_DIRS variable.
4. The ${ACC_MOD} directory
5. The ${PKG_MOD} directory

Libraries

1. locallib area (typically ../lib)
2. EXTRA_LIB area (optionally specified by the user. NOTE: The EXTRA variables are intended to allow building against a group development area that takes precedence of the standard CESR release)
3. ACC_LIB area (library area for currently specified CESR release)
4. PKG_LIB area (library area for outside CESR packages)
5. CERN_LIB area (CERN libraries as maintained by the CLEO Collaboration on LEPP computers)

Customizing Your Job Makefile

There are a number of variables that can be set in the job makefile for easy customization. For variables that contain lists, the lists should be delimited by simple spaces.

Variable Name	Default Value	Description
OBJ_SRC_DIRS	.	A list of directories to be searched for source code.
CONFIG_DIRS		List of directories within the project that hold configuration and other runtime data. Directories specified here (dir1 dir2 ...) will also be copied automatically to the $ACC_REL/config/[project]/[dir1], $ACC_REL/config/[project]/[dir2]
LOCAL_INCS	include	A list of directories to be searched for include files. These directories can be specified as relative paths (in which case they appear immediately after the local directory in the search order), as absolute paths (not recommended, but also searched immediately after the local directory in the search order), or as partial paths. Partial paths are a special feature of the CESR library structure where arguments of #include directives can be provided without full paths attached. Partial path specifications placed in this makefile variable (eg, CesrBPM/include) are applied relative to directories contained within an accelerator libraries (ACC) release. Partial path specifications work for referencing include file locations found underneath the following directories: .., ${ACC_SRC}, ${CESR_PKG}, and the ${CERN} (/nfs/cern).
LOCAL_MODS		A list of directories to be searched for F90/F95 module files.
MAIN_FILE		If you like to keep multiple test program/main files in a single directory, make needs to know which one you want to link. Default operation assumes only one program/main file per job directory.
LOCAL_OBJS		A list of local (pre-compiled) object files to link into the job executable.
CESR_OBJS		A list of CESR (pre-compiled) object files to link into the job executable.
PKG_OBJS		A list of 3rd-party PACKAGE (pre-compiled) object files to link into the job executable.
LOCAL_FFLAGS		User-specified fortran compiler flags.
LOCAL_CFLAGS		User-specified C compiler flags.
LOCAL_CXXFLAGS		User-specified C++ compiler flags.
LOCAL_LFLAGS		User-specified linker flags. NOTE: If you are building code that does NOT use a Fortran main routine, the linker flag -nofor-main must be present in this variable otherwise you will get linker errors. Since the Intel Fortran toolchain is used for linking regardless of language for builds performed with the CESR build system, the following applies. This option specifies that the main program is not written in Fortran. It is a link-time option that prevents the compiler from linking for_main.o into applications. For example, if the main program is written in C and calls a Fortran subprogram, specify -nofor-main when compiling the program with the ifort command. If you omit this option, the main program must be a Fortran program.
LOCAL_LIBS		A list of user-supplied libraries to link against.
CESR_LIBS	bmad dcslib recipes_f-90_LEPP cesr_utils	A list of CESR libraries to link against.
PKG_LIBS	forest	A list of 3rd-party packages to link against.
CERN_LIBS		A list of CERN libraries to link against.
SYM_UNDEFS		A list of symbol names which need to be force loaded during linking.
BUILD_EXE	Y	Flag indicating whether an executable should be built.
LINK_COMPILER		Flag indicating which compiler should be used for the linking phase. (Blank means the f90 compiler will be used)

TIP:

To allow access to routines found in libraries upon which one's custom program depends, one must:

• Add a reference to the include directory containing headers mentioned in the source code under development
• Add a reference to the library AND program ("M.lib" & "M.job") makefiles to the external library that contains routines the software you are developing requires

For example, to build software that depends on log_message (found in libc_utils)

• Add include/c_utils to LOCAL_INCS, since c_utils is part of the standard pre-build libraries release and a relative path like that is fine.
• Add c_utils to CESR_LIBS in the correct spot in the linking order. Lower-level libraries are further right. Higher level ones (those that depends on the low-level ones) to the left.

Adding Your Own Local Libraries

Testing ACC Library Modifications Locally

Any ACC library source directory can be placed in your development area (parallel to your job directory) and built locally. Since this is in the local user's area, this copy will take precedence over any copy in the official release.

Recommended Makefile Practices

• For any code that lives in a project (directory) that will be part of the nightly builds (a list of repository directories under /trunk/src/... that WILL be part of every nightly release build can be found * here*), makefiles must adhere to the following:
  • The directory /src/ must contain a file called 'Makefile' and it must conform to GNU make syntax.
  • For each subdirectory, i.e. /src// that holds code to be built into separate .o files or executables, there should be an M. file in /src/.
  • If you wish to build multiple makefiles from the project-level "Makefile" reference the necessary "M." files in the "M_FILE_LIST :=" variable and ensure that the value of the DO_EXTRA_MAKES variable in your environment is set before initiating the build with a 'gmake' command.

Recommended Coding Practices

• Please document your code as thoroughly as possible using some approximation of a standard style across all the code in a given project
  • Especially any code that might not be obvious in its intent to those who have not been staring at your code for the last X hours, weeks, etc...
  • Include broad overviews of the purpose or ontology of an entire library, application, or project in order to place the code in context.
• More practices to follow...

Job Makefiles

Job makefiles can be copied into a user's directory in order to link an executable and to build any local routines provided by the user. An example job makefile, M.job, is maintained in the ${ACC_GMAKE} area.

---

Please contact the CESR_Code_Librarian with questions or problems about this page.