Software/Geant4/UCL HEP Cluster

From PBTWiki
Jump to navigation Jump to search

Introduction

The UCL HEP Linux Cluster consists of a range of machines running Scientific Linux (CERN) 5 and Scientific Linux 6. This cluster provides a pre-compiled installation of Geant4 along with an environment for compiling code, running examples and viewing results.

Although there are over 100 individual machines that can be used — most of which are the individual Linux desktop PCs within the HEP group running SL6 — access from outside the HEP group subnet is only possible through the central Linux server, plus1, via ssh: 

$ ssh -Y username@plus1.hep.ucl.ac.uk

It is then possible to connect to individual Linux desktops, again by ssh. In order to do so, you will need to know the number of the PC you are connecting to: 

$ ssh -Y username@pc1XX.hep.ucl.ac.uk

...where 1XX is the number of the PC you wish to connect to.

In order to access these machines, you will need to obtain an account from the UCL HEP Group Computing Support. Please contact Dr. Simon Jolly for instructions on obtaining an account.

Folder structure

Disk

There is a separate (large) disk for PBT work mounted on the Linux system at /unix/pbt. When running simulations, you should create a subdirectory with your username at /unix/pbt/username

Software

All software is enclosed in a folder at /unix/pbt/software.

Source code

Source code for building and installing software packages is kept in a separate folder at /unix/pbt/software/src.

Environments

Two separate environments are provide for PBT simulations: the Production environment for general use and the Development environment for testing changes or updates to the code. The Production environment is further subdivided depending on whether the machine you are logged in to is running SLC5 or SL6: the SLC5 legacy environment features an older version of Geant4 and related software that will not be updated.

  • The recommended Production environment for SL6 is stored in /unix/pbt/software/prod-sl6;
  • The Development environment for SL6 is stored in /unix/pbt/software/dev: code is then subdivided into directories by application name;
  • The legacy Production environment for SLC5 is stored in /unix/pbt/software/prod: this is only accessible for users running on SLC5 machines.

It is possible to select whether the Development or Production environment is launched under SL6 by adding the appropriate flags when calling the setup script (see #Scripts below).

Scripts

Access to the PBT environments is implemented by a single script, pbt.sh. This calls a number of other utility scripts that set up the required PBT environment: these are all stored in /unix/pbt/software/scripts/. This will auto-detect whether you are running on a newer ContOS7 machine or a legacy SLC6 or SLC5 machine and set up the environment accordingly.

If you are running on a CentOS7 machine, we reccomand to set up all the required EasyBuild modules and environment variables sourcing the following script 

/unix/pbt/software/scripts/eb-centos7/pbt.sh

For the PBT Production and Development environments on SLC6 or SLC5 machines, you must source the followig 

/unix/pbt/software/scripts/pbt.sh

The steps carried out during setup are as follows:

  1. Check the environment and whether the user wishes to run in verbose or development mode (see below);
  2. Launch a new bash shell with the appropriate Scientific Linux Developer Toolset (see below);
  3. Call the appropriate bashrc script to begin environment setup; this in turn runs the following actions:
    1. Run the user's own .bashrc;
    2. Set the environment name;
    3. Greeting with information for users;
    4. Define exit function with exit message when user exits environment;
    5. Call shell script to set up environment.
  4. The specific shell script then carries out the following actions:
    1. Set specific environment variables with locations of compiler, Geant4 etc.
    2. Add locations to path variables and remove duplicates (calls setpaths.sh script);
    3. Define colours for scripts, prompt, etc (calls definecolours.sh script);
    4. Check versions of installed software and display results (calls checkversions.sh script);
    5. Modify prompt to show environment name;
    6. Run Geant4 setup script and set additional settings;

Adding the -v flag enables the scripts to run in verbose mode, which displays the full paths when checking the versions of the installed software. The -b flag sources the BDSIM PBT environment: this is only available on CentOS7 machines and using the script version available for EasyBuild modules (stored in /unix/pbt/software/scripts/eb-centos7). Adding the -d flag enables the Development environment to allow new software to be built without changing the Production setup needed by most users: this is only available on SL6 machines.

When pbt.sh is called, its first action after selecting the right environment is to launch a new bash shell that provides a degree of isolation for setting up the desired PBT environment. Once the user has finished with the PBT environment, exiting the shell will deposit the user back into their previous environment. Launching a new shell also provides access to the correct Scientific Linux Developer Toolset: on CentOS7 this is devtoolset-7, on SL6 this is devtoolset-3, whereas SLC5 uses the older devtoolset-2. In either case, the Developer Toolset is a Linux software collection providing various tools for developers, including more up-to-date versions of software required by Geant4, most notably including a newer version of GCC (the GNU Compiler Collection).

The following sub-directories are present within the scripts directory:

  • dev — contains scripts for setting up Development environment on SL6 machines;
  • prod-sl6 — contains scripts for setting up Production environment on SL6 machines;
  • prod-slc5 — contains scripts for setting up Production environment on SLC5 machines;
  • shared — shared scripts used in the setup of all environments (see below);
  • eb-centos7 — contains scripts for setting up PBT environment on CentOS7 machines using EasyBuild modules;
  • old — legacy scripts used in previous environment setup on SLC5.

The first 3 directories contain scripts that are almost identical, with small modifications to suit each environment. Along with the shared scripts, these are described below: substitute xxx for the particular environment. The eb-centos7 directory contains a version of all these scripts modified to work with EasyBuild modules.

/unix/pbt/software/scripts/xxx/pbt-xxx.sh

The 3 pbt-xxx.sh scripts carry out the majority of the customisation for the particular environment. In particular, the following tasks are carried out:

  1. Add locations to path variables;
  2. Define colours for scripts, prompt, etc.;
  3. Add locations to path variables and remove duplicates (calls setpaths.sh script);
  4. Check versions of installed software and display results (calls checkversions.sh script);
  5. Run Geant4 setup script;

These scripts in turn make use of the following shared scripts.

/unix/pbt/software/scripts/shared/definecolours.sh

This script defines various variables to represent colours for printing to the terminal and to file.

/unix/pbt/software/scripts/shared/setpaths.sh

This script sets the various PATH environment variables required for the specific environment and removes any duplicates.

/unix/pbt/software/scripts/shared/checkversions.sh

This script checks the versions (and optionally the paths/locations) of important software packages used in the PBT environment.

Installations

For each software package, the installation information is listed below. The basic method used is as follows:

  1. Download the source tarball latest version to the /unix/pbt/software/src folder.
  2. Extract the tarball to a subfolder within the src folder using tar -xvzf <filename>.tar.gz.
  3. Create a build folder at /unix/pbt/software/build and set the working directory here.
  4. Run the specific configure, build and install steps for the package.
  5. Remove the build folder and extracted source folder, but keep the source tarball.

The configure, build and install steps may be different for each package, and this is described per package below. However, in most cases the steps follow the general steps below:

  1. Change working directory to the build folder.
  2. Run the configuration script from the source folder to set the build options, including the customised installation prefix.
  3. Build the code.
  4. Install the code to the sandbox folder.
  5. (Optional) Move the compiled code to the desired location (only possible if the installation prefix cannot be specified).

Software Matrix

The following software is installed on the HEP Linux cluster.

CentOS7 Machines (not supported):

Software Version Source code Installed in dev? Installed by Working? Checked by Installed in prod? Installed by Working? Checked by
Qt 5.9.3 src/geant4-prerequisites/qt/qtbase-github Yes James Yes James Yes James Yes James
cmake 3.9.4 src/geant4-prerequisites/cmake/cmake-gitlab Yes Bootstrap (James) Yes James Yes Bootstrap (James) Yes James
xerces-c 3.2.0 src/geant4-prerequisites/xerces-c/xerces-c-github Yes James Yes James Yes James Yes James
CLHEP 2.3.4.5 src/geant4-prerequisites/clhep/clhep-github Yes James Yes James Yes James Yes James
simage 1.7.1a src/geant4-prerequisites/coin3d_mercurial/simage Yes James Yes James Yes James Yes James
Coin3d 4.0.0a src/geant4-prerequisites/coin3d_mercurial/coin-cmake Yes James Yes James Yes James Yes James
SoQt 1.6.0a src/geant4-prerequisites/coin3d_mercurial/soqt Yes James Yes James Yes - missing config file so says "not found" James Yes James
SoXt 1.3.1 src/geant4-prerequisites/coin3d_mercurial/soxt Yes James Yes James Yes James Yes James
Geant4 4.10.03.p02 src/geant4/geant4-github Yes James Yes James Yes James Yes James
DAWN 3.90b src/geant4-addons/dawn Yes James No James Yes (can copy from dev) James No Roisin
HepRApp 3.15.4 src/geant4-addons/heprapp Yes Simon Yes Simon (Standalone .jar file) Yes Simon Yes Simon (Standalone .jar file)
Java Analysis Studio 3.1.5 src/geant4-addons/jas Yes Simon Yes Simon (Standalone executables) Yes Simon Yes Simon (Standalone executables)
gMocren 4 src/geant4-addons/gmocren Yes Simon Unknown Simon (Standalone executables) Yes Simon Unknown Simon (Standalone executables)
BDSIM 0.9.9.3 src/bdsim/bdsim-edited Yes James Yes James No

SL6 Machines:

Software Version Source code Installed in dev? Installed by Working? Checked by Installed in prod? Installed by Working? Checked by
Qt 5.5.1 src/geant4-prerequisites/qt/qt-5.5.1 Yes Simon Yes Lawrence Yes Simon Yes Lawrence
cmake 3.5.2 src/geant4-prerequisites/cmake Yes Static binaries (Simon) Yes Lawrence Yes Static binaries (Simon) Yes Lawrence
xerces-c 3.1.3 src/geant4-prerequisites/xerces-c Yes Simon Yes Lawrence Yes Simon Yes Lawrence
CLHEP 2.3.1.1 src/geant4-prerequisites/clhep Yes Lawrence Yes Lawrence Yes Lawrence Yes Lawrence
simage 1.7.0 src/geant4-prerequisites/coin3d Yes Simon Yes Lawrence Yes Simon Yes Lawrence
Coin3d 3.1.3 src/geant4-prerequisites/coin3d Yes Simon Yes Lawrence Yes Simon Yes Lawrence
SoQt 1.5.0 src/geant4-prerequisites/coin3d Yes Simon Yes Lawrence No
SoXt 1.3.0 src/geant4-prerequisites/coin3d Yes Simon Yes Lawrence Yes Simon Yes Lawrence
Geant4 4.10.02.p01 src/geant4/geant4.10.02.p01-source Yes Lawrence Yes Lawrence Yes Lawrence Yes Lawrence
DAWN 3.90b src/geant4-addons/dawn Yes Simon Yes Simon Yes (can copy from dev) Simon Yes Roisin
HepRApp 3.15.4 src/geant4-addons/heprapp Yes Simon Yes Simon (Standalone .jar file) Yes Simon Yes Simon (Standalone .jar file)
Java Analysis Studio 3.1.5 src/geant4-addons/jas Yes Simon Yes Simon (Standalone executables) Yes Simon Yes Simon (Standalone executables)
gMocren 4 src/geant4-addons/gmocren Yes Simon Unknown Simon (Standalone executables) Yes Simon Unknown Simon (Standalone executables)
BDSIM 0.9.9.3 src/bdsim/bdsim-edited Yes James Yes James No
Retrieved from "https://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Software/Geant4/UCL_HEP_Cluster&oldid=2363"