Software/Geant4/UCL HEP Cluster: Difference between revisions
SimonJolly (talk | contribs) No edit summary |
|||
| (9 intermediate revisions by one other user not shown) | |||
| Line 75: | Line 75: | ||
## Set specific environment variables with locations of compiler, Geant4 etc. | ## Set specific environment variables with locations of compiler, Geant4 etc. | ||
## Add locations to path variables and remove duplicates (calls <code>setpaths.sh</code> script); | ## Add locations to path variables and remove duplicates (calls <code>setpaths.sh</code> script); | ||
## Define colours for scripts, prompt, etc (calls <code>definecolours.sh<code> script); | ## Define colours for scripts, prompt, etc (calls <code>definecolours.sh</code> script); | ||
## Check versions of installed software and display results (calls <code>checkversions.sh</code> script); | ## Check versions of installed software and display results (calls <code>checkversions.sh</code> script); | ||
## Modify prompt to show environment name; | ## Modify prompt to show environment name; | ||
| Line 81: | Line 81: | ||
Adding the <code>-v</code> flag enables the scripts to run in verbose mode, which displays the full paths when checking the versions of the installed software. | Adding the <code>-v</code> flag enables the scripts to run in verbose mode, which displays the full paths when checking the versions of the installed software. | ||
The <code>-b<code> flag sources the BDSIM PBT environment: this is only available on CentOS7 machines and using the script version available for EasyBuild modules (stored in <code>/unix/pbt/software/scripts/eb-centos7</code>). | The <code>-b</code> flag sources the BDSIM PBT environment: this is only available on CentOS7 machines and using the script version available for EasyBuild modules (stored in <code>/unix/pbt/software/scripts/eb-centos7</code>). | ||
Adding the <code>-d</code> flag enables the '''Development''' environment to allow new software to be built without changing the '''Production''' setup needed by most users: this is only available on SL6 machines. | Adding the <code>-d</code> flag enables the '''Development''' environment to allow new software to be built without changing the '''Production''' setup needed by most users: this is only available on SL6 machines. | ||
When <code>pbt.sh</code> is called, its first action after selecting the right environment is to launch a new <code>bash</code> shell that provides a degree of isolation for setting up the desired PBT environment. | When <code>pbt.sh</code> is called, its first action after selecting the right environment is to launch a new <code>bash</code> shell that provides a degree of isolation for setting up the desired PBT environment. | ||
Once the user has finished with the PBT environment, exiting the shell will deposit the user back into their previous environment. | Once the user has finished with the PBT environment, exiting the shell will deposit the user back into their previous environment. | ||
Launching a new shell also provides access to the correct Scientific Linux Developer Toolset: on SL6 this is <code>devtoolset-3</code>, whereas SLC5 uses the older <code>devtoolset-2</code>. | Launching a new shell also provides access to the correct Scientific Linux Developer Toolset: on CentOS7 this is <code>devtoolset-7</code>, on SL6 this is <code>devtoolset-3</code>, whereas SLC5 uses the older <code>devtoolset-2</code>. | ||
In either case, the Developer Toolset is a Linux software collection providing various tools for developers, including more up-to-date versions of software required by <code>Geant4</code>, most notably including a newer version of <code>GCC</code> (the '''GNU Compiler Collection'''). | In either case, the Developer Toolset is a Linux software collection providing various tools for developers, including more up-to-date versions of software required by <code>Geant4</code>, most notably including a newer version of <code>GCC</code> (the '''GNU Compiler Collection'''). | ||
The following sub-directories are present within the <code>scripts</code> directory: | The following sub-directories are present within the <code>scripts</code> directory: | ||
* <code>dev</code> — contains scripts for setting up '''Development''' environment; | * <code>dev</code> — contains scripts for setting up '''Development''' environment on SL6 machines; | ||
* <code>prod-sl6</code> — contains scripts for setting up '''Production''' environment on SL6 machines; | * <code>prod-sl6</code> — contains scripts for setting up '''Production''' environment on SL6 machines; | ||
* <code>prod-slc5</code> — contains scripts for setting up '''Production''' environment on SLC5 machines; | * <code>prod-slc5</code> — contains scripts for setting up '''Production''' environment on SLC5 machines; | ||
* <code>shared</code> — shared scripts used in the setup of all environments (see below); | * <code>shared</code> — shared scripts used in the setup of all environments (see below); | ||
* <code>eb-centos7</code> — contains scripts for setting up PBT environment on CentOS7 machines using EasyBuild modules; | |||
* <code>old</code> — legacy scripts used in previous environment setup on SLC5. | * <code>old</code> — legacy scripts used in previous environment setup on SLC5. | ||
The first 3 directories contain scripts that are almost identical, with small modifications to suit each environment. | The first 3 directories contain scripts that are almost identical, with small modifications to suit each environment. | ||
Along with the shared scripts, these are described below: substitute <code>xxx</code> for the particular environment. | Along with the shared scripts, these are described below: substitute <code>xxx</code> for the particular environment. | ||
The <code>eb-centos7</code> directory contains a version of all these scripts modified to work with EasyBuild modules. | |||
=== <code>/unix/pbt/software/scripts/xxx/pbt-xxx.sh</code> === | === <code>/unix/pbt/software/scripts/xxx/pbt-xxx.sh</code> === | ||
| Line 126: | Line 128: | ||
== Installations == | == Installations == | ||
For CentOS7 machines, each software is installed using EasyBuild. | |||
For each software package | For SL6 and SL5 machines, the installation procedure for each software package is listed below. | ||
The basic method used is as follows: | The basic method used is as follows: | ||
# Download the source tarball latest version to the <code>/unix/pbt/software/src</code> folder. | # Download the source tarball latest version to the <code>/unix/pbt/software/src</code> folder. | ||
| Line 146: | Line 149: | ||
== Software Matrix == | == Software Matrix == | ||
The following software is installed on the HEP Linux cluster. | The following software is installed on the HEP Linux cluster. Starting from CentOS7, the supported procedure requires EasyBuild. | ||
CentOS7 Machines: | CentOS7 Machines: (not supported) | ||
{| class="wikitable" | {| class="wikitable" | ||
Latest revision as of 02:39, 26 June 2024
Introduction
The UCL HEP Linux Cluster consists of a range of machines running Scientific Linux (CERN) 5 and Scientific Linux 6. This cluster provides a pre-compiled installation of Geant4 along with an environment for compiling code, running examples and viewing results.
Although there are over 100 individual machines that can be used — most of which are the individual Linux desktop PCs within the HEP group running SL6 — access from outside the HEP group subnet is only possible through the central Linux server, plus1, via ssh:
$ ssh -Y username@plus1.hep.ucl.ac.uk
It is then possible to connect to individual Linux desktops, again by ssh. In order to do so, you will need to know the number of the PC you are connecting to:
$ ssh -Y username@pc1XX.hep.ucl.ac.uk
...where 1XX is the number of the PC you wish to connect to.
In order to access these machines, you will need to obtain an account from the UCL HEP Group Computing Support. Please contact Dr. Simon Jolly for instructions on obtaining an account.
Folder structure
Disk
There is a separate (large) disk for PBT work mounted on the Linux system at /unix/pbt.
When running simulations, you should create a subdirectory with your username at /unix/pbt/username
Software
All software is enclosed in a folder at /unix/pbt/software.
Source code
Source code for building and installing software packages is kept in a separate folder at /unix/pbt/software/src.
Environments
Two separate environments are provide for PBT simulations: the Production environment for general use and the Development environment for testing changes or updates to the code. The Production environment is further subdivided depending on whether the machine you are logged in to is running SLC5 or SL6: the SLC5 legacy environment features an older version of Geant4 and related software that will not be updated.
- The recommended Production environment for SL6 is stored in
/unix/pbt/software/prod-sl6; - The Development environment for SL6 is stored in
/unix/pbt/software/dev: code is then subdivided into directories by application name; - The legacy Production environment for SLC5 is stored in
/unix/pbt/software/prod: this is only accessible for users running on SLC5 machines.
It is possible to select whether the Development or Production environment is launched under SL6 by adding the appropriate flags when calling the setup script (see #Scripts below).
Scripts
Access to the PBT environments is implemented by a single script, pbt.sh.
This calls a number of other utility scripts that set up the required PBT environment: these are all stored in /unix/pbt/software/scripts/.
This will auto-detect whether you are running on a newer ContOS7 machine or a legacy SLC6 or SLC5 machine and set up the environment accordingly.
If you are running on a CentOS7 machine, we reccomand to set up all the required EasyBuild modules and environment variables sourcing the following script
/unix/pbt/software/scripts/eb-centos7/pbt.sh
For the PBT Production and Development environments on SLC6 or SLC5 machines, you must source the followig
/unix/pbt/software/scripts/pbt.sh
The steps carried out during setup are as follows:
- Check the environment and whether the user wishes to run in verbose or development mode (see below);
- Launch a new
bashshell with the appropriate Scientific Linux Developer Toolset (see below); - Call the appropriate
bashrcscript to begin environment setup; this in turn runs the following actions:- Run the user's own
.bashrc; - Set the environment name;
- Greeting with information for users;
- Define exit function with exit message when user exits environment;
- Call shell script to set up environment.
- Run the user's own
- The specific shell script then carries out the following actions:
- Set specific environment variables with locations of compiler, Geant4 etc.
- Add locations to path variables and remove duplicates (calls
setpaths.shscript); - Define colours for scripts, prompt, etc (calls
definecolours.shscript); - Check versions of installed software and display results (calls
checkversions.shscript); - Modify prompt to show environment name;
- Run
Geant4setup script and set additional settings;
Adding the -v flag enables the scripts to run in verbose mode, which displays the full paths when checking the versions of the installed software.
The -b flag sources the BDSIM PBT environment: this is only available on CentOS7 machines and using the script version available for EasyBuild modules (stored in /unix/pbt/software/scripts/eb-centos7).
Adding the -d flag enables the Development environment to allow new software to be built without changing the Production setup needed by most users: this is only available on SL6 machines.
When pbt.sh is called, its first action after selecting the right environment is to launch a new bash shell that provides a degree of isolation for setting up the desired PBT environment.
Once the user has finished with the PBT environment, exiting the shell will deposit the user back into their previous environment.
Launching a new shell also provides access to the correct Scientific Linux Developer Toolset: on CentOS7 this is devtoolset-7, on SL6 this is devtoolset-3, whereas SLC5 uses the older devtoolset-2.
In either case, the Developer Toolset is a Linux software collection providing various tools for developers, including more up-to-date versions of software required by Geant4, most notably including a newer version of GCC (the GNU Compiler Collection).
The following sub-directories are present within the scripts directory:
dev— contains scripts for setting up Development environment on SL6 machines;prod-sl6— contains scripts for setting up Production environment on SL6 machines;prod-slc5— contains scripts for setting up Production environment on SLC5 machines;shared— shared scripts used in the setup of all environments (see below);eb-centos7— contains scripts for setting up PBT environment on CentOS7 machines using EasyBuild modules;old— legacy scripts used in previous environment setup on SLC5.
The first 3 directories contain scripts that are almost identical, with small modifications to suit each environment.
Along with the shared scripts, these are described below: substitute xxx for the particular environment.
The eb-centos7 directory contains a version of all these scripts modified to work with EasyBuild modules.
/unix/pbt/software/scripts/xxx/pbt-xxx.sh
The 3 pbt-xxx.sh scripts carry out the majority of the customisation for the particular environment.
In particular, the following tasks are carried out:
- Add locations to path variables;
- Define colours for scripts, prompt, etc.;
- Add locations to path variables and remove duplicates (calls
setpaths.shscript); - Check versions of installed software and display results (calls
checkversions.shscript); - Run Geant4 setup script;
These scripts in turn make use of the following shared scripts.
This script defines various variables to represent colours for printing to the terminal and to file.
This script sets the various PATH environment variables required for the specific environment and removes any duplicates.
This script checks the versions (and optionally the paths/locations) of important software packages used in the PBT environment.
Installations
For CentOS7 machines, each software is installed using EasyBuild.
For SL6 and SL5 machines, the installation procedure for each software package is listed below. The basic method used is as follows:
- Download the source tarball latest version to the
/unix/pbt/software/srcfolder. - Extract the tarball to a subfolder within the
srcfolder usingtar -xvzf <filename>.tar.gz. - Create a build folder at
/unix/pbt/software/buildand set the working directory here. - Run the specific configure, build and install steps for the package.
- Remove the build folder and extracted source folder, but keep the source tarball.
The configure, build and install steps may be different for each package, and this is described per package below. However, in most cases the steps follow the general steps below:
- Change working directory to the build folder.
- Run the configuration script from the source folder to set the build options, including the customised installation prefix.
- Build the code.
- Install the code to the sandbox folder.
- (Optional) Move the compiled code to the desired location (only possible if the installation prefix cannot be specified).
Software Matrix
The following software is installed on the HEP Linux cluster. Starting from CentOS7, the supported procedure requires EasyBuild.
CentOS7 Machines: (not supported)
| Software | Version | Source code | Installed in dev? | Installed by | Working? | Checked by | Installed in prod? | Installed by | Working? | Checked by |
| Qt | 5.9.3 | src/geant4-prerequisites/qt/qtbase-github | Yes | James | Yes | James | Yes | James | Yes | James |
| cmake | 3.9.4 | src/geant4-prerequisites/cmake/cmake-gitlab | Yes | Bootstrap (James) | Yes | James | Yes | Bootstrap (James) | Yes | James |
| xerces-c | 3.2.0 | src/geant4-prerequisites/xerces-c/xerces-c-github | Yes | James | Yes | James | Yes | James | Yes | James |
| CLHEP | 2.3.4.5 | src/geant4-prerequisites/clhep/clhep-github | Yes | James | Yes | James | Yes | James | Yes | James |
| simage | 1.7.1a | src/geant4-prerequisites/coin3d_mercurial/simage | Yes | James | Yes | James | Yes | James | Yes | James |
| Coin3d | 4.0.0a | src/geant4-prerequisites/coin3d_mercurial/coin-cmake | Yes | James | Yes | James | Yes | James | Yes | James |
| SoQt | 1.6.0a | src/geant4-prerequisites/coin3d_mercurial/soqt | Yes | James | Yes | James | Yes - missing config file so says "not found" | James | Yes | James |
| SoXt | 1.3.1 | src/geant4-prerequisites/coin3d_mercurial/soxt | Yes | James | Yes | James | Yes | James | Yes | James |
| Geant4 | 4.10.03.p02 | src/geant4/geant4-github | Yes | James | Yes | James | Yes | James | Yes | James |
| DAWN | 3.90b | src/geant4-addons/dawn | Yes | James | No | James | Yes (can copy from dev) | James | No | Roisin |
| HepRApp | 3.15.4 | src/geant4-addons/heprapp | Yes | Simon | Yes | Simon (Standalone .jar file) | Yes | Simon | Yes | Simon (Standalone .jar file) |
| Java Analysis Studio | 3.1.5 | src/geant4-addons/jas | Yes | Simon | Yes | Simon (Standalone executables) | Yes | Simon | Yes | Simon (Standalone executables) |
| gMocren | 4 | src/geant4-addons/gmocren | Yes | Simon | Unknown | Simon (Standalone executables) | Yes | Simon | Unknown | Simon (Standalone executables) |
| BDSIM | 0.9.9.3 | src/bdsim/bdsim-edited | Yes | James | Yes | James | No |
SL6 Machines:
| Software | Version | Source code | Installed in dev? | Installed by | Working? | Checked by | Installed in prod? | Installed by | Working? | Checked by |
| Qt | 5.5.1 | src/geant4-prerequisites/qt/qt-5.5.1 | Yes | Simon | Yes | Lawrence | Yes | Simon | Yes | Lawrence |
| cmake | 3.5.2 | src/geant4-prerequisites/cmake | Yes | Static binaries (Simon) | Yes | Lawrence | Yes | Static binaries (Simon) | Yes | Lawrence |
| xerces-c | 3.1.3 | src/geant4-prerequisites/xerces-c | Yes | Simon | Yes | Lawrence | Yes | Simon | Yes | Lawrence |
| CLHEP | 2.3.1.1 | src/geant4-prerequisites/clhep | Yes | Lawrence | Yes | Lawrence | Yes | Lawrence | Yes | Lawrence |
| simage | 1.7.0 | src/geant4-prerequisites/coin3d | Yes | Simon | Yes | Lawrence | Yes | Simon | Yes | Lawrence |
| Coin3d | 3.1.3 | src/geant4-prerequisites/coin3d | Yes | Simon | Yes | Lawrence | Yes | Simon | Yes | Lawrence |
| SoQt | 1.5.0 | src/geant4-prerequisites/coin3d | Yes | Simon | Yes | Lawrence | No | |||
| SoXt | 1.3.0 | src/geant4-prerequisites/coin3d | Yes | Simon | Yes | Lawrence | Yes | Simon | Yes | Lawrence |
| Geant4 | 4.10.02.p01 | src/geant4/geant4.10.02.p01-source | Yes | Lawrence | Yes | Lawrence | Yes | Lawrence | Yes | Lawrence |
| DAWN | 3.90b | src/geant4-addons/dawn | Yes | Simon | Yes | Simon | Yes (can copy from dev) | Simon | Yes | Roisin |
| HepRApp | 3.15.4 | src/geant4-addons/heprapp | Yes | Simon | Yes | Simon (Standalone .jar file) | Yes | Simon | Yes | Simon (Standalone .jar file) |
| Java Analysis Studio | 3.1.5 | src/geant4-addons/jas | Yes | Simon | Yes | Simon (Standalone executables) | Yes | Simon | Yes | Simon (Standalone executables) |
| gMocren | 4 | src/geant4-addons/gmocren | Yes | Simon | Unknown | Simon (Standalone executables) | Yes | Simon | Unknown | Simon (Standalone executables) |
| BDSIM | 0.9.9.3 | src/bdsim/bdsim-edited | Yes | James | Yes | James | No |