PBTWiki - User contributions [en]

Software/EasyBuild

2019-02-21T09:49:12Z

MatthieuHentz: /* Show loaded modules */

=Instructions for using software with EasyBuild.=

= Compile BDSIM =
=== James Chappell's question ===
Dear Laurie,

I’m trying to learn how to use the version of easy build that you setup to run some BDSIM simulations for AWAKE. I have a few questions for you.

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

2) Having looked through your eb files in /unix/pbt/software/eb/accsoft-eb-nevay, I can see that you define sources using a url and I’m assuming that in the process of compiling all the software, this source code is downloaded and then compiled. Is there a way to define the source from within the local system directories and then compile this version? Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

Sorry for all the questions, I’m just trying to get my head around how it all works! Thanks for your help,
James.

=== The answer ===
Dear James,

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

My bad -> I'll recompile it with awake on. I need a time slot when you won't be using it as it'll disappear during the rebuilding - pretty bad if someone's running jobs when I do it! Let me know when I can do this - only takes about 15 mins.

Is there a way to define the source from within the local system directories and then compile this version?
It looks like the system really only is designed to download things from what I can read online... Despite the url, it caches them in a sources directory, so if it was already there it would use that (a fudge though).

Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

We have so far in the Bdsim-1.1-GCC-4.9.4.eb easy build file:

configopts = ' -DCMAKE_CXX_FLAGS="-std=c++11"'

We can add to this:

configopts += '-DUSE_AWAKE=ON'

See ROOT-v6.08.06-GCC-4.9.3-Python-2.7.12.eb for lots of examples.

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

If you're modifying the C++ often I'd make your own build of BDSIM + AWAKE separately from the EasyBuild system and not bother trying to package it into a module. The module system is really for things that won't change.

You can get all the dependencies from EasyBuild and then make your own independent build of whatever you want. I think this was done for Gate for example and we do this just now for our nightly testing of BDSIM. I load the BDSIM module, then unload it - this leaves all the dependencies loaded.

module purge
module load Bdsim
module unload Bdsim
module list

# still has all libraries loaded including Geant4 etc..
# somewhere else, make a build dir and build from your source.

When you load Geant4 and ROOT, note you should really source their setup scripts:

source $EBROOTGEANT4/bin/geant4.sh
source $EBROOTROOT/bin/thisroot.sh

--->>> You should make a copy of the bdsim.sh script that does most of this and just add the unload to the end.

then...

mkdir build
cd build
cmake ../relative/path/to/bdsim -DCMAKE_CXX_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/g++
-DCMAKE_C_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/gcc
-DCLHEP_LIBRARY_DIR=/home/accsoft/SL68/software/CLHEP/2.3.1.1-GCC-4.9.3/lib/
make -j10

* paths are example on my system at RHUL

Normally you wouldn't have to specify these options as CMake would find these on its own. However, practice shows it often finds a system compiler before it finds the one in easybuild (this is required to be the same for all the software to be consistent), so we explicitly specify them.

Note once a build is make and it finds a compiler, the compiler can never be changed in CMake. You must wipe the build directory contents to try again.

How do you use this new build? Simple, you must have the same environment loaded as when it was built (so hence the modified bdsim.sh) then execute from that directory. This could easily be done in your shell script for the simulation or via an alias (fictional paths here as an example)...

alias bdsim-awake-devel="/home/jchappell/my-builds/build123/bdsim"
bdsim-awake-devel --help

Note if you haven't changed the output or the analysis you can just use the regular bdsim from easybuild if you do any analysis later on the output. If you have changed it, you'll need to use your build and modify your bdsim.sh - pretty clear where to change for the analysis stuff.

Ok, hopefully that should help you both get the build working and help you understand.

Cheers,
Laurie

=== Additional info ===
Hello,

Ok, I've updated the general BDSIM build in easy build now. It should have the AWAKE module turned on. Note, this is the code of v1.1 and not the latest develop version.

I made an example environment only script in:

/unix/pbt/software/eb/bdsim-own-build.sh

I tried making my own build and it worked just fine. I only needed to do also:

module load CMake

to get a more up to date version. From my home dir:

mkdir build-custom-build
cd build-custom-build
cmake ../bdsim
# check compiler is GCC4.9.3 form easy build, and ROOT and Geant4 and CLHEP -> all ok
ccmake .
turn on USE_AWAKE
<c>
<g>
make -j15
works!

It's in /home/lnevay/bdsim-custom-build

Cheers,
Laurie

= Use modules =
=== Set path to modules ===
Before modules can be accessed, the module tool needs to be given the path to the desired set of modules. For example,

<pre>
module use /unix/pbt/software/eb/CentOS7/modules/*
</pre>

=== List available modules ===
You can then list all modules available with

<pre>module avail</pre>

or check what versions of a given module are available using

<pre>module avail [modulename]</pre>

For example,

<pre>module avail Geant4</pre>

prints out the following

<pre>
-------------------------------------------------------------------- /unix/pbt/software/eb/CentOS7/modules/phys --------------------------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3 Geant4/10.04.p02-mt-GCC-4.9.3

-------------------------------------------------------------------- /unix/pbt/software/eb/CentOS7/modules/all ---------------------------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3 Geant4/10.04.p02-mt-GCC-4.9.3
</pre>

=== Load modules ===
Load a given module using the command

<pre>module load</pre>

This will load the specified module along with all its dependencies. For instance,

<pre>
module load Geant4/10.04.p02-GCC-4.9.3
</pre>

loads Geant4 and all of the following

1) GCC/4.9.3 7) libxml2/2.9.3-GCC-4.9.3 13) libdrm/2.4.68-GCC-4.9.3 19) libffi/3.2.1-GCC-4.9.3
2) expat/2.1.1-GCC-4.9.3 8) libpng/1.6.21-GCC-4.9.3 14) ncurses/6.0-GCC-4.9.3 20) gettext/0.19.6-GCC-4.9.3
3) CLHEP/2.3.3.0-GCC-4.9.3 9) bzip2/1.0.6-GCC-4.9.3 15) LLVM/3.8.0-GCC-4.9.3 21) PCRE/8.38-GCC-4.9.3
4) cURL/7.44.0-GCC-4.9.3 10) freetype/2.6.3-GCC-4.9.3 16) eudev/3.1.5-GCC-4.9.3 22) GLib/2.49.5-GCC-4.9.3
5) Xerces-C++/3.1.2-GCC-4.9.3 11) fontconfig/2.11.95-GCC-4.9.3 17) Mesa/11.2.1-GCC-4.9.3 23) Qt5/5.7.0-GCC-4.9.3
6) zlib/1.2.8-GCC-4.9.3 12) X11/20160819-GCC-4.9.3 18) libGLU/9.0.0-GCC-4.9.3-Mesa-11.2.1 24) Geant4/10.04.p02-GCC-4.9.3

Don't forget to source the relevant environment variables where required, e.g.

<pre>
source $EBROOTGEANT4/bin/geant4.sh
</pre>

=== Show loaded modules ===
To see what's loaded use

<pre>
module list
</pre>

=== Remove loaded modules ===
To get rid of all loaded modules:

<pre>
module purge
</pre>

= Install modules =

=== Installing modules ===

Example command to install module:

eb Bdsim-1.2.1-GCC-4.9.3.eb --prefix=/unix/pbt/software/eb/CentOS7 -robot --robot-path=./: --modules-tool=EnvironmentModulesC --module-syntax=Tcl --allow-modules-tool-mismatch --optarch=GENERIC --buildpath=/tmp/easybuild/

Two types of the same version (i.e. one single-threaded and one multi-threaded) can be installed side-by-side. To get EasyBuild to not replace a current version, the module can be renamed in the EasyBuild file. When a module is built, EasyBuild checks for modules of the same name and version and if one already exists, the build won't complete. If the options are updated, the build will be updated.

When you build a module, you should do the `module use` command so it can find the others already available (it checks the requirements of the one you're building) and also the EasyBuild module itself. You should have a clean environment with no other modules loaded.

/unix/pbt/software/eb/easybuildonly.sh

seems to provide this.

I tried my example and found you should edit the version rather than the name for it to work. I've done this and it's setup now. You can import it as.

module load Geant4/10.04.p02-mt-GCC-4.9.3

Cheers,
Laurie

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

=== The question ===
Dear Laurie,

I'm trying to compile a multi-threaded version of Geant4 using EasyBuild but would like to make sure I don't break everything.

Looking at the easyconfig file for the most recent Geant4 version installed on our system and the directory structure of the `eb/CentOS7/software` directory, I can see that the location of the installation depends on the name and the version of the package and on the toolchain used to compile it. This leads me to believe that if I were to set `GEANT4_BUILD_MULTITHREADED=ON` in the easyconfig file and run the following command,

eb Geant4-10.04.p02-GCC-4.9.3.eb --robot

it would overwrite the current (non-mutli-threaded) version of Geant4. I have two questions relating to this:

1) Am I correct in thinking it would overwrite it?
2) If so, would you mind showing me how I can compile it alongside the existing version?

The best solution I could come up with would be to install it in a different location using the command line option `--installpath-software=INSTALLPATH-SOFTWARE` but I'm unsure if that breaks compatibility with the modules tool and I'd also like to avoid overwriting the existing easyconfig file, which I don't know how to ensure.

= Note on how to install Geant4 on Mac =
Hello,

Ah we tried it briefly on mac, but it was a little frustrating so we stuck with our usual way, which is macports. For your mac, I'd recommend that. I know it's different than EasyBuild, but it's generally easier for mac and for scientific software.

https://www.macports.org

Before getting it you should xcode + the command line tools (from app store) installed (test is you can run 'make' on the terminal). You should also have XQuartz (https://www.xquartz.org) installed for X11 support. Then download and install the macports software (small, little mac installer). Gives you a command on the terminal 'port'. Everything is by default installed in /opt/local so you can nuke that directory if you don't want it anymore. It works by adding a line to your profile that adds that directory to your paths.

port search packagename

sudo port install py27-matplotlib

I'd install everything up to CLHEP from macports, then make my own geant4 installation. I think there is a geant4 port on there, but I haven't used it. For the geant4 installation you download hte source somewhere, make a build directory (outside the source) and then an install directory. Usual CMake pattern.

I'd start with a high level package in macports and it'll work out all the requirements. Like...

sudo port install clhep qt5 py27-matplotlib

I do everything apart from geant4 and bdsim through macports.

Hope that helps,
Cheers,
Laurie

= Summary from Raffy about EasyBuild, Environment Modules and GEANT4 =
== Geant4 ==
===Reference===
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/gettingstarted.html
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
===GEANT4 OS/Software Prerequisites===
the following source/software must be present to build Geant4:
* Geant4 Toolkit Source Code.
* C++ Compiler and Standard Library supporting the C++11 Standard
* Linux (Scientific Linux CERN 6 and Linux CentOS 7 ): GNU Compiler Collection 4.8.5 or higher ( recommended to use the GCC compiler supplied by the package management system of your distribution unless this does not meet the minimum version requirement. The minimum required versions of GCC may be installed on SLC6 systems via the freeDeveloper Toolset)
* CMake 3.3 or higher (as provided through the package management system of your distribution, unless it does not meet the minimum version requirement.)
The main requirement is that the system has a GCC of sufficient version to support C++11 installed.

===User Interface and Visualization Drivers===
* Qt User Interface and Visualization
** Qt4 (>=4.6) or Qt5 headers and libraries
* OpenGL or MesaGL headers and libraries.
** X11 OpenGL Visualization (Linux and macOS)
* Open Inventor Visualization
* Coin3D libraries and headers with SoXt(SoWin) graphics binding on Linux/macOS(Windows), compiled against the C++11 standard.
* Motif User Interface and Visualization (Linux and macOS)

===Standard GEANT4 Building and Installation===
* Unpack the Geant4 source package geant4.10.05.tar.gz to a location of your choice: source directory /path/to/geant4.10.05
* create a directory in which to configure and run the build and store the build products /path/to/geant4.10.05-build
* $ cmake -DCMAKE_INSTALL_PREFIX=/path/to/geant4.10.05-install /path/to/geant4.10.05
* CMake Variable CMAKE_INSTALL_PREFIX is used to set the install directory, the directory under which the Geant4 libraries, headers and support files will be installed. It must be supplied as an absolute path. The second argument to CMake is the path to the source directory.
* Additional arguments may be passed to CMake to activate optional components of Geant4, such as visualization drivers, or tune the build and install parameters.
* If you run CMake and decide afterwards you want to activate additional options, simply rerun CMake in the build directory, passing it the extra options plus the build directory path.
* Datasets are not required to be present to build Geant4, but may be required to run your application, depending on the physics models you use. If you wish to download and install the datasets automatically as part of your build of Geant4, simply add the option-DGEANT4_INSTALL_DATA=ON to the arguments passed to CMake.
* After the configuration has run, CMake will have generated Unix Makefiles for building Geant4. To run the build, simply execute make in the build directory
* Once the build has completed, you can install Geant4 to the directory you specified earlier in CMAKE_INSTALL_PREFIX
* additional options can be passed to CMake to adjust the way Geant4 is built and installed and to enable optional components. Some options enable components of Geant4 which require external software.If these options are enabled, the required software will be searched for, and hence there are also options which control where CMake should look for these packages.These options may be set by passing their name and value to the cmake command via -D flags.

===Standard Options===
* CMAKE_INSTALL_PREFIX installation prefix for Geant4. (default Unix: /usr/local)
* CMAKE_BUILD_TYPE : (DEFAULT : Release) Sets the type of build. It defaults to Release which gives a fully optimized build without debugging symbols.
* GEANT4_BUILD_MULTITHREADED : (DEFAULT : OFF) If set to ON, build Geant4 libraries with support for multithreading. At present, this is only supported on Unix systems. (Requires: Compiler/C++ Standard Library with support for C++ Threading Support)
* GEANT4_INSTALL_DATA : (DEFAULT : OFF) to ON, download and install any Geant4 datasets missing from GEANT4_INSTALL_DATADIR.
* GEANT4_INSTALL_DATADIR : (DEFAULT : CMAKE_INSTALL_DATAROOTDIR) Installation directory for Geant4 datasets.
* GEANT4_USE_G3TOG4 : (DEFAULT : OFF) If set to ON, build the G3ToG4 library for reading ASCII call list files generated from Geant3 geometries.
* GEANT4_USE_GDML : (DEFAULT : OFF) If set to ON, build the G4persistency library with support for GDML.
* GEANT4_USE_QT (DEFAULT : OFF) If set to ON, build Qt4/5 User Interface and Visualization drivers. Requires: Qt4 or Qt5 and OpenGL libraries and headers. Qt5 will be searched for first. If it is not found Qt4 will be searched for. This behaviour can be adjusted through the advanced option GEANT4_FORCE_QT4.
* GEANT4_USE_OPENGL_X11 (DEFAULT : OFF, Unix Only)If set to ON, build the X11 OpenGL visualization driver.
* GEANT4_USE_RAYTRACER_X11 (DEFAULT : OFF, Unix only) If set to ON, build RayTracer visualization driver with X11 support.
* GEANT4_USE_INVENTOR (DEFAULT : OFF) If set to ON, build the OpenInventor visualization driver.
* GEANT4_USE_XM (DEFAULT : OFF, Unix Only) If set to ON, build Motif User Interface and Visualization drivers.
* GEANT4_USE_SYSTEM_CLHEP (DEFAULT : OFF) If set to ON, build Geant4 with an external install of CLHEP. You should not set this unless your usage of Geant4 mandates a specific external CLHEP installation (e.g. if your project’s software uses CLHEP in other tools and requires consistent use of the same CLHEP across the software stack).Requires: CLHEP libraries and headers. See the CLHEP_XXX and CMAKE_PREFIX_PATHAdvanced Options if CMake has trouble locating CLHEP.
* GEANT4_USE_SYSTEM_EXPAT (DEFAULT : ON) If set to ON, build Geant4 with an external install of Expat.
* GEANT4_USE_SYSTEM_ZLIB (DEFAULT : OFF) If set to ON, build Geant4 with an external install of zlib.

===Advanced Options===
* http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
*Among the advanced, I list only the one used in the easy build recipe file https://bitbucket.org/jairhul/accsoft-eb/src/5f285502686e28d656dfa4d3ce1408d2551ee5d1/Geant4-10.00.p04-GCC-4.9.3.eb?at=master&fileviewer=file-view-default
** EXPAT_LIBRARY If CMake cannot locate your external EXPAT installation, set this to the full path to the EXPAT library, e.g. /usr/lib/libexpat.so
** EXPAT_INCLUDE_DIR If CMake cannot locate your external EXPAT installation, set this to the directory containing the EXPAT headers (e.g. if you have /foobar/expat.h, then set this to /foobar).

== EasyBuild ==
===Reference===
https://media.readthedocs.org/pdf/caylo-easybuild/docfix/caylo-easybuild.pdf
EasyBuild is a flexible software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way. Divert from the standard configure / make / make install with custom procedures. Allows for easily reproducing previous builds.

===EasyBuild===
EasyBuild consists of a collection of Python modules and packages that interact with each other, dynamically picking up additional Python modules as needed for building and installing a (stack of) software package(s) specified via simple specification files

In EasyBuild terminology: the EasyBuild framework leverages easyblocks to automatically build and install software using particular compiler toolchains, as specified by one or multiple easyconfig files.

The EasyBuild framework provides functionality commonly needed when installing scientific software on HPC systems. For example, it deals with downloading, unpacking and patching of sources, loading module files for dependencies, setting up the build environment, autonomously running (interactive) shell commands, creating module files that match the specification files, etc.
The implementation of a particular software build and install procedure is done in a Python module, which is aptly referred to as an easyblock.
For each software package being built, the EasyBuild framework will determine which easyblock should be used, based on the name of the software package or the value of the easyblock specification parameter (see Easyblock specification). Since EasyBuild v2.0, an easyblock must be specified in case no matching easyblock is found based on the software name (cfr. Automagic fallback to ConfigureMake).
The specification files that are supplied to EasyBuild are referred to as easyconfig files (or simply easyconfigs .eb), which are basically plain text files containing (mostly) key-value assignments for build parameters supported by the framework, also referred to as easyconfig parameters (see Writing easyconfig files: the basics for more information).
An easyconfig file serves as a build specification for EasyBuild.
Easyconfigs typically follow a (fixed) strict naming scheme, i.e. <name>-<version>[-<toolchain>][<versionsuffix>
A handful of easyconfig parameters are mandatory:
* name, version: specify what software (version) to build
* homepage, description: metadata (used for module help)
* toolchain: specifies name and version of compiler toolchain to use
* Using external modules as dependencies
* Configure/build/install command options:
** configopts: options for configure command
** preconfigopts: options used as prefix for configure command
** buildopts, prebuildopts: options for build command
** installopts, preinstallopts: options for install

EasyBuild allows to easily reproduce software builds/installations and install multiple versions.

==Environment Modules==
===Reference===
http://modules.sourceforge.net/man/module.html
===Environment Modules===
Typically users initialize their environment when they log in by setting environment information for every application they will reference during the session.
The Environment Modules system is a tool to help users manage their Unix or Linux shell environment, by allowing groups of related environment-variable settings to be made or removed dynamically. The modules system is based on modulefiles,[6] which specify groups of environment settings that need to be made together. Modulefiles can be installed in a central location for general use, or in a user directory for personal use. Environment Modules modulefiles are written in the Tcl (Tool Command Language) and are interpreted by the modulecmd program via the module[7] user interface. The key advantage of Environment Modules is that it is shell independent and supports all major shells such as bash, ksh, zsh, sh, tcsh, and csh. The second key advantage is that it allows to use multiple versions of the program or package from the same account by just loading proper module. modulefiles are created on per application per version basis. They can be dynamically loaded, unloaded, or switched. Along with the capability of using multiple versions of the same software it also can be used to implement site policies regarding the access and use of applications.
https://modules.readthedocs.io/en/latest/FAQ.html

===New commands for centOS 7 ===

# Tell module tool where the modules are
module use /unix/pbt/software/eb/CentOS7/modules/*
# Load Geant4
# This loads all the necessary libraries like CLHEP, Qt5, etc.
module load Geant4/10.04.p02-GCC-4.9.3
source $EBROOTGEANT4/bin/geant4.sh
module load CMake/3.5.2-GCC-4.9.3

'''what does this mean'''
* module use [-a|--append] directory…
Prepend one or more directories to the MODULEPATH environment variable.
* load modulefile...
Load modulefile(s) into the shell environment.

when Geant4 is installed as a module
To see which versions are available, execute: module avail geant4
To then use a particular version, execute for example
module load geant4/10.02.p03
After you load the module, you can see where the software is installed by executing
echo $EBROOTGEANT4

Set GEANT environment variables
source $EBROOTGEANT4/bin/geant4.sh

Software/EasyBuild

2019-02-21T09:48:55Z

MatthieuHentz: /* Load modules */

=Instructions for using software with EasyBuild.=

= Compile BDSIM =
=== James Chappell's question ===
Dear Laurie,

I’m trying to learn how to use the version of easy build that you setup to run some BDSIM simulations for AWAKE. I have a few questions for you.

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

2) Having looked through your eb files in /unix/pbt/software/eb/accsoft-eb-nevay, I can see that you define sources using a url and I’m assuming that in the process of compiling all the software, this source code is downloaded and then compiled. Is there a way to define the source from within the local system directories and then compile this version? Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

Sorry for all the questions, I’m just trying to get my head around how it all works! Thanks for your help,
James.

=== The answer ===
Dear James,

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

My bad -> I'll recompile it with awake on. I need a time slot when you won't be using it as it'll disappear during the rebuilding - pretty bad if someone's running jobs when I do it! Let me know when I can do this - only takes about 15 mins.

Is there a way to define the source from within the local system directories and then compile this version?
It looks like the system really only is designed to download things from what I can read online... Despite the url, it caches them in a sources directory, so if it was already there it would use that (a fudge though).

Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

We have so far in the Bdsim-1.1-GCC-4.9.4.eb easy build file:

configopts = ' -DCMAKE_CXX_FLAGS="-std=c++11"'

We can add to this:

configopts += '-DUSE_AWAKE=ON'

See ROOT-v6.08.06-GCC-4.9.3-Python-2.7.12.eb for lots of examples.

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

If you're modifying the C++ often I'd make your own build of BDSIM + AWAKE separately from the EasyBuild system and not bother trying to package it into a module. The module system is really for things that won't change.

You can get all the dependencies from EasyBuild and then make your own independent build of whatever you want. I think this was done for Gate for example and we do this just now for our nightly testing of BDSIM. I load the BDSIM module, then unload it - this leaves all the dependencies loaded.

module purge
module load Bdsim
module unload Bdsim
module list

# still has all libraries loaded including Geant4 etc..
# somewhere else, make a build dir and build from your source.

When you load Geant4 and ROOT, note you should really source their setup scripts:

source $EBROOTGEANT4/bin/geant4.sh
source $EBROOTROOT/bin/thisroot.sh

--->>> You should make a copy of the bdsim.sh script that does most of this and just add the unload to the end.

then...

mkdir build
cd build
cmake ../relative/path/to/bdsim -DCMAKE_CXX_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/g++
-DCMAKE_C_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/gcc
-DCLHEP_LIBRARY_DIR=/home/accsoft/SL68/software/CLHEP/2.3.1.1-GCC-4.9.3/lib/
make -j10

* paths are example on my system at RHUL

Normally you wouldn't have to specify these options as CMake would find these on its own. However, practice shows it often finds a system compiler before it finds the one in easybuild (this is required to be the same for all the software to be consistent), so we explicitly specify them.

Note once a build is make and it finds a compiler, the compiler can never be changed in CMake. You must wipe the build directory contents to try again.

How do you use this new build? Simple, you must have the same environment loaded as when it was built (so hence the modified bdsim.sh) then execute from that directory. This could easily be done in your shell script for the simulation or via an alias (fictional paths here as an example)...

alias bdsim-awake-devel="/home/jchappell/my-builds/build123/bdsim"
bdsim-awake-devel --help

Note if you haven't changed the output or the analysis you can just use the regular bdsim from easybuild if you do any analysis later on the output. If you have changed it, you'll need to use your build and modify your bdsim.sh - pretty clear where to change for the analysis stuff.

Ok, hopefully that should help you both get the build working and help you understand.

Cheers,
Laurie

=== Additional info ===
Hello,

Ok, I've updated the general BDSIM build in easy build now. It should have the AWAKE module turned on. Note, this is the code of v1.1 and not the latest develop version.

I made an example environment only script in:

/unix/pbt/software/eb/bdsim-own-build.sh

I tried making my own build and it worked just fine. I only needed to do also:

module load CMake

to get a more up to date version. From my home dir:

mkdir build-custom-build
cd build-custom-build
cmake ../bdsim
# check compiler is GCC4.9.3 form easy build, and ROOT and Geant4 and CLHEP -> all ok
ccmake .
turn on USE_AWAKE
<c>
<g>
make -j15
works!

It's in /home/lnevay/bdsim-custom-build

Cheers,
Laurie

= Use modules =
=== Set path to modules ===
Before modules can be accessed, the module tool needs to be given the path to the desired set of modules. For example,

<pre>
module use /unix/pbt/software/eb/CentOS7/modules/*
</pre>

=== List available modules ===
You can then list all modules available with

<pre>module avail</pre>

or check what versions of a given module are available using

<pre>module avail [modulename]</pre>

For example,

<pre>module avail Geant4</pre>

prints out the following

<pre>
-------------------------------------------------------------------- /unix/pbt/software/eb/CentOS7/modules/phys --------------------------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3 Geant4/10.04.p02-mt-GCC-4.9.3

-------------------------------------------------------------------- /unix/pbt/software/eb/CentOS7/modules/all ---------------------------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3 Geant4/10.04.p02-mt-GCC-4.9.3
</pre>

=== Load modules ===
Load a given module using the command

<pre>module load</pre>

This will load the specified module along with all its dependencies. For instance,

<pre>
module load Geant4/10.04.p02-GCC-4.9.3
</pre>

loads Geant4 and all of the following

1) GCC/4.9.3 7) libxml2/2.9.3-GCC-4.9.3 13) libdrm/2.4.68-GCC-4.9.3 19) libffi/3.2.1-GCC-4.9.3
2) expat/2.1.1-GCC-4.9.3 8) libpng/1.6.21-GCC-4.9.3 14) ncurses/6.0-GCC-4.9.3 20) gettext/0.19.6-GCC-4.9.3
3) CLHEP/2.3.3.0-GCC-4.9.3 9) bzip2/1.0.6-GCC-4.9.3 15) LLVM/3.8.0-GCC-4.9.3 21) PCRE/8.38-GCC-4.9.3
4) cURL/7.44.0-GCC-4.9.3 10) freetype/2.6.3-GCC-4.9.3 16) eudev/3.1.5-GCC-4.9.3 22) GLib/2.49.5-GCC-4.9.3
5) Xerces-C++/3.1.2-GCC-4.9.3 11) fontconfig/2.11.95-GCC-4.9.3 17) Mesa/11.2.1-GCC-4.9.3 23) Qt5/5.7.0-GCC-4.9.3
6) zlib/1.2.8-GCC-4.9.3 12) X11/20160819-GCC-4.9.3 18) libGLU/9.0.0-GCC-4.9.3-Mesa-11.2.1 24) Geant4/10.04.p02-GCC-4.9.3

Don't forget to source the relevant environment variables where required, e.g.

<pre>
source $EBROOTGEANT4/bin/geant4.sh
</pre>

=== Show loaded modules ===
To see what's loaded, use

<pre>
module list
</pre>

which prints, for example,

1) GCC/4.9.3 9) bzip2/1.0.6-GCC-4.9.3 17) Mesa/11.2.1-GCC-4.9.3
2) expat/2.1.1-GCC-4.9.3 10) freetype/2.6.3-GCC-4.9.3 18) libGLU/9.0.0-GCC-4.9.3-Mesa-11.2.1
3) CLHEP/2.3.3.0-GCC-4.9.3 11) fontconfig/2.11.95-GCC-4.9.3 19) libffi/3.2.1-GCC-4.9.3
4) cURL/7.44.0-GCC-4.9.3 12) X11/20160819-GCC-4.9.3 20) gettext/0.19.6-GCC-4.9.3
5) Xerces-C++/3.1.2-GCC-4.9.3 13) libdrm/2.4.68-GCC-4.9.3 21) PCRE/8.38-GCC-4.9.3
6) zlib/1.2.8-GCC-4.9.3 14) ncurses/6.0-GCC-4.9.3 22) GLib/2.49.5-GCC-4.9.3
7) libxml2/2.9.3-GCC-4.9.3 15) LLVM/3.8.0-GCC-4.9.3 23) Qt5/5.7.0-GCC-4.9.3
8) libpng/1.6.21-GCC-4.9.3 16) eudev/3.1.5-GCC-4.9.3 24) Geant4/10.04.p02-GCC-4.9.3

=== Remove loaded modules ===
To get rid of all loaded modules:

<pre>
module purge
</pre>

= Install modules =

=== Installing modules ===

Example command to install module:

eb Bdsim-1.2.1-GCC-4.9.3.eb --prefix=/unix/pbt/software/eb/CentOS7 -robot --robot-path=./: --modules-tool=EnvironmentModulesC --module-syntax=Tcl --allow-modules-tool-mismatch --optarch=GENERIC --buildpath=/tmp/easybuild/

Two types of the same version (i.e. one single-threaded and one multi-threaded) can be installed side-by-side. To get EasyBuild to not replace a current version, the module can be renamed in the EasyBuild file. When a module is built, EasyBuild checks for modules of the same name and version and if one already exists, the build won't complete. If the options are updated, the build will be updated.

When you build a module, you should do the `module use` command so it can find the others already available (it checks the requirements of the one you're building) and also the EasyBuild module itself. You should have a clean environment with no other modules loaded.

/unix/pbt/software/eb/easybuildonly.sh

seems to provide this.

I tried my example and found you should edit the version rather than the name for it to work. I've done this and it's setup now. You can import it as.

module load Geant4/10.04.p02-mt-GCC-4.9.3

Cheers,
Laurie

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

=== The question ===
Dear Laurie,

I'm trying to compile a multi-threaded version of Geant4 using EasyBuild but would like to make sure I don't break everything.

Looking at the easyconfig file for the most recent Geant4 version installed on our system and the directory structure of the `eb/CentOS7/software` directory, I can see that the location of the installation depends on the name and the version of the package and on the toolchain used to compile it. This leads me to believe that if I were to set `GEANT4_BUILD_MULTITHREADED=ON` in the easyconfig file and run the following command,

eb Geant4-10.04.p02-GCC-4.9.3.eb --robot

it would overwrite the current (non-mutli-threaded) version of Geant4. I have two questions relating to this:

1) Am I correct in thinking it would overwrite it?
2) If so, would you mind showing me how I can compile it alongside the existing version?

The best solution I could come up with would be to install it in a different location using the command line option `--installpath-software=INSTALLPATH-SOFTWARE` but I'm unsure if that breaks compatibility with the modules tool and I'd also like to avoid overwriting the existing easyconfig file, which I don't know how to ensure.

= Note on how to install Geant4 on Mac =
Hello,

Ah we tried it briefly on mac, but it was a little frustrating so we stuck with our usual way, which is macports. For your mac, I'd recommend that. I know it's different than EasyBuild, but it's generally easier for mac and for scientific software.

https://www.macports.org

Before getting it you should xcode + the command line tools (from app store) installed (test is you can run 'make' on the terminal). You should also have XQuartz (https://www.xquartz.org) installed for X11 support. Then download and install the macports software (small, little mac installer). Gives you a command on the terminal 'port'. Everything is by default installed in /opt/local so you can nuke that directory if you don't want it anymore. It works by adding a line to your profile that adds that directory to your paths.

port search packagename

sudo port install py27-matplotlib

I'd install everything up to CLHEP from macports, then make my own geant4 installation. I think there is a geant4 port on there, but I haven't used it. For the geant4 installation you download hte source somewhere, make a build directory (outside the source) and then an install directory. Usual CMake pattern.

I'd start with a high level package in macports and it'll work out all the requirements. Like...

sudo port install clhep qt5 py27-matplotlib

I do everything apart from geant4 and bdsim through macports.

Hope that helps,
Cheers,
Laurie

= Summary from Raffy about EasyBuild, Environment Modules and GEANT4 =
== Geant4 ==
===Reference===
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/gettingstarted.html
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
===GEANT4 OS/Software Prerequisites===
the following source/software must be present to build Geant4:
* Geant4 Toolkit Source Code.
* C++ Compiler and Standard Library supporting the C++11 Standard
* Linux (Scientific Linux CERN 6 and Linux CentOS 7 ): GNU Compiler Collection 4.8.5 or higher ( recommended to use the GCC compiler supplied by the package management system of your distribution unless this does not meet the minimum version requirement. The minimum required versions of GCC may be installed on SLC6 systems via the freeDeveloper Toolset)
* CMake 3.3 or higher (as provided through the package management system of your distribution, unless it does not meet the minimum version requirement.)
The main requirement is that the system has a GCC of sufficient version to support C++11 installed.

===User Interface and Visualization Drivers===
* Qt User Interface and Visualization
** Qt4 (>=4.6) or Qt5 headers and libraries
* OpenGL or MesaGL headers and libraries.
** X11 OpenGL Visualization (Linux and macOS)
* Open Inventor Visualization
* Coin3D libraries and headers with SoXt(SoWin) graphics binding on Linux/macOS(Windows), compiled against the C++11 standard.
* Motif User Interface and Visualization (Linux and macOS)

===Standard GEANT4 Building and Installation===
* Unpack the Geant4 source package geant4.10.05.tar.gz to a location of your choice: source directory /path/to/geant4.10.05
* create a directory in which to configure and run the build and store the build products /path/to/geant4.10.05-build
* $ cmake -DCMAKE_INSTALL_PREFIX=/path/to/geant4.10.05-install /path/to/geant4.10.05
* CMake Variable CMAKE_INSTALL_PREFIX is used to set the install directory, the directory under which the Geant4 libraries, headers and support files will be installed. It must be supplied as an absolute path. The second argument to CMake is the path to the source directory.
* Additional arguments may be passed to CMake to activate optional components of Geant4, such as visualization drivers, or tune the build and install parameters.
* If you run CMake and decide afterwards you want to activate additional options, simply rerun CMake in the build directory, passing it the extra options plus the build directory path.
* Datasets are not required to be present to build Geant4, but may be required to run your application, depending on the physics models you use. If you wish to download and install the datasets automatically as part of your build of Geant4, simply add the option-DGEANT4_INSTALL_DATA=ON to the arguments passed to CMake.
* After the configuration has run, CMake will have generated Unix Makefiles for building Geant4. To run the build, simply execute make in the build directory
* Once the build has completed, you can install Geant4 to the directory you specified earlier in CMAKE_INSTALL_PREFIX
* additional options can be passed to CMake to adjust the way Geant4 is built and installed and to enable optional components. Some options enable components of Geant4 which require external software.If these options are enabled, the required software will be searched for, and hence there are also options which control where CMake should look for these packages.These options may be set by passing their name and value to the cmake command via -D flags.

===Standard Options===
* CMAKE_INSTALL_PREFIX installation prefix for Geant4. (default Unix: /usr/local)
* CMAKE_BUILD_TYPE : (DEFAULT : Release) Sets the type of build. It defaults to Release which gives a fully optimized build without debugging symbols.
* GEANT4_BUILD_MULTITHREADED : (DEFAULT : OFF) If set to ON, build Geant4 libraries with support for multithreading. At present, this is only supported on Unix systems. (Requires: Compiler/C++ Standard Library with support for C++ Threading Support)
* GEANT4_INSTALL_DATA : (DEFAULT : OFF) to ON, download and install any Geant4 datasets missing from GEANT4_INSTALL_DATADIR.
* GEANT4_INSTALL_DATADIR : (DEFAULT : CMAKE_INSTALL_DATAROOTDIR) Installation directory for Geant4 datasets.
* GEANT4_USE_G3TOG4 : (DEFAULT : OFF) If set to ON, build the G3ToG4 library for reading ASCII call list files generated from Geant3 geometries.
* GEANT4_USE_GDML : (DEFAULT : OFF) If set to ON, build the G4persistency library with support for GDML.
* GEANT4_USE_QT (DEFAULT : OFF) If set to ON, build Qt4/5 User Interface and Visualization drivers. Requires: Qt4 or Qt5 and OpenGL libraries and headers. Qt5 will be searched for first. If it is not found Qt4 will be searched for. This behaviour can be adjusted through the advanced option GEANT4_FORCE_QT4.
* GEANT4_USE_OPENGL_X11 (DEFAULT : OFF, Unix Only)If set to ON, build the X11 OpenGL visualization driver.
* GEANT4_USE_RAYTRACER_X11 (DEFAULT : OFF, Unix only) If set to ON, build RayTracer visualization driver with X11 support.
* GEANT4_USE_INVENTOR (DEFAULT : OFF) If set to ON, build the OpenInventor visualization driver.
* GEANT4_USE_XM (DEFAULT : OFF, Unix Only) If set to ON, build Motif User Interface and Visualization drivers.
* GEANT4_USE_SYSTEM_CLHEP (DEFAULT : OFF) If set to ON, build Geant4 with an external install of CLHEP. You should not set this unless your usage of Geant4 mandates a specific external CLHEP installation (e.g. if your project’s software uses CLHEP in other tools and requires consistent use of the same CLHEP across the software stack).Requires: CLHEP libraries and headers. See the CLHEP_XXX and CMAKE_PREFIX_PATHAdvanced Options if CMake has trouble locating CLHEP.
* GEANT4_USE_SYSTEM_EXPAT (DEFAULT : ON) If set to ON, build Geant4 with an external install of Expat.
* GEANT4_USE_SYSTEM_ZLIB (DEFAULT : OFF) If set to ON, build Geant4 with an external install of zlib.

===Advanced Options===
* http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
*Among the advanced, I list only the one used in the easy build recipe file https://bitbucket.org/jairhul/accsoft-eb/src/5f285502686e28d656dfa4d3ce1408d2551ee5d1/Geant4-10.00.p04-GCC-4.9.3.eb?at=master&fileviewer=file-view-default
** EXPAT_LIBRARY If CMake cannot locate your external EXPAT installation, set this to the full path to the EXPAT library, e.g. /usr/lib/libexpat.so
** EXPAT_INCLUDE_DIR If CMake cannot locate your external EXPAT installation, set this to the directory containing the EXPAT headers (e.g. if you have /foobar/expat.h, then set this to /foobar).

== EasyBuild ==
===Reference===
https://media.readthedocs.org/pdf/caylo-easybuild/docfix/caylo-easybuild.pdf
EasyBuild is a flexible software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way. Divert from the standard configure / make / make install with custom procedures. Allows for easily reproducing previous builds.

===EasyBuild===
EasyBuild consists of a collection of Python modules and packages that interact with each other, dynamically picking up additional Python modules as needed for building and installing a (stack of) software package(s) specified via simple specification files

In EasyBuild terminology: the EasyBuild framework leverages easyblocks to automatically build and install software using particular compiler toolchains, as specified by one or multiple easyconfig files.

The EasyBuild framework provides functionality commonly needed when installing scientific software on HPC systems. For example, it deals with downloading, unpacking and patching of sources, loading module files for dependencies, setting up the build environment, autonomously running (interactive) shell commands, creating module files that match the specification files, etc.
The implementation of a particular software build and install procedure is done in a Python module, which is aptly referred to as an easyblock.
For each software package being built, the EasyBuild framework will determine which easyblock should be used, based on the name of the software package or the value of the easyblock specification parameter (see Easyblock specification). Since EasyBuild v2.0, an easyblock must be specified in case no matching easyblock is found based on the software name (cfr. Automagic fallback to ConfigureMake).
The specification files that are supplied to EasyBuild are referred to as easyconfig files (or simply easyconfigs .eb), which are basically plain text files containing (mostly) key-value assignments for build parameters supported by the framework, also referred to as easyconfig parameters (see Writing easyconfig files: the basics for more information).
An easyconfig file serves as a build specification for EasyBuild.
Easyconfigs typically follow a (fixed) strict naming scheme, i.e. <name>-<version>[-<toolchain>][<versionsuffix>
A handful of easyconfig parameters are mandatory:
* name, version: specify what software (version) to build
* homepage, description: metadata (used for module help)
* toolchain: specifies name and version of compiler toolchain to use
* Using external modules as dependencies
* Configure/build/install command options:
** configopts: options for configure command
** preconfigopts: options used as prefix for configure command
** buildopts, prebuildopts: options for build command
** installopts, preinstallopts: options for install

EasyBuild allows to easily reproduce software builds/installations and install multiple versions.

==Environment Modules==
===Reference===
http://modules.sourceforge.net/man/module.html
===Environment Modules===
Typically users initialize their environment when they log in by setting environment information for every application they will reference during the session.
The Environment Modules system is a tool to help users manage their Unix or Linux shell environment, by allowing groups of related environment-variable settings to be made or removed dynamically. The modules system is based on modulefiles,[6] which specify groups of environment settings that need to be made together. Modulefiles can be installed in a central location for general use, or in a user directory for personal use. Environment Modules modulefiles are written in the Tcl (Tool Command Language) and are interpreted by the modulecmd program via the module[7] user interface. The key advantage of Environment Modules is that it is shell independent and supports all major shells such as bash, ksh, zsh, sh, tcsh, and csh. The second key advantage is that it allows to use multiple versions of the program or package from the same account by just loading proper module. modulefiles are created on per application per version basis. They can be dynamically loaded, unloaded, or switched. Along with the capability of using multiple versions of the same software it also can be used to implement site policies regarding the access and use of applications.
https://modules.readthedocs.io/en/latest/FAQ.html

===New commands for centOS 7 ===

# Tell module tool where the modules are
module use /unix/pbt/software/eb/CentOS7/modules/*
# Load Geant4
# This loads all the necessary libraries like CLHEP, Qt5, etc.
module load Geant4/10.04.p02-GCC-4.9.3
source $EBROOTGEANT4/bin/geant4.sh
module load CMake/3.5.2-GCC-4.9.3

'''what does this mean'''
* module use [-a|--append] directory…
Prepend one or more directories to the MODULEPATH environment variable.
* load modulefile...
Load modulefile(s) into the shell environment.

when Geant4 is installed as a module
To see which versions are available, execute: module avail geant4
To then use a particular version, execute for example
module load geant4/10.02.p03
After you load the module, you can see where the software is installed by executing
echo $EBROOTGEANT4

Set GEANT environment variables
source $EBROOTGEANT4/bin/geant4.sh

Software/EasyBuild

2019-02-21T09:44:57Z

MatthieuHentz: /* Use modules */

=Instructions for using software with EasyBuild.=

= Compile BDSIM =
=== James Chappell's question ===
Dear Laurie,

I’m trying to learn how to use the version of easy build that you setup to run some BDSIM simulations for AWAKE. I have a few questions for you.

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

2) Having looked through your eb files in /unix/pbt/software/eb/accsoft-eb-nevay, I can see that you define sources using a url and I’m assuming that in the process of compiling all the software, this source code is downloaded and then compiled. Is there a way to define the source from within the local system directories and then compile this version? Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

Sorry for all the questions, I’m just trying to get my head around how it all works! Thanks for your help,
James.

=== The answer ===
Dear James,

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

My bad -> I'll recompile it with awake on. I need a time slot when you won't be using it as it'll disappear during the rebuilding - pretty bad if someone's running jobs when I do it! Let me know when I can do this - only takes about 15 mins.

Is there a way to define the source from within the local system directories and then compile this version?
It looks like the system really only is designed to download things from what I can read online... Despite the url, it caches them in a sources directory, so if it was already there it would use that (a fudge though).

Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

We have so far in the Bdsim-1.1-GCC-4.9.4.eb easy build file:

configopts = ' -DCMAKE_CXX_FLAGS="-std=c++11"'

We can add to this:

configopts += '-DUSE_AWAKE=ON'

See ROOT-v6.08.06-GCC-4.9.3-Python-2.7.12.eb for lots of examples.

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

If you're modifying the C++ often I'd make your own build of BDSIM + AWAKE separately from the EasyBuild system and not bother trying to package it into a module. The module system is really for things that won't change.

You can get all the dependencies from EasyBuild and then make your own independent build of whatever you want. I think this was done for Gate for example and we do this just now for our nightly testing of BDSIM. I load the BDSIM module, then unload it - this leaves all the dependencies loaded.

module purge
module load Bdsim
module unload Bdsim
module list

# still has all libraries loaded including Geant4 etc..
# somewhere else, make a build dir and build from your source.

When you load Geant4 and ROOT, note you should really source their setup scripts:

source $EBROOTGEANT4/bin/geant4.sh
source $EBROOTROOT/bin/thisroot.sh

--->>> You should make a copy of the bdsim.sh script that does most of this and just add the unload to the end.

then...

mkdir build
cd build
cmake ../relative/path/to/bdsim -DCMAKE_CXX_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/g++
-DCMAKE_C_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/gcc
-DCLHEP_LIBRARY_DIR=/home/accsoft/SL68/software/CLHEP/2.3.1.1-GCC-4.9.3/lib/
make -j10

* paths are example on my system at RHUL

Normally you wouldn't have to specify these options as CMake would find these on its own. However, practice shows it often finds a system compiler before it finds the one in easybuild (this is required to be the same for all the software to be consistent), so we explicitly specify them.

Note once a build is make and it finds a compiler, the compiler can never be changed in CMake. You must wipe the build directory contents to try again.

How do you use this new build? Simple, you must have the same environment loaded as when it was built (so hence the modified bdsim.sh) then execute from that directory. This could easily be done in your shell script for the simulation or via an alias (fictional paths here as an example)...

alias bdsim-awake-devel="/home/jchappell/my-builds/build123/bdsim"
bdsim-awake-devel --help

Note if you haven't changed the output or the analysis you can just use the regular bdsim from easybuild if you do any analysis later on the output. If you have changed it, you'll need to use your build and modify your bdsim.sh - pretty clear where to change for the analysis stuff.

Ok, hopefully that should help you both get the build working and help you understand.

Cheers,
Laurie

=== Additional info ===
Hello,

Ok, I've updated the general BDSIM build in easy build now. It should have the AWAKE module turned on. Note, this is the code of v1.1 and not the latest develop version.

I made an example environment only script in:

/unix/pbt/software/eb/bdsim-own-build.sh

I tried making my own build and it worked just fine. I only needed to do also:

module load CMake

to get a more up to date version. From my home dir:

mkdir build-custom-build
cd build-custom-build
cmake ../bdsim
# check compiler is GCC4.9.3 form easy build, and ROOT and Geant4 and CLHEP -> all ok
ccmake .
turn on USE_AWAKE
<c>
<g>
make -j15
works!

It's in /home/lnevay/bdsim-custom-build

Cheers,
Laurie

= Use modules =
=== Set path to modules ===
Before modules can be accessed, the module tool needs to be given the path to the desired set of modules. For example,

<pre>
module use /unix/pbt/software/eb/CentOS7/modules/*
</pre>

=== List available modules ===
You can then list all modules available with

<pre>module avail</pre>

or check what versions of a given module are available using

<pre>module avail [modulename]</pre>

For example,

<pre>module avail Geant4</pre>

prints out the following

<pre>
-------------------------------------------------------------------- /unix/pbt/software/eb/CentOS7/modules/phys --------------------------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3 Geant4/10.04.p02-mt-GCC-4.9.3

-------------------------------------------------------------------- /unix/pbt/software/eb/CentOS7/modules/all ---------------------------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3 Geant4/10.04.p02-mt-GCC-4.9.3
</pre>

=== Load modules ===
Load a given module using

<pre>
module load Geant4/10.04.p02-GCC-4.9.3
</pre>

and where required source the relevant environment variables, e.g.

<pre>
source $EBROOTGEANT4/bin/geant4.sh
</pre>

=== Show loaded modules ===
To see what's loaded, use

<pre>
module list
</pre>

which prints, for example,

1) GCC/4.9.3 9) bzip2/1.0.6-GCC-4.9.3 17) Mesa/11.2.1-GCC-4.9.3
2) expat/2.1.1-GCC-4.9.3 10) freetype/2.6.3-GCC-4.9.3 18) libGLU/9.0.0-GCC-4.9.3-Mesa-11.2.1
3) CLHEP/2.3.3.0-GCC-4.9.3 11) fontconfig/2.11.95-GCC-4.9.3 19) libffi/3.2.1-GCC-4.9.3
4) cURL/7.44.0-GCC-4.9.3 12) X11/20160819-GCC-4.9.3 20) gettext/0.19.6-GCC-4.9.3
5) Xerces-C++/3.1.2-GCC-4.9.3 13) libdrm/2.4.68-GCC-4.9.3 21) PCRE/8.38-GCC-4.9.3
6) zlib/1.2.8-GCC-4.9.3 14) ncurses/6.0-GCC-4.9.3 22) GLib/2.49.5-GCC-4.9.3
7) libxml2/2.9.3-GCC-4.9.3 15) LLVM/3.8.0-GCC-4.9.3 23) Qt5/5.7.0-GCC-4.9.3
8) libpng/1.6.21-GCC-4.9.3 16) eudev/3.1.5-GCC-4.9.3 24) Geant4/10.04.p02-GCC-4.9.3

=== Remove loaded modules ===
To get rid of all loaded modules:

<pre>
module purge
</pre>

= Install modules =

=== Installing modules ===

Example command to install module:

eb Bdsim-1.2.1-GCC-4.9.3.eb --prefix=/unix/pbt/software/eb/CentOS7 -robot --robot-path=./: --modules-tool=EnvironmentModulesC --module-syntax=Tcl --allow-modules-tool-mismatch --optarch=GENERIC --buildpath=/tmp/easybuild/

Two types of the same version (i.e. one single-threaded and one multi-threaded) can be installed side-by-side. To get EasyBuild to not replace a current version, the module can be renamed in the EasyBuild file. When a module is built, EasyBuild checks for modules of the same name and version and if one already exists, the build won't complete. If the options are updated, the build will be updated.

When you build a module, you should do the `module use` command so it can find the others already available (it checks the requirements of the one you're building) and also the EasyBuild module itself. You should have a clean environment with no other modules loaded.

/unix/pbt/software/eb/easybuildonly.sh

seems to provide this.

I tried my example and found you should edit the version rather than the name for it to work. I've done this and it's setup now. You can import it as.

module load Geant4/10.04.p02-mt-GCC-4.9.3

Cheers,
Laurie

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

=== The question ===
Dear Laurie,

I'm trying to compile a multi-threaded version of Geant4 using EasyBuild but would like to make sure I don't break everything.

Looking at the easyconfig file for the most recent Geant4 version installed on our system and the directory structure of the `eb/CentOS7/software` directory, I can see that the location of the installation depends on the name and the version of the package and on the toolchain used to compile it. This leads me to believe that if I were to set `GEANT4_BUILD_MULTITHREADED=ON` in the easyconfig file and run the following command,

eb Geant4-10.04.p02-GCC-4.9.3.eb --robot

it would overwrite the current (non-mutli-threaded) version of Geant4. I have two questions relating to this:

1) Am I correct in thinking it would overwrite it?
2) If so, would you mind showing me how I can compile it alongside the existing version?

The best solution I could come up with would be to install it in a different location using the command line option `--installpath-software=INSTALLPATH-SOFTWARE` but I'm unsure if that breaks compatibility with the modules tool and I'd also like to avoid overwriting the existing easyconfig file, which I don't know how to ensure.

= Note on how to install Geant4 on Mac =
Hello,

Ah we tried it briefly on mac, but it was a little frustrating so we stuck with our usual way, which is macports. For your mac, I'd recommend that. I know it's different than EasyBuild, but it's generally easier for mac and for scientific software.

https://www.macports.org

Before getting it you should xcode + the command line tools (from app store) installed (test is you can run 'make' on the terminal). You should also have XQuartz (https://www.xquartz.org) installed for X11 support. Then download and install the macports software (small, little mac installer). Gives you a command on the terminal 'port'. Everything is by default installed in /opt/local so you can nuke that directory if you don't want it anymore. It works by adding a line to your profile that adds that directory to your paths.

port search packagename

sudo port install py27-matplotlib

I'd install everything up to CLHEP from macports, then make my own geant4 installation. I think there is a geant4 port on there, but I haven't used it. For the geant4 installation you download hte source somewhere, make a build directory (outside the source) and then an install directory. Usual CMake pattern.

I'd start with a high level package in macports and it'll work out all the requirements. Like...

sudo port install clhep qt5 py27-matplotlib

I do everything apart from geant4 and bdsim through macports.

Hope that helps,
Cheers,
Laurie

= Summary from Raffy about EasyBuild, Environment Modules and GEANT4 =
== Geant4 ==
===Reference===
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/gettingstarted.html
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
===GEANT4 OS/Software Prerequisites===
the following source/software must be present to build Geant4:
* Geant4 Toolkit Source Code.
* C++ Compiler and Standard Library supporting the C++11 Standard
* Linux (Scientific Linux CERN 6 and Linux CentOS 7 ): GNU Compiler Collection 4.8.5 or higher ( recommended to use the GCC compiler supplied by the package management system of your distribution unless this does not meet the minimum version requirement. The minimum required versions of GCC may be installed on SLC6 systems via the freeDeveloper Toolset)
* CMake 3.3 or higher (as provided through the package management system of your distribution, unless it does not meet the minimum version requirement.)
The main requirement is that the system has a GCC of sufficient version to support C++11 installed.

===User Interface and Visualization Drivers===
* Qt User Interface and Visualization
** Qt4 (>=4.6) or Qt5 headers and libraries
* OpenGL or MesaGL headers and libraries.
** X11 OpenGL Visualization (Linux and macOS)
* Open Inventor Visualization
* Coin3D libraries and headers with SoXt(SoWin) graphics binding on Linux/macOS(Windows), compiled against the C++11 standard.
* Motif User Interface and Visualization (Linux and macOS)

===Standard GEANT4 Building and Installation===
* Unpack the Geant4 source package geant4.10.05.tar.gz to a location of your choice: source directory /path/to/geant4.10.05
* create a directory in which to configure and run the build and store the build products /path/to/geant4.10.05-build
* $ cmake -DCMAKE_INSTALL_PREFIX=/path/to/geant4.10.05-install /path/to/geant4.10.05
* CMake Variable CMAKE_INSTALL_PREFIX is used to set the install directory, the directory under which the Geant4 libraries, headers and support files will be installed. It must be supplied as an absolute path. The second argument to CMake is the path to the source directory.
* Additional arguments may be passed to CMake to activate optional components of Geant4, such as visualization drivers, or tune the build and install parameters.
* If you run CMake and decide afterwards you want to activate additional options, simply rerun CMake in the build directory, passing it the extra options plus the build directory path.
* Datasets are not required to be present to build Geant4, but may be required to run your application, depending on the physics models you use. If you wish to download and install the datasets automatically as part of your build of Geant4, simply add the option-DGEANT4_INSTALL_DATA=ON to the arguments passed to CMake.
* After the configuration has run, CMake will have generated Unix Makefiles for building Geant4. To run the build, simply execute make in the build directory
* Once the build has completed, you can install Geant4 to the directory you specified earlier in CMAKE_INSTALL_PREFIX
* additional options can be passed to CMake to adjust the way Geant4 is built and installed and to enable optional components. Some options enable components of Geant4 which require external software.If these options are enabled, the required software will be searched for, and hence there are also options which control where CMake should look for these packages.These options may be set by passing their name and value to the cmake command via -D flags.

===Standard Options===
* CMAKE_INSTALL_PREFIX installation prefix for Geant4. (default Unix: /usr/local)
* CMAKE_BUILD_TYPE : (DEFAULT : Release) Sets the type of build. It defaults to Release which gives a fully optimized build without debugging symbols.
* GEANT4_BUILD_MULTITHREADED : (DEFAULT : OFF) If set to ON, build Geant4 libraries with support for multithreading. At present, this is only supported on Unix systems. (Requires: Compiler/C++ Standard Library with support for C++ Threading Support)
* GEANT4_INSTALL_DATA : (DEFAULT : OFF) to ON, download and install any Geant4 datasets missing from GEANT4_INSTALL_DATADIR.
* GEANT4_INSTALL_DATADIR : (DEFAULT : CMAKE_INSTALL_DATAROOTDIR) Installation directory for Geant4 datasets.
* GEANT4_USE_G3TOG4 : (DEFAULT : OFF) If set to ON, build the G3ToG4 library for reading ASCII call list files generated from Geant3 geometries.
* GEANT4_USE_GDML : (DEFAULT : OFF) If set to ON, build the G4persistency library with support for GDML.
* GEANT4_USE_QT (DEFAULT : OFF) If set to ON, build Qt4/5 User Interface and Visualization drivers. Requires: Qt4 or Qt5 and OpenGL libraries and headers. Qt5 will be searched for first. If it is not found Qt4 will be searched for. This behaviour can be adjusted through the advanced option GEANT4_FORCE_QT4.
* GEANT4_USE_OPENGL_X11 (DEFAULT : OFF, Unix Only)If set to ON, build the X11 OpenGL visualization driver.
* GEANT4_USE_RAYTRACER_X11 (DEFAULT : OFF, Unix only) If set to ON, build RayTracer visualization driver with X11 support.
* GEANT4_USE_INVENTOR (DEFAULT : OFF) If set to ON, build the OpenInventor visualization driver.
* GEANT4_USE_XM (DEFAULT : OFF, Unix Only) If set to ON, build Motif User Interface and Visualization drivers.
* GEANT4_USE_SYSTEM_CLHEP (DEFAULT : OFF) If set to ON, build Geant4 with an external install of CLHEP. You should not set this unless your usage of Geant4 mandates a specific external CLHEP installation (e.g. if your project’s software uses CLHEP in other tools and requires consistent use of the same CLHEP across the software stack).Requires: CLHEP libraries and headers. See the CLHEP_XXX and CMAKE_PREFIX_PATHAdvanced Options if CMake has trouble locating CLHEP.
* GEANT4_USE_SYSTEM_EXPAT (DEFAULT : ON) If set to ON, build Geant4 with an external install of Expat.
* GEANT4_USE_SYSTEM_ZLIB (DEFAULT : OFF) If set to ON, build Geant4 with an external install of zlib.

===Advanced Options===
* http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
*Among the advanced, I list only the one used in the easy build recipe file https://bitbucket.org/jairhul/accsoft-eb/src/5f285502686e28d656dfa4d3ce1408d2551ee5d1/Geant4-10.00.p04-GCC-4.9.3.eb?at=master&fileviewer=file-view-default
** EXPAT_LIBRARY If CMake cannot locate your external EXPAT installation, set this to the full path to the EXPAT library, e.g. /usr/lib/libexpat.so
** EXPAT_INCLUDE_DIR If CMake cannot locate your external EXPAT installation, set this to the directory containing the EXPAT headers (e.g. if you have /foobar/expat.h, then set this to /foobar).

== EasyBuild ==
===Reference===
https://media.readthedocs.org/pdf/caylo-easybuild/docfix/caylo-easybuild.pdf
EasyBuild is a flexible software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way. Divert from the standard configure / make / make install with custom procedures. Allows for easily reproducing previous builds.

===EasyBuild===
EasyBuild consists of a collection of Python modules and packages that interact with each other, dynamically picking up additional Python modules as needed for building and installing a (stack of) software package(s) specified via simple specification files

In EasyBuild terminology: the EasyBuild framework leverages easyblocks to automatically build and install software using particular compiler toolchains, as specified by one or multiple easyconfig files.

The EasyBuild framework provides functionality commonly needed when installing scientific software on HPC systems. For example, it deals with downloading, unpacking and patching of sources, loading module files for dependencies, setting up the build environment, autonomously running (interactive) shell commands, creating module files that match the specification files, etc.
The implementation of a particular software build and install procedure is done in a Python module, which is aptly referred to as an easyblock.
For each software package being built, the EasyBuild framework will determine which easyblock should be used, based on the name of the software package or the value of the easyblock specification parameter (see Easyblock specification). Since EasyBuild v2.0, an easyblock must be specified in case no matching easyblock is found based on the software name (cfr. Automagic fallback to ConfigureMake).
The specification files that are supplied to EasyBuild are referred to as easyconfig files (or simply easyconfigs .eb), which are basically plain text files containing (mostly) key-value assignments for build parameters supported by the framework, also referred to as easyconfig parameters (see Writing easyconfig files: the basics for more information).
An easyconfig file serves as a build specification for EasyBuild.
Easyconfigs typically follow a (fixed) strict naming scheme, i.e. <name>-<version>[-<toolchain>][<versionsuffix>
A handful of easyconfig parameters are mandatory:
* name, version: specify what software (version) to build
* homepage, description: metadata (used for module help)
* toolchain: specifies name and version of compiler toolchain to use
* Using external modules as dependencies
* Configure/build/install command options:
** configopts: options for configure command
** preconfigopts: options used as prefix for configure command
** buildopts, prebuildopts: options for build command
** installopts, preinstallopts: options for install

EasyBuild allows to easily reproduce software builds/installations and install multiple versions.

==Environment Modules==
===Reference===
http://modules.sourceforge.net/man/module.html
===Environment Modules===
Typically users initialize their environment when they log in by setting environment information for every application they will reference during the session.
The Environment Modules system is a tool to help users manage their Unix or Linux shell environment, by allowing groups of related environment-variable settings to be made or removed dynamically. The modules system is based on modulefiles,[6] which specify groups of environment settings that need to be made together. Modulefiles can be installed in a central location for general use, or in a user directory for personal use. Environment Modules modulefiles are written in the Tcl (Tool Command Language) and are interpreted by the modulecmd program via the module[7] user interface. The key advantage of Environment Modules is that it is shell independent and supports all major shells such as bash, ksh, zsh, sh, tcsh, and csh. The second key advantage is that it allows to use multiple versions of the program or package from the same account by just loading proper module. modulefiles are created on per application per version basis. They can be dynamically loaded, unloaded, or switched. Along with the capability of using multiple versions of the same software it also can be used to implement site policies regarding the access and use of applications.
https://modules.readthedocs.io/en/latest/FAQ.html

===New commands for centOS 7 ===

# Tell module tool where the modules are
module use /unix/pbt/software/eb/CentOS7/modules/*
# Load Geant4
# This loads all the necessary libraries like CLHEP, Qt5, etc.
module load Geant4/10.04.p02-GCC-4.9.3
source $EBROOTGEANT4/bin/geant4.sh
module load CMake/3.5.2-GCC-4.9.3

'''what does this mean'''
* module use [-a|--append] directory…
Prepend one or more directories to the MODULEPATH environment variable.
* load modulefile...
Load modulefile(s) into the shell environment.

when Geant4 is installed as a module
To see which versions are available, execute: module avail geant4
To then use a particular version, execute for example
module load geant4/10.02.p03
After you load the module, you can see where the software is installed by executing
echo $EBROOTGEANT4

Set GEANT environment variables
source $EBROOTGEANT4/bin/geant4.sh

Software/EasyBuild

2019-02-20T19:38:13Z

MatthieuHentz: /* Install modules */

=Instructions for using software with EasyBuild.=

= Compile BDSIM =
=== James Chappell's question ===
Dear Laurie,

I’m trying to learn how to use the version of easy build that you setup to run some BDSIM simulations for AWAKE. I have a few questions for you.

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

2) Having looked through your eb files in /unix/pbt/software/eb/accsoft-eb-nevay, I can see that you define sources using a url and I’m assuming that in the process of compiling all the software, this source code is downloaded and then compiled. Is there a way to define the source from within the local system directories and then compile this version? Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

Sorry for all the questions, I’m just trying to get my head around how it all works! Thanks for your help,
James.

=== The answer ===
Dear James,

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

My bad -> I'll recompile it with awake on. I need a time slot when you won't be using it as it'll disappear during the rebuilding - pretty bad if someone's running jobs when I do it! Let me know when I can do this - only takes about 15 mins.

Is there a way to define the source from within the local system directories and then compile this version?
It looks like the system really only is designed to download things from what I can read online... Despite the url, it caches them in a sources directory, so if it was already there it would use that (a fudge though).

Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

We have so far in the Bdsim-1.1-GCC-4.9.4.eb easy build file:

configopts = ' -DCMAKE_CXX_FLAGS="-std=c++11"'

We can add to this:

configopts += '-DUSE_AWAKE=ON'

See ROOT-v6.08.06-GCC-4.9.3-Python-2.7.12.eb for lots of examples.

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

If you're modifying the C++ often I'd make your own build of BDSIM + AWAKE separately from the EasyBuild system and not bother trying to package it into a module. The module system is really for things that won't change.

You can get all the dependencies from EasyBuild and then make your own independent build of whatever you want. I think this was done for Gate for example and we do this just now for our nightly testing of BDSIM. I load the BDSIM module, then unload it - this leaves all the dependencies loaded.

module purge
module load Bdsim
module unload Bdsim
module list

# still has all libraries loaded including Geant4 etc..
# somewhere else, make a build dir and build from your source.

When you load Geant4 and ROOT, note you should really source their setup scripts:

source $EBROOTGEANT4/bin/geant4.sh
source $EBROOTROOT/bin/thisroot.sh

--->>> You should make a copy of the bdsim.sh script that does most of this and just add the unload to the end.

then...

mkdir build
cd build
cmake ../relative/path/to/bdsim -DCMAKE_CXX_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/g++
-DCMAKE_C_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/gcc
-DCLHEP_LIBRARY_DIR=/home/accsoft/SL68/software/CLHEP/2.3.1.1-GCC-4.9.3/lib/
make -j10

* paths are example on my system at RHUL

Normally you wouldn't have to specify these options as CMake would find these on its own. However, practice shows it often finds a system compiler before it finds the one in easybuild (this is required to be the same for all the software to be consistent), so we explicitly specify them.

Note once a build is make and it finds a compiler, the compiler can never be changed in CMake. You must wipe the build directory contents to try again.

How do you use this new build? Simple, you must have the same environment loaded as when it was built (so hence the modified bdsim.sh) then execute from that directory. This could easily be done in your shell script for the simulation or via an alias (fictional paths here as an example)...

alias bdsim-awake-devel="/home/jchappell/my-builds/build123/bdsim"
bdsim-awake-devel --help

Note if you haven't changed the output or the analysis you can just use the regular bdsim from easybuild if you do any analysis later on the output. If you have changed it, you'll need to use your build and modify your bdsim.sh - pretty clear where to change for the analysis stuff.

Ok, hopefully that should help you both get the build working and help you understand.

Cheers,
Laurie

=== Additional info ===
Hello,

Ok, I've updated the general BDSIM build in easy build now. It should have the AWAKE module turned on. Note, this is the code of v1.1 and not the latest develop version.

I made an example environment only script in:

/unix/pbt/software/eb/bdsim-own-build.sh

I tried making my own build and it worked just fine. I only needed to do also:

module load CMake

to get a more up to date version. From my home dir:

mkdir build-custom-build
cd build-custom-build
cmake ../bdsim
# check compiler is GCC4.9.3 form easy build, and ROOT and Geant4 and CLHEP -> all ok
ccmake .
turn on USE_AWAKE
<c>
<g>
make -j15
works!

It's in /home/lnevay/bdsim-custom-build

Cheers,
Laurie

= Use modules =
As Simon says, I've made modules ('environment modules for unix') on the system using EasyBuild. I've installed the latest Geant4 for BDSIM (our application) but I've also now put on the latest patch version of all recent Geant4 versions (was quite easy). You can access them from a CentOS7 node (I'm presuming Simon's here):

module use /unix/pbt/software/eb/CentOS7/modules/*
module avail Geant4

.. this prints out the following

------------------------------------------------ /unix/pbt/software/eb/CentOS7/modules/all -------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3

module load Geant4/10.04.p02-GCC-4.9.3
source $EBROOTGEANT4/bin/geant4.sh

The last line sources the required environmental variable for Geant4 to be found. This will load the corresponding CLHEP version required (multiple available).

To see what's loaded:

module list

Currently Loaded Modulefiles:
1) GCC/4.9.3 9) bzip2/1.0.6-GCC-4.9.3 17) Mesa/11.2.1-GCC-4.9.3
2) expat/2.1.1-GCC-4.9.3 10) freetype/2.6.3-GCC-4.9.3 18) libGLU/9.0.0-GCC-4.9.3-Mesa-11.2.1
3) CLHEP/2.3.3.0-GCC-4.9.3 11) fontconfig/2.11.95-GCC-4.9.3 19) libffi/3.2.1-GCC-4.9.3
4) cURL/7.44.0-GCC-4.9.3 12) X11/20160819-GCC-4.9.3 20) gettext/0.19.6-GCC-4.9.3
5) Xerces-C++/3.1.2-GCC-4.9.3 13) libdrm/2.4.68-GCC-4.9.3 21) PCRE/8.38-GCC-4.9.3
6) zlib/1.2.8-GCC-4.9.3 14) ncurses/6.0-GCC-4.9.3 22) GLib/2.49.5-GCC-4.9.3
7) libxml2/2.9.3-GCC-4.9.3 15) LLVM/3.8.0-GCC-4.9.3 23) Qt5/5.7.0-GCC-4.9.3
8) libpng/1.6.21-GCC-4.9.3 16) eudev/3.1.5-GCC-4.9.3 24) Geant4/10.04.p02-GCC-4.9.3

To get rid of these loaded modules:

module purge

To see all modules:

module avail

There's the latest ROOT 6 on there too as well as a few versions of CLHEP (note Geant4 versions use different ones).

The module use and module load should be done in each new environment / terminal.

I tried making an easybuild recipe for GATE, but it fails at the end (I even tried a patch to remove their hard-coded make files for gjm) but still no success.

= Install modules =

=== Installing modules ===

Example command to install module:

eb Bdsim-1.2.1-GCC-4.9.3.eb --prefix=/unix/pbt/software/eb/CentOS7 -robot --robot-path=./: --modules-tool=EnvironmentModulesC --module-syntax=Tcl --allow-modules-tool-mismatch --optarch=GENERIC --buildpath=/tmp/easybuild/

Two types of the same version (i.e. one single-threaded and one multi-threaded) can be installed side-by-side. To get EasyBuild to not replace a current version, the module can be renamed in the EasyBuild file. When a module is built, EasyBuild checks for modules of the same name and version and if one already exists, the build won't complete. If the options are updated, the build will be updated.

When you build a module, you should do the `module use` command so it can find the others already available (it checks the requirements of the one you're building) and also the EasyBuild module itself. You should have a clean environment with no other modules loaded.

/unix/pbt/software/eb/easybuildonly.sh

seems to provide this.

I tried my example and found you should edit the version rather than the name for it to work. I've done this and it's setup now. You can import it as.

module load Geant4/10.04.p02-mt-GCC-4.9.3

Cheers,
Laurie

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

=== The question ===
Dear Laurie,

I'm trying to compile a multi-threaded version of Geant4 using EasyBuild but would like to make sure I don't break everything.

Looking at the easyconfig file for the most recent Geant4 version installed on our system and the directory structure of the `eb/CentOS7/software` directory, I can see that the location of the installation depends on the name and the version of the package and on the toolchain used to compile it. This leads me to believe that if I were to set `GEANT4_BUILD_MULTITHREADED=ON` in the easyconfig file and run the following command,

eb Geant4-10.04.p02-GCC-4.9.3.eb --robot

it would overwrite the current (non-mutli-threaded) version of Geant4. I have two questions relating to this:

1) Am I correct in thinking it would overwrite it?
2) If so, would you mind showing me how I can compile it alongside the existing version?

The best solution I could come up with would be to install it in a different location using the command line option `--installpath-software=INSTALLPATH-SOFTWARE` but I'm unsure if that breaks compatibility with the modules tool and I'd also like to avoid overwriting the existing easyconfig file, which I don't know how to ensure.

= Note on how to install Geant4 on Mac =
Hello,

Ah we tried it briefly on mac, but it was a little frustrating so we stuck with our usual way, which is macports. For your mac, I'd recommend that. I know it's different than EasyBuild, but it's generally easier for mac and for scientific software.

https://www.macports.org

Before getting it you should xcode + the command line tools (from app store) installed (test is you can run 'make' on the terminal). You should also have XQuartz (https://www.xquartz.org) installed for X11 support. Then download and install the macports software (small, little mac installer). Gives you a command on the terminal 'port'. Everything is by default installed in /opt/local so you can nuke that directory if you don't want it anymore. It works by adding a line to your profile that adds that directory to your paths.

port search packagename

sudo port install py27-matplotlib

I'd install everything up to CLHEP from macports, then make my own geant4 installation. I think there is a geant4 port on there, but I haven't used it. For the geant4 installation you download hte source somewhere, make a build directory (outside the source) and then an install directory. Usual CMake pattern.

I'd start with a high level package in macports and it'll work out all the requirements. Like...

sudo port install clhep qt5 py27-matplotlib

I do everything apart from geant4 and bdsim through macports.

Hope that helps,
Cheers,
Laurie

= Summary from Raffy about EasyBuild, Environment Modules and GEANT4 =
== Geant4 ==
===Reference===
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/gettingstarted.html
http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
===GEANT4 OS/Software Prerequisites===
the following source/software must be present to build Geant4:
* Geant4 Toolkit Source Code.
* C++ Compiler and Standard Library supporting the C++11 Standard
* Linux (Scientific Linux CERN 6 and Linux CentOS 7 ): GNU Compiler Collection 4.8.5 or higher ( recommended to use the GCC compiler supplied by the package management system of your distribution unless this does not meet the minimum version requirement. The minimum required versions of GCC may be installed on SLC6 systems via the freeDeveloper Toolset)
* CMake 3.3 or higher (as provided through the package management system of your distribution, unless it does not meet the minimum version requirement.)
The main requirement is that the system has a GCC of sufficient version to support C++11 installed.

===User Interface and Visualization Drivers===
* Qt User Interface and Visualization
** Qt4 (>=4.6) or Qt5 headers and libraries
* OpenGL or MesaGL headers and libraries.
** X11 OpenGL Visualization (Linux and macOS)
* Open Inventor Visualization
* Coin3D libraries and headers with SoXt(SoWin) graphics binding on Linux/macOS(Windows), compiled against the C++11 standard.
* Motif User Interface and Visualization (Linux and macOS)

===Standard GEANT4 Building and Installation===
* Unpack the Geant4 source package geant4.10.05.tar.gz to a location of your choice: source directory /path/to/geant4.10.05
* create a directory in which to configure and run the build and store the build products /path/to/geant4.10.05-build
* $ cmake -DCMAKE_INSTALL_PREFIX=/path/to/geant4.10.05-install /path/to/geant4.10.05
* CMake Variable CMAKE_INSTALL_PREFIX is used to set the install directory, the directory under which the Geant4 libraries, headers and support files will be installed. It must be supplied as an absolute path. The second argument to CMake is the path to the source directory.
* Additional arguments may be passed to CMake to activate optional components of Geant4, such as visualization drivers, or tune the build and install parameters.
* If you run CMake and decide afterwards you want to activate additional options, simply rerun CMake in the build directory, passing it the extra options plus the build directory path.
* Datasets are not required to be present to build Geant4, but may be required to run your application, depending on the physics models you use. If you wish to download and install the datasets automatically as part of your build of Geant4, simply add the option-DGEANT4_INSTALL_DATA=ON to the arguments passed to CMake.
* After the configuration has run, CMake will have generated Unix Makefiles for building Geant4. To run the build, simply execute make in the build directory
* Once the build has completed, you can install Geant4 to the directory you specified earlier in CMAKE_INSTALL_PREFIX
* additional options can be passed to CMake to adjust the way Geant4 is built and installed and to enable optional components. Some options enable components of Geant4 which require external software.If these options are enabled, the required software will be searched for, and hence there are also options which control where CMake should look for these packages.These options may be set by passing their name and value to the cmake command via -D flags.

===Standard Options===
* CMAKE_INSTALL_PREFIX installation prefix for Geant4. (default Unix: /usr/local)
* CMAKE_BUILD_TYPE : (DEFAULT : Release) Sets the type of build. It defaults to Release which gives a fully optimized build without debugging symbols.
* GEANT4_BUILD_MULTITHREADED : (DEFAULT : OFF) If set to ON, build Geant4 libraries with support for multithreading. At present, this is only supported on Unix systems. (Requires: Compiler/C++ Standard Library with support for C++ Threading Support)
* GEANT4_INSTALL_DATA : (DEFAULT : OFF) to ON, download and install any Geant4 datasets missing from GEANT4_INSTALL_DATADIR.
* GEANT4_INSTALL_DATADIR : (DEFAULT : CMAKE_INSTALL_DATAROOTDIR) Installation directory for Geant4 datasets.
* GEANT4_USE_G3TOG4 : (DEFAULT : OFF) If set to ON, build the G3ToG4 library for reading ASCII call list files generated from Geant3 geometries.
* GEANT4_USE_GDML : (DEFAULT : OFF) If set to ON, build the G4persistency library with support for GDML.
* GEANT4_USE_QT (DEFAULT : OFF) If set to ON, build Qt4/5 User Interface and Visualization drivers. Requires: Qt4 or Qt5 and OpenGL libraries and headers. Qt5 will be searched for first. If it is not found Qt4 will be searched for. This behaviour can be adjusted through the advanced option GEANT4_FORCE_QT4.
* GEANT4_USE_OPENGL_X11 (DEFAULT : OFF, Unix Only)If set to ON, build the X11 OpenGL visualization driver.
* GEANT4_USE_RAYTRACER_X11 (DEFAULT : OFF, Unix only) If set to ON, build RayTracer visualization driver with X11 support.
* GEANT4_USE_INVENTOR (DEFAULT : OFF) If set to ON, build the OpenInventor visualization driver.
* GEANT4_USE_XM (DEFAULT : OFF, Unix Only) If set to ON, build Motif User Interface and Visualization drivers.
* GEANT4_USE_SYSTEM_CLHEP (DEFAULT : OFF) If set to ON, build Geant4 with an external install of CLHEP. You should not set this unless your usage of Geant4 mandates a specific external CLHEP installation (e.g. if your project’s software uses CLHEP in other tools and requires consistent use of the same CLHEP across the software stack).Requires: CLHEP libraries and headers. See the CLHEP_XXX and CMAKE_PREFIX_PATHAdvanced Options if CMake has trouble locating CLHEP.
* GEANT4_USE_SYSTEM_EXPAT (DEFAULT : ON) If set to ON, build Geant4 with an external install of Expat.
* GEANT4_USE_SYSTEM_ZLIB (DEFAULT : OFF) If set to ON, build Geant4 with an external install of zlib.

===Advanced Options===
* http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/installguide.html
*Among the advanced, I list only the one used in the easy build recipe file https://bitbucket.org/jairhul/accsoft-eb/src/5f285502686e28d656dfa4d3ce1408d2551ee5d1/Geant4-10.00.p04-GCC-4.9.3.eb?at=master&fileviewer=file-view-default
** EXPAT_LIBRARY If CMake cannot locate your external EXPAT installation, set this to the full path to the EXPAT library, e.g. /usr/lib/libexpat.so
** EXPAT_INCLUDE_DIR If CMake cannot locate your external EXPAT installation, set this to the directory containing the EXPAT headers (e.g. if you have /foobar/expat.h, then set this to /foobar).

== EasyBuild ==
===Reference===
https://media.readthedocs.org/pdf/caylo-easybuild/docfix/caylo-easybuild.pdf
EasyBuild is a flexible software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way. Divert from the standard configure / make / make install with custom procedures. Allows for easily reproducing previous builds.

===EasyBuild===
EasyBuild consists of a collection of Python modules and packages that interact with each other, dynamically picking up additional Python modules as needed for building and installing a (stack of) software package(s) specified via simple specification files

In EasyBuild terminology: the EasyBuild framework leverages easyblocks to automatically build and install software using particular compiler toolchains, as specified by one or multiple easyconfig files.

The EasyBuild framework provides functionality commonly needed when installing scientific software on HPC systems. For example, it deals with downloading, unpacking and patching of sources, loading module files for dependencies, setting up the build environment, autonomously running (interactive) shell commands, creating module files that match the specification files, etc.
The implementation of a particular software build and install procedure is done in a Python module, which is aptly referred to as an easyblock.
For each software package being built, the EasyBuild framework will determine which easyblock should be used, based on the name of the software package or the value of the easyblock specification parameter (see Easyblock specification). Since EasyBuild v2.0, an easyblock must be specified in case no matching easyblock is found based on the software name (cfr. Automagic fallback to ConfigureMake).
The specification files that are supplied to EasyBuild are referred to as easyconfig files (or simply easyconfigs .eb), which are basically plain text files containing (mostly) key-value assignments for build parameters supported by the framework, also referred to as easyconfig parameters (see Writing easyconfig files: the basics for more information).
An easyconfig file serves as a build specification for EasyBuild.
Easyconfigs typically follow a (fixed) strict naming scheme, i.e. <name>-<version>[-<toolchain>][<versionsuffix>
A handful of easyconfig parameters are mandatory:
* name, version: specify what software (version) to build
* homepage, description: metadata (used for module help)
* toolchain: specifies name and version of compiler toolchain to use
* Using external modules as dependencies
* Configure/build/install command options:
** configopts: options for configure command
** preconfigopts: options used as prefix for configure command
** buildopts, prebuildopts: options for build command
** installopts, preinstallopts: options for install

EasyBuild allows to easily reproduce software builds/installations and install multiple versions.

==Environment Modules==
===Reference===
http://modules.sourceforge.net/man/module.html
===Environment Modules===
Typically users initialize their environment when they log in by setting environment information for every application they will reference during the session.
The Environment Modules system is a tool to help users manage their Unix or Linux shell environment, by allowing groups of related environment-variable settings to be made or removed dynamically. The modules system is based on modulefiles,[6] which specify groups of environment settings that need to be made together. Modulefiles can be installed in a central location for general use, or in a user directory for personal use. Environment Modules modulefiles are written in the Tcl (Tool Command Language) and are interpreted by the modulecmd program via the module[7] user interface. The key advantage of Environment Modules is that it is shell independent and supports all major shells such as bash, ksh, zsh, sh, tcsh, and csh. The second key advantage is that it allows to use multiple versions of the program or package from the same account by just loading proper module. modulefiles are created on per application per version basis. They can be dynamically loaded, unloaded, or switched. Along with the capability of using multiple versions of the same software it also can be used to implement site policies regarding the access and use of applications.
https://modules.readthedocs.io/en/latest/FAQ.html

===New commands for centOS 7 ===

# Tell module tool where the modules are
module use /unix/pbt/software/eb/CentOS7/modules/*
# Load Geant4
# This loads all the necessary libraries like CLHEP, Qt5, etc.
module load Geant4/10.04.p02-GCC-4.9.3
source $EBROOTGEANT4/bin/geant4.sh
module load CMake/3.5.2-GCC-4.9.3

'''what does this mean'''
* module use [-a|--append] directory…
Prepend one or more directories to the MODULEPATH environment variable.
* load modulefile...
Load modulefile(s) into the shell environment.

when Geant4 is installed as a module
To see which versions are available, execute: module avail geant4
To then use a particular version, execute for example
module load geant4/10.02.p03
After you load the module, you can see where the software is installed by executing
echo $EBROOTGEANT4

Set GEANT environment variables
source $EBROOTGEANT4/bin/geant4.sh

ELogs/MatthieuHentz

2019-02-19T12:19:11Z

MatthieuHentz: /* Lower priority */

Electronic Log for Matthieu Hentz

== To Do ==
=== Urgent ===
* Nada

=== Higher priority ===
* Work on progress report
* Write code for benchmarking metrics analysis: CDF, noise and relative error
** Design more appropriate phantom to evaluate MTF (cf. Plautz et al., 2016 and Mori and Machida, 2009)
* Get in touch with R Schulte to get access to DROP and TVS-DROP algorithms (Update 18/02/2019: Ask Robert Johnson (rjohnson@ucsc.edu) for access to GitHub repo.)
* Run preliminary simulations to test the metrics

=== Lower priority ===
* Do reconstruction of simulation with >1 billion protons
* Convert ITK images to DICOM
* Assign ID to each proton to simplify matching
** Concatenating run ID and event ID to a long string does not work as the string is truncated for some reason.
** Alternative: Use a short hash? -> Need a hash function
* Write script to move the required number of projection files into the current directory
* Edit script 'backproject' to take an argument for the number of projections used

== Completed ==
* (01/02/2019) Implement phantoms to be used to determine MTF and CDF in Geant4: one uniform phantom with a single central non-uniform pixel and another truly uniform phantom
* (14/02/2019) Write code for benchmarking metrics analysis: MTF

== Current Reading List ==

=== Image quality assessment ===
* Plautz et al. (including Schulte) 2016 An evaluation of spatial resolution of a prototype proton CT scanner ''Med. Phys.'' '''43''' (12)

=== Proton CT ===
* Paganetti H 2012 Range uncertainties in proton therapy and the role of Monte Carlo simulations ''Phys. Med. Biol.'' '''57'''
* Johnson R P 2018 Review of medical radiography and tomography with proton beams ''Rep. Prog. Phys.'' '''81'''
* Arbor N ''et al.'' 2015 Monte Carlo comparison of x-ray and proton CT for range calculations of proton therapy beams ''Phys. Med. Biol.'' '''60'''
* Rädler M 2018 Two-dimensional noise reconstruction in proton computed tomography using distance-driven filtered back-projection of simulated projections ''Phys. Med. Biol.'' '''63'''
* Brooke M and Penfold S 2018 An inhomogeneous most likely path formalism for proton computed tomography ''arXiv:1808.00122v1''

ELogs/MatthieuHentz

2019-02-19T10:28:07Z

MatthieuHentz:

Electronic Log for Matthieu Hentz

== To Do ==
=== Urgent ===
* Nada

=== Higher priority ===
* Work on progress report
* Write code for benchmarking metrics analysis: CDF, noise and relative error
** Design more appropriate phantom to evaluate MTF (cf. Plautz et al., 2016 and Mori and Machida, 2009)
* Get in touch with R Schulte to get access to DROP and TVS-DROP algorithms (Update 18/02/2019: Ask Robert Johnson (rjohnson@ucsc.edu) for access to GitHub repo.)
* Run preliminary simulations to test the metrics

=== Lower priority ===
* Do reconstruction of simulation with >1 billion protons
* Convert ITK images to DICOM
* Assign ID to each proton to simplify matching
** Concatenating run ID and event ID to a long string does not work as the string is truncated for some reason.
** Alternative: Use a short hash? -> Need a hash function

== Completed ==
* (01/02/2019) Implement phantoms to be used to determine MTF and CDF in Geant4: one uniform phantom with a single central non-uniform pixel and another truly uniform phantom
* (14/02/2019) Write code for benchmarking metrics analysis: MTF

== Current Reading List ==

=== Image quality assessment ===
* Plautz et al. (including Schulte) 2016 An evaluation of spatial resolution of a prototype proton CT scanner ''Med. Phys.'' '''43''' (12)

=== Proton CT ===
* Paganetti H 2012 Range uncertainties in proton therapy and the role of Monte Carlo simulations ''Phys. Med. Biol.'' '''57'''
* Johnson R P 2018 Review of medical radiography and tomography with proton beams ''Rep. Prog. Phys.'' '''81'''
* Arbor N ''et al.'' 2015 Monte Carlo comparison of x-ray and proton CT for range calculations of proton therapy beams ''Phys. Med. Biol.'' '''60'''
* Rädler M 2018 Two-dimensional noise reconstruction in proton computed tomography using distance-driven filtered back-projection of simulated projections ''Phys. Med. Biol.'' '''63'''
* Brooke M and Penfold S 2018 An inhomogeneous most likely path formalism for proton computed tomography ''arXiv:1808.00122v1''

Software/EasyBuild

2019-02-12T14:11:22Z

MatthieuHentz:

Software/EasyBuild

2019-02-12T14:10:44Z

MatthieuHentz: /* Install modules */

Software/EasyBuild

2019-02-12T14:08:53Z

MatthieuHentz: /* Compile BDSIM */

Software/EasyBuild

2019-02-12T14:08:25Z

MatthieuHentz: /* Compile BDSIM */

=Instructions for using software with EasyBuild.=

= Compile BDSIM =
== James Chappell's question ==
Dear Laurie,

I’m trying to learn how to use the version of easy build that you setup to run some BDSIM simulations for AWAKE. I have a few questions for you.

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

2) Having looked through your eb files in /unix/pbt/software/eb/accsoft-eb-nevay, I can see that you define sources using a url and I’m assuming that in the process of compiling all the software, this source code is downloaded and then compiled. Is there a way to define the source from within the local system directories and then compile this version? Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

Sorry for all the questions, I’m just trying to get my head around how it all works! Thanks for your help,
James.

== The answer ==
Dear James,

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

My bad -> I'll recompile it with awake on. I need a time slot when you won't be using it as it'll disappear during the rebuilding - pretty bad if someone's running jobs when I do it! Let me know when I can do this - only takes about 15 mins.

Is there a way to define the source from within the local system directories and then compile this version?
It looks like the system really only is designed to download things from what I can read online... Despite the url, it caches them in a sources directory, so if it was already there it would use that (a fudge though).

Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

We have so far in the Bdsim-1.1-GCC-4.9.4.eb easy build file:

configopts = ' -DCMAKE_CXX_FLAGS="-std=c++11"'

We can add to this:

configopts += '-DUSE_AWAKE=ON'

See ROOT-v6.08.06-GCC-4.9.3-Python-2.7.12.eb for lots of examples.

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

If you're modifying the C++ often I'd make your own build of BDSIM + AWAKE separately from the EasyBuild system and not bother trying to package it into a module. The module system is really for things that won't change.

You can get all the dependencies from EasyBuild and then make your own independent build of whatever you want. I think this was done for Gate for example and we do this just now for our nightly testing of BDSIM. I load the BDSIM module, then unload it - this leaves all the dependencies loaded.

module purge
module load Bdsim
module unload Bdsim
module list

# still has all libraries loaded including Geant4 etc..
# somewhere else, make a build dir and build from your source.

When you load Geant4 and ROOT, note you should really source their setup scripts:

source $EBROOTGEANT4/bin/geant4.sh
source $EBROOTROOT/bin/thisroot.sh

--->>> You should make a copy of the bdsim.sh script that does most of this and just add the unload to the end.

then...

mkdir build
cd build
cmake ../relative/path/to/bdsim -DCMAKE_CXX_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/g++
-DCMAKE_C_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/gcc
-DCLHEP_LIBRARY_DIR=/home/accsoft/SL68/software/CLHEP/2.3.1.1-GCC-4.9.3/lib/
make -j10

* paths are example on my system at RHUL

Normally you wouldn't have to specify these options as CMake would find these on its own. However, practice shows it often finds a system compiler before it finds the one in easybuild (this is required to be the same for all the software to be consistent), so we explicitly specify them.

Note once a build is make and it finds a compiler, the compiler can never be changed in CMake. You must wipe the build directory contents to try again.

How do you use this new build? Simple, you must have the same environment loaded as when it was built (so hence the modified bdsim.sh) then execute from that directory. This could easily be done in your shell script for the simulation or via an alias (fictional paths here as an example)...

alias bdsim-awake-devel="/home/jchappell/my-builds/build123/bdsim"
bdsim-awake-devel --help

Note if you haven't changed the output or the analysis you can just use the regular bdsim from easybuild if you do any analysis later on the output. If you have changed it, you'll need to use your build and modify your bdsim.sh - pretty clear where to change for the analysis stuff.

Ok, hopefully that should help you both get the build working and help you understand.

Cheers,
Laurie

== Additional info ==
Hello,

Ok, I've updated the general BDSIM build in easy build now. It should have the AWAKE module turned on. Note, this is the code of v1.1 and not the latest develop version.

I made an example environment only script in:

/unix/pbt/software/eb/bdsim-own-build.sh

I tried making my own build and it worked just fine. I only needed to do also:

module load CMake

to get a more up to date version. From my home dir:

mkdir build-custom-build
cd build-custom-build
cmake ../bdsim
# check compiler is GCC4.9.3 form easy build, and ROOT and Geant4 and CLHEP -> all ok
ccmake .
turn on USE_AWAKE
<c>
<g>
make -j15
works!

It's in /home/lnevay/bdsim-custom-build

Cheers,
Laurie

= Use modules =
As Simon says, I've made modules ('environment modules for unix') on the system using EasyBuild. I've installed the latest Geant4 for BDSIM (our application) but I've also now put on the latest patch version of all recent Geant4 versions (was quite easy). You can access them from a CentOS7 node (I'm presuming Simon's here):

module use /unix/pbt/software/eb/CentOS7/modules/*
module avail Geant4

.. this prints out the following

------------------------------------------------ /unix/pbt/software/eb/CentOS7/modules/all -------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3

module load Geant4/10.04.p02-GCC-4.9.3
source $EBROOTGEANT4/bin/geant4.sh

The last line sources the required environmental variable for Geant4 to be found. This will load the corresponding CLHEP version required (multiple available).

To see what's loaded:

module list

Currently Loaded Modulefiles:
1) GCC/4.9.3 9) bzip2/1.0.6-GCC-4.9.3 17) Mesa/11.2.1-GCC-4.9.3
2) expat/2.1.1-GCC-4.9.3 10) freetype/2.6.3-GCC-4.9.3 18) libGLU/9.0.0-GCC-4.9.3-Mesa-11.2.1
3) CLHEP/2.3.3.0-GCC-4.9.3 11) fontconfig/2.11.95-GCC-4.9.3 19) libffi/3.2.1-GCC-4.9.3
4) cURL/7.44.0-GCC-4.9.3 12) X11/20160819-GCC-4.9.3 20) gettext/0.19.6-GCC-4.9.3
5) Xerces-C++/3.1.2-GCC-4.9.3 13) libdrm/2.4.68-GCC-4.9.3 21) PCRE/8.38-GCC-4.9.3
6) zlib/1.2.8-GCC-4.9.3 14) ncurses/6.0-GCC-4.9.3 22) GLib/2.49.5-GCC-4.9.3
7) libxml2/2.9.3-GCC-4.9.3 15) LLVM/3.8.0-GCC-4.9.3 23) Qt5/5.7.0-GCC-4.9.3
8) libpng/1.6.21-GCC-4.9.3 16) eudev/3.1.5-GCC-4.9.3 24) Geant4/10.04.p02-GCC-4.9.3

To get rid of these loaded modules:

module purge

To see all modules:

module avail

There's the latest ROOT 6 on there too as well as a few versions of CLHEP (note Geant4 versions use different ones).

The module use and module load should be done in each new environment / terminal.

I tried making an easybuild recipe for GATE, but it fails at the end (I even tried a patch to remove their hard-coded make files for gjm) but still no success.

= Install modules =

* BDSIM has to use the single threaded version -> "high throughput" only vs "high performance"
* BDSIM requires external CLHEP so Geant4's internal one isn't mixed in -> otherwise, whilst BDSIM would work, the random number generator found is unclear and the results are not strongly reproducible.

Example command: eb Bdsim-1.2.1-GCC-4.9.3.eb --prefix=/unix/pbt/software/eb/CentOS7 -robot --robot-path=./: --modules-tool=EnvironmentModulesC --module-syntax=Tcl --allow-modules-tool-mismatch --optarch=GENERIC --buildpath=/tmp/easybuild/

The trick to get it to not replace a current version is to rename the module in the easybuild file. When you make one, easybuild checks for ones of the same name and version and if one with the same name and recipe is built it won't build it. If the options are updated, I believe it will update the build. Anyway, if you change the name it will create a new one. We used the name also as part of the source URL, so you should just hard code that. I've made an example here:

/unix/pbt/software/eb/accsoft-eb-nevay/Geant4-10.04.p02-mt-GCC-4.9.3.eb
(copied from the regular geant4 one, turned multithreading on, changed version to have "-mt" and hardcoded source url)

When you build a module, you should do the module use command so it can find the others already available (it checks the requirements of the one you're building) and also the EasyBuild module itself. You should have a clean environment with no other modules loaded.

/unix/pbt/software/eb/easybuildonly.sh

seems to provide this.

I tried my example and found you should edit the version rather than the name for it to work. I've done this and it's setup now. You can import it as.

module load Geant4/10.04.p02-mt-GCC-4.9.3

Cheers,
Laurie

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

Software/EasyBuild

2019-02-12T14:07:39Z

MatthieuHentz:

=Instructions for using software with EasyBuild.=

= Compile BDSIM =
== James Chappell's question ==
Dear Laurie,

I’m trying to learn how to use the version of easy build that you setup to run some BDSIM simulations for AWAKE. I have a few questions for you.

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

2) Having looked through your eb files in /unix/pbt/software/eb/accsoft-eb-nevay, I can see that you define sources using a url and I’m assuming that in the process of compiling all the software, this source code is downloaded and then compiled. Is there a way to define the source from within the local system directories and then compile this version? Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

Sorry for all the questions, I’m just trying to get my head around how it all works! Thanks for your help,
James.

== The answer ==
Dear James,

1) The version of bdsim that you wrote the shell script for within /unix/pbt/software/eb/bdsim.sh does not have the AWAKE module compiled. Do you know if there is another version that I can link to which does for testing purposes for now?

My bad -> I'll recompile it with awake on. I need a time slot when you won't be using it as it'll disappear during the rebuilding - pretty bad if someone's running jobs when I do it! Let me know when I can do this - only takes about 15 mins.

Is there a way to define the source from within the local system directories and then compile this version?
It looks like the system really only is designed to download things from what I can read online... Despite the url, it caches them in a sources directory, so if it was already there it would use that (a fudge though).

Furthermore, to compile the AWAKE module, normally I use “ccmake .” to edit the compilation options - is the equivalent using the “configopts” command within an *.eb script? I’m assuming the command will be something like: “-USE_AWAKE=on”?

We have so far in the Bdsim-1.1-GCC-4.9.4.eb easy build file:

configopts = ' -DCMAKE_CXX_FLAGS="-std=c++11"'

We can add to this:

configopts += '-DUSE_AWAKE=ON'

See ROOT-v6.08.06-GCC-4.9.3-Python-2.7.12.eb for lots of examples.

3) For our simulations, we edit the source code of the AWAKE module to make changes to the spectrometer and then recompile BDSIM before using it. How will this setup need to change when using easy build? Will it require recompiling the whole system of dependencies every time (I can see from your email to Simon previously that this took 5 hours)? Is it the case that we can compile everything up to BDSIM in one place (i.e. Geant4 and all its dependencies), and then just load our own version of BDSIM (or even just the AWAKE module) on top of this?

If you're modifying the C++ often I'd make your own build of BDSIM + AWAKE separately from the EasyBuild system and not bother trying to package it into a module. The module system is really for things that won't change.

You can get all the dependencies from EasyBuild and then make your own independent build of whatever you want. I think this was done for Gate for example and we do this just now for our nightly testing of BDSIM. I load the BDSIM module, then unload it - this leaves all the dependencies loaded.

module purge
module load Bdsim
module unload Bdsim
module list

# still has all libraries loaded including Geant4 etc..
# somewhere else, make a build dir and build from your source.

When you load Geant4 and ROOT, note you should really source their setup scripts:

source $EBROOTGEANT4/bin/geant4.sh
source $EBROOTROOT/bin/thisroot.sh

--->>> You should make a copy of the bdsim.sh script that does most of this and just add the unload to the end.

then...

mkdir build
cd build
cmake ../relative/path/to/bdsim -DCMAKE_CXX_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/g++
-DCMAKE_C_COMPILER=/home/accsoft/SL68/software/GCC/4.9.3/bin/gcc
-DCLHEP_LIBRARY_DIR=/home/accsoft/SL68/software/CLHEP/2.3.1.1-GCC-4.9.3/lib/
make -j10

* paths are example on my system at RHUL

Normally you wouldn't have to specify these options as CMake would find these on its own. However, practice shows it often finds a system compiler before it finds the one in easybuild (this is required to be the same for all the software to be consistent), so we explicitly specify them.

Note once a build is make and it finds a compiler, the compiler can never be changed in CMake. You must wipe the build directory contents to try again.

How do you use this new build? Simple, you must have the same environment loaded as when it was built (so hence the modified bdsim.sh) then execute from that directory. This could easily be done in your shell script for the simulation or via an alias (fictional paths here as an example)...

alias bdsim-awake-devel="/home/jchappell/my-builds/build123/bdsim"
bdsim-awake-devel --help

Note if you haven't changed the output or the analysis you can just use the regular bdsim from easybuild if you do any analysis later on the output. If you have changed it, you'll need to use your build and modify your bdsim.sh - pretty clear where to change for the analysis stuff.

Ok, hopefully that should help you both get the build working and help you understand.

Cheers,
Laurie

= Use modules =
As Simon says, I've made modules ('environment modules for unix') on the system using EasyBuild. I've installed the latest Geant4 for BDSIM (our application) but I've also now put on the latest patch version of all recent Geant4 versions (was quite easy). You can access them from a CentOS7 node (I'm presuming Simon's here):

module use /unix/pbt/software/eb/CentOS7/modules/*
module avail Geant4

.. this prints out the following

------------------------------------------------ /unix/pbt/software/eb/CentOS7/modules/all -------------------------------------------------
Geant4/10.00.p04-GCC-4.9.3 Geant4/10.01.p03-GCC-4.9.3 Geant4/10.02.p03-GCC-4.9.3 Geant4/10.03.p03-GCC-4.9.3 Geant4/10.04.p02-GCC-4.9.3

module load Geant4/10.04.p02-GCC-4.9.3
source $EBROOTGEANT4/bin/geant4.sh

The last line sources the required environmental variable for Geant4 to be found. This will load the corresponding CLHEP version required (multiple available).

To see what's loaded:

module list

Currently Loaded Modulefiles:
1) GCC/4.9.3 9) bzip2/1.0.6-GCC-4.9.3 17) Mesa/11.2.1-GCC-4.9.3
2) expat/2.1.1-GCC-4.9.3 10) freetype/2.6.3-GCC-4.9.3 18) libGLU/9.0.0-GCC-4.9.3-Mesa-11.2.1
3) CLHEP/2.3.3.0-GCC-4.9.3 11) fontconfig/2.11.95-GCC-4.9.3 19) libffi/3.2.1-GCC-4.9.3
4) cURL/7.44.0-GCC-4.9.3 12) X11/20160819-GCC-4.9.3 20) gettext/0.19.6-GCC-4.9.3
5) Xerces-C++/3.1.2-GCC-4.9.3 13) libdrm/2.4.68-GCC-4.9.3 21) PCRE/8.38-GCC-4.9.3
6) zlib/1.2.8-GCC-4.9.3 14) ncurses/6.0-GCC-4.9.3 22) GLib/2.49.5-GCC-4.9.3
7) libxml2/2.9.3-GCC-4.9.3 15) LLVM/3.8.0-GCC-4.9.3 23) Qt5/5.7.0-GCC-4.9.3
8) libpng/1.6.21-GCC-4.9.3 16) eudev/3.1.5-GCC-4.9.3 24) Geant4/10.04.p02-GCC-4.9.3

To get rid of these loaded modules:

module purge

To see all modules:

module avail

There's the latest ROOT 6 on there too as well as a few versions of CLHEP (note Geant4 versions use different ones).

The module use and module load should be done in each new environment / terminal.

I tried making an easybuild recipe for GATE, but it fails at the end (I even tried a patch to remove their hard-coded make files for gjm) but still no success.

= Install modules =

* BDSIM has to use the single threaded version -> "high throughput" only vs "high performance"
* BDSIM requires external CLHEP so Geant4's internal one isn't mixed in -> otherwise, whilst BDSIM would work, the random number generator found is unclear and the results are not strongly reproducible.

Example command: eb Bdsim-1.2.1-GCC-4.9.3.eb --prefix=/unix/pbt/software/eb/CentOS7 -robot --robot-path=./: --modules-tool=EnvironmentModulesC --module-syntax=Tcl --allow-modules-tool-mismatch --optarch=GENERIC --buildpath=/tmp/easybuild/

The trick to get it to not replace a current version is to rename the module in the easybuild file. When you make one, easybuild checks for ones of the same name and version and if one with the same name and recipe is built it won't build it. If the options are updated, I believe it will update the build. Anyway, if you change the name it will create a new one. We used the name also as part of the source URL, so you should just hard code that. I've made an example here:

/unix/pbt/software/eb/accsoft-eb-nevay/Geant4-10.04.p02-mt-GCC-4.9.3.eb
(copied from the regular geant4 one, turned multithreading on, changed version to have "-mt" and hardcoded source url)

When you build a module, you should do the module use command so it can find the others already available (it checks the requirements of the one you're building) and also the EasyBuild module itself. You should have a clean environment with no other modules loaded.

/unix/pbt/software/eb/easybuildonly.sh

seems to provide this.

I tried my example and found you should edit the version rather than the name for it to work. I've done this and it's setup now. You can import it as.

module load Geant4/10.04.p02-mt-GCC-4.9.3

Cheers,
Laurie

PS we keep a note of the big incantation. You could get around most of these with some environmental variables, but we got with this so far.

PPS, remember you should source the geant4.sh from that installation before use.
source $EBROOTGEANT4/bin/geant4.sh

ELogs/MatthieuHentz

2019-02-06T14:50:15Z

MatthieuHentz: /* Completed */

Electronic Log for Matthieu Hentz

== To Do ==
=== Urgent ===
* Nada

=== Higher priority ===
* Work on progress report
* Write code for benchmarking metrics analysis: MTF, CDF, noise and relative error
* Get in touch with R Schulte to get access to DROP and TVS-DROP algorithms
* Run preliminary simulations to test the metrics

=== Lower priority ===
* Do reconstruction of simulation with >1 billion protons
* Convert ITK images to DICOM
* Assign ID to each proton to simplify matching
** Concatenating run ID and event ID to a long string does not work as the string is truncated for some reason.
** Alternative: Use a short hash? -> Need a hash function

== Completed ==
* (01/02/2018) Implement phantoms to be used to determine MTF and CDF in Geant4: one uniform phantom with a single central non-uniform pixel and another truly uniform phantom

== Current Reading List ==
* Paganetti H 2012 Range uncertainties in proton therapy and the role of Monte Carlo simulations ''Phys. Med. Biol.'' '''57'''
* Johnson R P 2018 Review of medical radiography and tomography with proton beams ''Rep. Prog. Phys.'' '''81'''
* Arbor N ''et al.'' 2015 Monte Carlo comparison of x-ray and proton CT for range calculations of proton therapy beams ''Phys. Med. Biol.'' '''60'''
* Rädler M 2018 Two-dimensional noise reconstruction in proton computed tomography using distance-driven filtered back-projection of simulated projections ''Phys. Med. Biol.'' '''63'''
* Brooke M and Penfold S 2018 An inhomogeneous most likely path formalism for proton computed tomography ''arXiv:1808.00122v1''

ELogs/MatthieuHentz

2019-02-06T14:50:01Z

MatthieuHentz: /* Completed */

Electronic Log for Matthieu Hentz

== To Do ==
=== Urgent ===
* Nada

=== Higher priority ===
* Work on progress report
* Write code for benchmarking metrics analysis: MTF, CDF, noise and relative error
* Get in touch with R Schulte to get access to DROP and TVS-DROP algorithms
* Run preliminary simulations to test the metrics

=== Lower priority ===
* Do reconstruction of simulation with >1 billion protons
* Convert ITK images to DICOM
* Assign ID to each proton to simplify matching
** Concatenating run ID and event ID to a long string does not work as the string is truncated for some reason.
** Alternative: Use a short hash? -> Need a hash function

== Completed ==
* Implement phantoms to be used to determine MTF and CDF in Geant4: one uniform phantom with a single central non-uniform pixel and another truly uniform phantom (01/02/2018)

== Current Reading List ==
* Paganetti H 2012 Range uncertainties in proton therapy and the role of Monte Carlo simulations ''Phys. Med. Biol.'' '''57'''
* Johnson R P 2018 Review of medical radiography and tomography with proton beams ''Rep. Prog. Phys.'' '''81'''
* Arbor N ''et al.'' 2015 Monte Carlo comparison of x-ray and proton CT for range calculations of proton therapy beams ''Phys. Med. Biol.'' '''60'''
* Rädler M 2018 Two-dimensional noise reconstruction in proton computed tomography using distance-driven filtered back-projection of simulated projections ''Phys. Med. Biol.'' '''63'''
* Brooke M and Penfold S 2018 An inhomogeneous most likely path formalism for proton computed tomography ''arXiv:1808.00122v1''

ELogs/MatthieuHentz

2019-02-06T14:49:27Z

MatthieuHentz: /* To Do */

Electronic Log for Matthieu Hentz

== To Do ==
=== Urgent ===
* Nada

=== Higher priority ===
* Work on progress report
* Write code for benchmarking metrics analysis: MTF, CDF, noise and relative error
* Get in touch with R Schulte to get access to DROP and TVS-DROP algorithms
* Run preliminary simulations to test the metrics

=== Lower priority ===
* Do reconstruction of simulation with >1 billion protons
* Convert ITK images to DICOM
* Assign ID to each proton to simplify matching
** Concatenating run ID and event ID to a long string does not work as the string is truncated for some reason.
** Alternative: Use a short hash? -> Need a hash function

== Completed ==

== Current Reading List ==
* Paganetti H 2012 Range uncertainties in proton therapy and the role of Monte Carlo simulations ''Phys. Med. Biol.'' '''57'''
* Johnson R P 2018 Review of medical radiography and tomography with proton beams ''Rep. Prog. Phys.'' '''81'''
* Arbor N ''et al.'' 2015 Monte Carlo comparison of x-ray and proton CT for range calculations of proton therapy beams ''Phys. Med. Biol.'' '''60'''
* Rädler M 2018 Two-dimensional noise reconstruction in proton computed tomography using distance-driven filtered back-projection of simulated projections ''Phys. Med. Biol.'' '''63'''
* Brooke M and Penfold S 2018 An inhomogeneous most likely path formalism for proton computed tomography ''arXiv:1808.00122v1''

ELogs/MatthieuHentz

2019-01-24T11:06:03Z

MatthieuHentz:

Electronic Log for Matthieu Hentz

== To Do ==
=== Urgent ===
* Nada

=== Higher priority ===
* Work on progress report
* Implement phantoms to be used to determine MTF and CDF in Geant4: one uniform phantom with a single central non-uniform pixel and another truly uniform phantom
* Write code for benchmarking metrics analysis: MTF, CDF, noise and relative error
* Get in touch with R Schulte to get access to DROP and TVS-DROP algorithms
* Run preliminary simulations to test the metrics

=== Lower priority ===
* Do reconstruction of simulation with >1 billion protons
* Convert ITK images to DICOM
* Assign ID to each proton to simplify matching
** Concatenating run ID and event ID to a long string does not work as the string is truncated for some reason.
** Alternative: Use a short hash? -> Need a hash function

== Completed ==

== Current Reading List ==
* Paganetti H 2012 Range uncertainties in proton therapy and the role of Monte Carlo simulations ''Phys. Med. Biol.'' '''57'''
* Johnson R P 2018 Review of medical radiography and tomography with proton beams ''Rep. Prog. Phys.'' '''81'''
* Arbor N ''et al.'' 2015 Monte Carlo comparison of x-ray and proton CT for range calculations of proton therapy beams ''Phys. Med. Biol.'' '''60'''
* Rädler M 2018 Two-dimensional noise reconstruction in proton computed tomography using distance-driven filtered back-projection of simulated projections ''Phys. Med. Biol.'' '''63'''
* Brooke M and Penfold S 2018 An inhomogeneous most likely path formalism for proton computed tomography ''arXiv:1808.00122v1''

ELogs/MatthieuHentz

2019-01-23T19:38:59Z

MatthieuHentz: /* To Do */

ELogs/MatthieuHentz

2019-01-14T11:48:00Z

MatthieuHentz:

Electronic Log for Matthieu Hentz

== To Do ==
* Write progress report
* Do reconstruction of simulation with >1 billion protons
* Convert ITK images to DICOM
* Assign ID to each proton to simplify matching
** Concatenating run ID and event ID to a long string does not work as the string is truncated for some reason.
** Alternative: Use a short hash? -> Need a hash function

== Completed ==

ELogs/MatthieuHentz

2019-01-14T11:30:48Z

MatthieuHentz:

Electronic Log for Matthieu Hentz

== To Do ==

Clatterbridge

2018-03-28T16:39:35Z

MatthieuHentz: /* Building the simulation */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Building the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:
<pre>
[username@pc1XX ~]$ mkdir Clatterbridge_sim
[username@pc1XX ~]$ cd Clatterbridge_sim
[username@pc1XX Clatterbridge_sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX Clatterbridge_sim]$ mkdir build
[username@pc1XX Clatterbridge_sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

Alternatively, the simulation can be cloned from the [https://github.com/mhentz/Clatterbridge_sim.git GitHub repository] as follows:
<pre>
[username@pc1XX ~]$ git clone https://github.com/mhentz/Clatterbridge_sim.git
[username@pc1XX ~]$ cd Clatterbridge_sim
[username@pc1XX Clatterbridge_sim]$ mkdir build
[username@pc1XX Clatterbridge_sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

When cloning the repository, the executable scripts will be created without execute permissions. These must be given using
<pre>
[username@pc1XX build]$ chmod 755 *.sh
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

= Files =
[[Clatterbridge files|List of source files.]]

Click here to download source as a zip: [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim.zip download]

Proton Calorimetry/Meetings/2018/03/14

2018-03-28T16:33:20Z

MatthieuHentz: /* Present */

== Minutes for UCL Proton Calorimetry Meeting, 14th March 2018 (D17, Physics & Astronomy, UCL) ==

=== Present ===

'''Anastasia Basharina-Freshville''', '''Dan Walker''', '''Jordan Silverman''', '''Laurent Kelleter''', '''Matthieu Hentz''', '''Simon Jolly''', '''Ruben Saakyan'''

'''RS''':
* Interested in seeing dose deposition vs depth, even if not calibrated, dump into histogram and check what we see from both sensors.
* Seeing huge variations in current. Is this problem for us?
** If variation of current affects relative variation in sheets then it's a problem.
** '''SJ''' suggest you can see variation in current but can wash it out if picking a 5 ms exposure on dlsr.
* Consider 300 um scintillator sheets for very fine resolution around Bragg peak.
** '''SJ''' and '''LK''' in perfect unison: this is fine for single energy but problematic when scanning different energies.
* Good opportunity to check 5-stage tube if time allows.

'''MH''':
* Finished writing documentation for simulation.
* Working on next coursework and CDT group project.
* Jacinta Jap would like to know:
**Where are the beam parameters used in the Clatterbridge simulation from?
*** '''SJ''' obtained Excel spreadsheet of Twiss parameters from Hywel Owen. As the scattering perturbs the beam a lot the shape should not be crucial. Making sure the energy is right is more important.
** Who wrote the MCNPX simulation?
*** '''SJ''' only recalls Colin Baker using MCNPX.

'''JS''':
* Sucessfully converted CAD model to a GDML file.
* Asks if he should import it into the simulation before the report is due (deadline is 26 March)?
** '''SJ''' suggests to work on it until Monday 19 March and focus on writing report from then onwards.
** '''JS''' knows what steps are required, only needs to implement them.
** '''MH''' suggests getting rid of beamline definition in DetectorConstruction, import GDML file and reintroduce any missing elements one by one for easier debugging.

'''DW''':
* Will be around next year as he received offer from CDT in Quantum Technologies (woo woo).
* Comments on Medaustron run:
** Timestamps are separated by 3 minutes 38 seconds, i.e. our data taken after theirs. There are no sensible matches between datasets.
** '''SJ''' explains that absence of discrete trigger means no concrete correlation between the two data sets. Adjusting trigger levels throws away events so may end up with different numbers between calorimeter and tracker.
** '''SJ''' suggest that it won't matter in write up for project. DW can show spectra and should also focus on writing after Monday evening (19 March).

Proton Calorimetry/Meetings/2018/03/14

2018-03-28T10:33:41Z

MatthieuHentz: /* Present */

== Minutes for UCL Proton Calorimetry Meeting, 14th March 2018 (D17, Physics & Astronomy, UCL) ==

=== Present ===

'''Anastasia Basharina-Freshville''', '''Dan Walker''', '''Jordan Silverman''', '''Laurent Kelleter''', '''Matthieu Hentz''', '''Simon Jolly''', '''Ruben Saakyan'''

'''RS''':
* Interested in seeing dose deposition vs depth, even if not calibrated, dump into histogram and check what we see from both sensors.
* Seeing huge variations in current. Is this problem for us?
** If variation of current affects relative variation in sheets then it's a problem.
** '''SJ''' suggest you can see variation in current but can wash it out if picking a 5 ms exposure on dlsr.
* Consider 300 um scintillator sheets for very fine resolution around Bragg peak.
** '''SJ''' and '''LK''' in perfect unison: this is fine for single energy but problematic when scanning different energies.
* Good opportunity to check 5-stage tube if time allows.

'''MH''':
* Finished writing documentation for simulation.
* Working on next coursework and CDT group project.
* Jacinta Jap would like to know:
** Where are the beam parameters used in the Clatterbridge simulation from?
** '''SJ''' obtained Excel spreadsheet of Twiss parameters from Hywel Owen. As the scattering perturbs the beam a lot the shape should not be crucial. Making sure the energy is right is more important.
* Who wrote the MCNPX simulation?
** '''SJ''' only recalls Colin Baker using MCNPX.

'''JS''':
* Sucessfully converted CAD model to a GDML file.
* Asks if he should import it into the simulation before the report is due (deadline is 26 March)?
** '''SJ''' suggests to work on it until Monday 19 March and focus on writing report from then onwards.
** '''JS''' knows what steps are required, only needs to implement them.
** '''MH''' suggests getting rid of beamline definition in DetectorConstruction, import GDML file and reintroduce any missing elements one by one for easier debugging.

'''DW''':
* Will be around next year as he received offer from CDT in Quantum Technologies (woo woo).
* Comments on Medaustron run:
** Timestamps are separated by 3 minutes 38 seconds, i.e. our data taken after theirs. There are no sensible matches between datasets.
** '''SJ''' explains that absence of discrete trigger means no concrete correlation between the two data sets. Adjusting trigger levels throws away events so may end up with different numbers between calorimeter and tracker.
** '''SJ''' suggest that it won't matter in write up for project. DW can show spectra and should also focus on writing after Monday evening (19 March).

Clatterbridge files

2018-03-18T09:37:46Z

MatthieuHentz: /* Data analysis */

[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge Wiki page]

== Simulation Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc clatterbridgeSim.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac proton.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac score_init.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac score_dump.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/visualisation.mac visualisation.mac]

=== Parallel Simulations ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh cb_parallel.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh run.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh submit.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh combine.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh combine_all.sh]

=== Data analysis ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/analysis.py analysis.py]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/bragg.py bragg.py]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/lateral.py lateral.py]

== Header Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorConstruction.hh DetectorConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorMessenger.hh DetectorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventAction.hh EventAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventActionMessenger.hh EventActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/FileReader.hh FileReader.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldConstruction.hh ParallelWorldConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldMessenger.hh ParallelWorldMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldParam.hh ParallelWorldParam.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceHit.hh PhaseSpaceHit.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceSD.hh PhaseSpaceSD.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsList.hh PhysicsList.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsListMessenger.hh PhysicsListMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorAction.hh PrimaryGeneratorAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorMessenger.hh PrimaryGeneratorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAccumulator.hh RunAccumulator.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAction.hh RunAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunActionMessenger.hh RunActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMax.hh StepMax.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMaxMessenger.hh StepMaxMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingAction.hh SteppingAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingActionMessenger.hh SteppingActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingVerbose.hh SteppingVerbose.hh]

== Source Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc DetectorConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorMessenger.cc DetectorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventAction.cc EventAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventActionMessenger.cc EventActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc FileReader.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc ParallelWorldConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldMessenger.cc ParallelWorldMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc ParallelWorldParam.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc PhaseSpaceHit.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc PhaseSpaceSD.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc PhysicsList.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsListMessenger.cc PhysicsListMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc PrimaryGeneratorAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorMessenger.cc PrimaryGeneratorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc RunAccumulator.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAction.cc RunAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunActionMessenger.cc RunActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMax.cc StepMax.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMaxMessenger.cc StepMaxMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc SteppingAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingActionMessenger.cc SteppingActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingVerbose.cc SteppingVerbose.cc]

Clatterbridge

2018-03-13T15:54:29Z

MatthieuHentz: /* Build the simulation */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Building the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:
<pre>
[username@pc1XX ~]$ mkdir Clatterbridge_sim
[username@pc1XX ~]$ cd Clatterbridge_sim
[username@pc1XX Clatterbridge_sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX Clatterbridge_sim]$ mkdir build
[username@pc1XX Clatterbridge_sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

Alternatively, the simulation can be cloned from the [https://github.com/mhentz/Clatterbridge_sim.git GitHub repository] as follows:
<pre>
[username@pc1XX ~]$ git clone https://github.com/mhentz/Clatterbridge_sim.git
[username@pc1XX ~]$ cd Clatterbridge_sim
[username@pc1XX Clatterbridge_sim]$ mkdir build
[username@pc1XX Clatterbridge_sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

= Files =
[[Clatterbridge files|List of source files.]]

Click here to download source as a zip: [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim.zip download]

Clatterbridge

2018-03-13T15:54:08Z

MatthieuHentz: /* Build the simulation */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:
<pre>
[username@pc1XX ~]$ mkdir Clatterbridge_sim
[username@pc1XX ~]$ cd Clatterbridge_sim
[username@pc1XX Clatterbridge_sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX Clatterbridge_sim]$ mkdir build
[username@pc1XX Clatterbridge_sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

Alternatively, the simulation can be cloned from the [https://github.com/mhentz/Clatterbridge_sim.git GitHub repository] as follows:
<pre>
[username@pc1XX ~]$ git clone https://github.com/mhentz/Clatterbridge_sim.git
[username@pc1XX ~]$ cd Clatterbridge_sim
[username@pc1XX Clatterbridge_sim]$ mkdir build
[username@pc1XX Clatterbridge_sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

= Files =
[[Clatterbridge files|List of source files.]]

Click here to download source as a zip: [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim.zip download]

Clatterbridge files

2018-03-13T12:27:43Z

MatthieuHentz:

[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge Wiki page]

== Simulation Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc clatterbridgeSim.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac proton.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac score_init.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac score_dump.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/visualisation.mac visualisation.mac]

=== Parallel Simulations ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh cb_parallel.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh run.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh submit.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh combine.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh combine_all.sh]

=== Data analysis ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_rstephens/ProtonPB_source/analysis.py analysis.py]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_rstephens/ProtonPB_source/bragg.py bragg.py]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_rstephens/ProtonPB_source/lateral.py lateral.py]

== Header Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorConstruction.hh DetectorConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorMessenger.hh DetectorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventAction.hh EventAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventActionMessenger.hh EventActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/FileReader.hh FileReader.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldConstruction.hh ParallelWorldConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldMessenger.hh ParallelWorldMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldParam.hh ParallelWorldParam.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceHit.hh PhaseSpaceHit.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceSD.hh PhaseSpaceSD.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsList.hh PhysicsList.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsListMessenger.hh PhysicsListMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorAction.hh PrimaryGeneratorAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorMessenger.hh PrimaryGeneratorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAccumulator.hh RunAccumulator.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAction.hh RunAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunActionMessenger.hh RunActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMax.hh StepMax.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMaxMessenger.hh StepMaxMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingAction.hh SteppingAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingActionMessenger.hh SteppingActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingVerbose.hh SteppingVerbose.hh]

== Source Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc DetectorConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorMessenger.cc DetectorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventAction.cc EventAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventActionMessenger.cc EventActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc FileReader.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc ParallelWorldConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldMessenger.cc ParallelWorldMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc ParallelWorldParam.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc PhaseSpaceHit.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc PhaseSpaceSD.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc PhysicsList.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsListMessenger.cc PhysicsListMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc PrimaryGeneratorAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorMessenger.cc PrimaryGeneratorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc RunAccumulator.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAction.cc RunAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunActionMessenger.cc RunActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMax.cc StepMax.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMaxMessenger.cc StepMaxMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc SteppingAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingActionMessenger.cc SteppingActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingVerbose.cc SteppingVerbose.cc]

Clatterbridge files

2018-03-13T12:25:12Z

MatthieuHentz: /* Data analysis */

== Simulation Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc clatterbridgeSim.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac proton.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac score_init.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac score_dump.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/visualisation.mac visualisation.mac]

=== Parallel Simulations ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh cb_parallel.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh run.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh submit.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh combine.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh combine_all.sh]

=== Data analysis ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_rstephens/ProtonPB_source/analysis.py analysis.py]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_rstephens/ProtonPB_source/bragg.py bragg.py]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_rstephens/ProtonPB_source/lateral.py lateral.py]

== Header Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorConstruction.hh DetectorConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorMessenger.hh DetectorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventAction.hh EventAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventActionMessenger.hh EventActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/FileReader.hh FileReader.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldConstruction.hh ParallelWorldConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldMessenger.hh ParallelWorldMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldParam.hh ParallelWorldParam.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceHit.hh PhaseSpaceHit.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceSD.hh PhaseSpaceSD.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsList.hh PhysicsList.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsListMessenger.hh PhysicsListMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorAction.hh PrimaryGeneratorAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorMessenger.hh PrimaryGeneratorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAccumulator.hh RunAccumulator.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAction.hh RunAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunActionMessenger.hh RunActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMax.hh StepMax.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMaxMessenger.hh StepMaxMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingAction.hh SteppingAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingActionMessenger.hh SteppingActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingVerbose.hh SteppingVerbose.hh]

== Source Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc DetectorConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorMessenger.cc DetectorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventAction.cc EventAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventActionMessenger.cc EventActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc FileReader.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc ParallelWorldConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldMessenger.cc ParallelWorldMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc ParallelWorldParam.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc PhaseSpaceHit.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc PhaseSpaceSD.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc PhysicsList.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsListMessenger.cc PhysicsListMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc PrimaryGeneratorAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorMessenger.cc PrimaryGeneratorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc RunAccumulator.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAction.cc RunAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunActionMessenger.cc RunActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMax.cc StepMax.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMaxMessenger.cc StepMaxMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc SteppingAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingActionMessenger.cc SteppingActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingVerbose.cc SteppingVerbose.cc]

Clatterbridge files

2018-03-13T12:24:27Z

MatthieuHentz: /* Simulation Files */

== Simulation Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc clatterbridgeSim.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac proton.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac score_init.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac score_dump.mac]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/visualisation.mac visualisation.mac]

=== Parallel Simulations ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh cb_parallel.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh run.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh submit.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh combine.sh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh combine_all.sh]

=== Data analysis ===

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_rstephens/ProtonPB_source/simulation_analysis.C simulation_analysis.C]

== Header Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorConstruction.hh DetectorConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/DetectorMessenger.hh DetectorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventAction.hh EventAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/EventActionMessenger.hh EventActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/FileReader.hh FileReader.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldConstruction.hh ParallelWorldConstruction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldMessenger.hh ParallelWorldMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/ParallelWorldParam.hh ParallelWorldParam.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceHit.hh PhaseSpaceHit.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhaseSpaceSD.hh PhaseSpaceSD.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsList.hh PhysicsList.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PhysicsListMessenger.hh PhysicsListMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorAction.hh PrimaryGeneratorAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/PrimaryGeneratorMessenger.hh PrimaryGeneratorMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAccumulator.hh RunAccumulator.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunAction.hh RunAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/RunActionMessenger.hh RunActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMax.hh StepMax.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/StepMaxMessenger.hh StepMaxMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingAction.hh SteppingAction.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingActionMessenger.hh SteppingActionMessenger.hh]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/include/SteppingVerbose.hh SteppingVerbose.hh]

== Source Files ==

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc DetectorConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorMessenger.cc DetectorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventAction.cc EventAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/EventActionMessenger.cc EventActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc FileReader.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc ParallelWorldConstruction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldMessenger.cc ParallelWorldMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc ParallelWorldParam.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc PhaseSpaceHit.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc PhaseSpaceSD.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc PhysicsList.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsListMessenger.cc PhysicsListMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc PrimaryGeneratorAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorMessenger.cc PrimaryGeneratorMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc RunAccumulator.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAction.cc RunAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunActionMessenger.cc RunActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMax.cc StepMax.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/StepMaxMessenger.cc StepMaxMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc SteppingAction.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingActionMessenger.cc SteppingActionMessenger.cc]

[http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingVerbose.cc SteppingVerbose.cc]

Clatterbridge

2018-03-13T12:15:56Z

MatthieuHentz: /* Files */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

= Files =
[[Clatterbridge files|List of source files.]]

Click here to download source as a zip: [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim.zip download]

Clatterbridge

2018-03-13T12:15:40Z

MatthieuHentz: /* Files */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

= Files =
[[Clatterbridge files|List of source files.]]

Click here to download source files as a zip: [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim.zip download]

Clatterbridge files

2018-03-13T12:14:59Z

MatthieuHentz:

Clatterbridge files

2018-03-13T12:14:40Z

MatthieuHentz: /* Simulation Files */

Clatterbridge files

2018-03-13T12:14:25Z

MatthieuHentz: /* Simulation Files */

Clatterbridge

2018-03-13T12:12:52Z

MatthieuHentz: /* Files */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

= Files =
[[Clatterbridge files|List of files for Clatterbridge simulation]].

Click here to download source files as a zip: [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim.zip download]

Clatterbridge

2018-03-13T12:06:34Z

MatthieuHentz: /* Changing parameters */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-13T12:04:19Z

MatthieuHentz: /* Phase space file */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD::ProcessHits</code>] accordingly. The z-coordinates are given relative to the position of the source and the z-axis lies along the beamline with its positive end pointing towards the centre of the room. An example file is shown below (<code>psf_z0.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-13T12:02:19Z

MatthieuHentz: /* Output files */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Phase space file ===
The output files <code>psf_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator::Dump</code>] (<code>X</code> is the position in z where the particles were recorded). Currently the recorded quantities are the parent ID indicating whether a particle is a primary or secondary, the particle name, the position vector in mm, the momentum direction vector, and the kinetic energy in MeV. Other quantities can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceHit.cc <code>PhaseSpaceHit</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# parentID, name, x [mm], y [mm], z [mm], mom_x, mom_y, mom_z, ke [MeV]
0,proton,1.862704,-8.483117,0.0,0.003526,0.001614,0.999992,62.677102
0,proton,0.087498,3.564388,0.0,0.002226,-0.000444,0.999997,62.506999
0,proton,2.636636,5.235856,0.0,0.000542,0.000050,1.000000,62.569683
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-13T11:55:35Z

MatthieuHentz: /* Scoring */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Note: scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-13T11:53:22Z

MatthieuHentz: /* Gathering data */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
'''Scorers are not currently used in the simulation.'''

Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-13T11:52:30Z

MatthieuHentz: /* Monitoring job status */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

=== Combining data ===
The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine.sh <code>combine.sh</code>] [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_all.sh <code>combine_all.sh</code>]. If only combining a file corresponding to one position:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine.sh
</pre>
The user will be prompted to provide the position in mm of the file that should be combined.

If combining all files:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ ./combine_all.sh
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-13T11:18:22Z

MatthieuHentz: /* Monitoring job status */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command to refresh periodically):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-13T11:13:50Z

MatthieuHentz: /* Running simulations in parallel */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

*If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the <code>-r</code> flag must be supplied as an argument to <code>submit.sh</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] and since the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be adjusted by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code>. Other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files. The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] and writes the result to a job submission script <code>cb_parallel_X.sh</code> for all 100 simulations. Besides setting parameters and navigating to the required directory, the job submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] and writes the result to a new macro <code>protonX.mac</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Building_the_simulation Building the simulation] " unless you have already done so. The script <code>run.sh</code> can be supplied with a note when being called (e.g. note of number of primaries). Parallel simulations are then run as follows:
<pre>
[username@pc1XX build]$ ./run.sh "This is a note that will be written to notes.txt"
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T20:44:08Z

MatthieuHentz: /* Running simulations in parallel */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

* If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> and the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
[username@pc1XX build]$ ./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] creates a timestamped directory and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the short queue using the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by changing the arguments supplied to <code>submit.sh</code> in <code>run.sh</code> (other queues available are <code>medium</code> and <code>long</code> and the <code>-r</code> flag can be used to set parallel simulations to read from input files):
<pre>
# Submit a given number of simulations to a given queue
# If reading primaries from a file, set the split option
../submit.sh short 100
#../submit.sh short 100 -r
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T20:13:38Z

MatthieuHentz: /* Running simulations in parallel */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

* If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> and the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

As mentioned above, simulations can be run in parallel while reading primaries from an input file. This file should be split and distributed evenly across the simulations using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] in order to make use of the parallelism. The script calculates the number of lines each simulation should receive given a number of simulations and then writes chunks of the initial file to separate files which will then be used as input files for the parallel simulation. It should be called in the simulation's base directory as follows:
<pre>
./split_input.sh /path/to/file nsimulations
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

=== Monitoring job status ===

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T20:03:19Z

MatthieuHentz: /* Running simulations */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

* If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> and the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T20:02:43Z

MatthieuHentz: /* Running simulations */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

== Build the simulation ==
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

== Running the simulation in batch mode with macro ==

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

== Running simulations in parallel ==
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

* If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> and the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T20:02:25Z

MatthieuHentz:

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For parallel simulations, some parameters need to be set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>].
*When running simulations in parallel, each simulation is executed two directories further down from where the parallel run is initialised. Hence, the correct path to the <code>gps.mac</code> macro should be set in <code>proton.mac</code>:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
# Parallel version
/control/execute ../../gps.mac
</pre>

*Separate simulations should have different random number generator seeds so that they are independent but in order for simulations to be reproducible, the same sequence of seeds should be assigned for each run. To achieve this, the seeds in <code>proton.mac</code> are set by the job submission script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/cb_parallel.sh <code>cb_parallel.sh</code>] using the index of the current simulation. For the <code>proton.mac</code> macro to use the seeds set by the submission script, the following line should be set:
<pre>
#-------------------------------------------------------------------------------
# Set seeds for randon number generators
#-------------------------------------------------------------------------------
...
# Parallel version
/random/setSeeds ${seed1} ${seed2}
</pre>

* If generating primaries from a phase space file that was split using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/split_input.sh <code>split_input.sh</code>] as described below, the number of events is set by <code>cb_parallel.sh</code> in <code>proton.mac</code> and the following line should be uncommented:
<pre>
#-------------------------------------------------------------------------------
# Run simulation
#-------------------------------------------------------------------------------
...
# Parallel version with split input file
/run/beamOn ${nevents}
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T17:38:47Z

MatthieuHentz: /* Recording in a single position for staged simulations */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position (for staged simulations) ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For split simulations, the parameters are set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>]. The main difference to [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] is the lower number of events and the variable seeds
<pre>
/random/setSeeds ${seed1} ${seed2}
</pre>
which are set in the PBS job submission script <code>clatterbridge_X.sh</code> described below (X is an integer). This ensures that all simulations are independent (since they have different seeds) but as the same set of seeds is used each time, the simulations remain repeatable.

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T17:38:24Z

MatthieuHentz: /* Recording at regular intervals using parallel world */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using a parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position for staged simulations ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For split simulations, the parameters are set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>]. The main difference to [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] is the lower number of events and the variable seeds
<pre>
/random/setSeeds ${seed1} ${seed2}
</pre>
which are set in the PBS job submission script <code>clatterbridge_X.sh</code> described below (X is an integer). This ensures that all simulations are independent (since they have different seeds) but as the same set of seeds is used each time, the simulations remain repeatable.

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T17:38:01Z

MatthieuHentz: /* Reading primaries from phase space file */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from a phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position for staged simulations ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For split simulations, the parameters are set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>]. The main difference to [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] is the lower number of events and the variable seeds
<pre>
/random/setSeeds ${seed1} ${seed2}
</pre>
which are set in the PBS job submission script <code>clatterbridge_X.sh</code> described below (X is an integer). This ensures that all simulations are independent (since they have different seeds) but as the same set of seeds is used each time, the simulations remain repeatable.

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].

Clatterbridge

2018-03-12T17:37:41Z

MatthieuHentz: /* Using the /gps/ commands */

This simulation is a model of the monoenergetic 62.5 MeV proton beam at the [http://www.clatterbridgecc.nhs.uk/patients/treatment-and-support/proton-therapy Clatterbridge Cancer Centre] as it traverses the components of the beamline and finally hits a volume of water. The simulation was built on the example in <code>examples/extended/electromagnetic/TestEm7</code> supplied with the Geant4 package and detailed [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/Tutorials/Basic/Monoenergetic_Proton_Pencil_Beam here]. The physics list used is QGSP_BIC_HP, the standard for simulating clinical proton beams.

= Geometry =
=== Treatment room ===
The schematic below shows the Clatterbridge treatment room, illustrating the layout of the geometries defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>]. The whole beamline is contained within the "inner room" volume, which is placed in the "outer room" volume. As the outer volume is a solid concrete box and the inner volume is a box filled with air, the small overlap models the concrete walls of the treatment room. The proton source is placed against the wall in the inner room (-4200 mm from the origin), and all beamline components are placed relative to this reference point.

[[File:treatmentRoomSchematic.png|centre|700px]]

=== Beamline ===
[[Media:Beamline_doc.pdf|The positions of the components relative to the source are shown here (annotated PDF)]]. The beamline is shown below. The top figure is a perspective view. The second figure is a top-down view which also shows the borated plastic shielding (f) that was previously hidden to allow the full beamline to be visible.

[[File:beamline_perspective.png|border|right|600px]]
[[File:beamline_top_down.png|border|right|600px]]

The beamline consists of the following components:

*an aluminium tube (a) containing from left to right;
**a brass collimator
**a tungsten scattering foil
**a brass beam stopper
**another tungsten scattering foil
**a mylar window

*an empty aluminium box (b)
*an iron block (c)
*a second aluminium tube (d)
*a second aluminium box (e) containing from left to right;
**a brass collimator
**two dose monitors
**cross wires

*a brass nozzle (g)

The water phantom (h) is also shown for completeness.

=== Dual scattering tube ===
[[Media:Tube_doc.pdf|The positions of the components in the tube are shown here (annotated PDF)]]. The proton beam at Clatterbridge is delivered using passive spreading. That is, the beam is spread out using dual scattering where the proton beam is scattered through a first scattering foil followed by a beam stopper and a second scattering foil. The beam stopper is used to reduce the intensity of the beam at its centre and a high-Z material such as tungsten is chosen for the foils as it is highly scattering.

A visualisation of the first tube with 500 primary protons represented by blue tracks is shown below. The protons first travel through the collimator, followed by the first scattering foil which spreads out the beam. The brass stopper then blocks out approximately half of the remaining protons before the beam is again spread out through the second scattering foil. The intention is to produce a wide, homogeneous beam. The particles exit the first tube through a Kapton window which is used to keep the tube under vacuum to reduce random scattering off air molecules.

[[File:First_tube_500_sim.png|centre|700px]]

=== Dosimetry box ===
[[Media:Dosimetry_box.pdf|The positions of the components in the dosimetry box are shown here (annotated PDF)]]. The aluminium box shown below contains the dosimetry equipment. The beam is first collimated using a brass collimator to remove any stray protons. The beam then travels through the dose monitors, the cross wires and finally exits the box through the brass nozzle.

[[File:Second_box.png|centre|700px]]

=== Dose monitor ===
[[Media:Dose_monitor_doc.pdf|The positions of the components in the dose monitors are shown here (annotated PDF)]]. The dose monitors are drift chambers used to measure the dose deposited by the proton beam. An exploded view of a dose monitor as used in the dosimetry box is shown below. It consists of a set of aluminised mylar foils wedged between layers of perspex to hold them in place. The guard ring is used to create a sealed volume of air between the foils. The aluminium layers face towards the centre of the dose monitor such that the assembly acts as a drift chamber when a potential difference is applied.

[[File:Dose_monitor_exploded.png|centre|700px]]

=== Visualisation ===
The geometry and particle tracks in Geant4 [http://geant4.slac.stanford.edu/Presentations/vis/G4DAWNTutorial/G4DAWNTutorial.html can be visualised using the DAWN visualiser]. Other, perhaps more suitable, visualisers are available (see [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch08.html]) but DAWN is useful in that it produces very high-resolution visualisations. The main drawback of DAWN is that it does not allow for click-and-drag functionality. Instead, parameters must be set before each visualisation is drawn which tends to require a fair amount of trial-and-error to obtain good images. All images of the geometry on this page were created using DAWN and an example visualisation of the beamline with all particle tracks drawn can be seen below. The first tube and second box are visualised using the wireframe setting so that their inner components are visible. This example contains 500 primary protons. Protons are shown in blue, electrons are red, positrons are cyan, gamma rays are green, and neutrons are yellow.

[[File:Beamline_500_sim_perspective.png|border|centre|800px]]

= Generating primary protons =
=== Using the <code>/gps/</code> commands ===
The protons are generated using the [http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/ch02s07.html <code>G4GeneralParticleSource</code>] (GPS) class which allows for a range of properties of the primary protons to be set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] file (primaries are the initial particles generated by the GPS). First, the particle source is positioned against the wall on one side of the room which is at -4200 mm from the centre. In an attempt to achieve a more realistic beam, the primary protons are distributed normally in the x and y directions centred at 0 with standard deviations of 4.0 mm and 4.5 mm respectively. This gives the beam width and results in a nearly circular profile. The primaries are generated with initial energies following a Gaussian distribution with mean 62.5 MeV and standard deviation 0.082 MeV, and Gaussian radial distributions with respect to the z-axis with standard deviations of 2.3 mrad and 1.2 mrad in the x and y directions respectively.

The parameters are set in the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/gps.mac <code>gps.mac</code>] macro as follows:
<pre>
#-------------------------------------------------------------------------------
# Set particle gun settings
#-------------------------------------------------------------------------------
/gps/particle proton
/gps/number 1 # per event

# Energy
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV

# Source position
/gps/position 0.0 0.0 -4200 mm

# Beam width
/gps/pos/type Beam
/gps/pos/sigma_x 4.0 mm
/gps/pos/sigma_y 4.5 mm

# Angular distribution of primaries for realistic emittance
/gps/ang/type beam2d
/gps/ang/sigma_x 2.3 mrad
/gps/ang/sigma_y 1.2 mrad
/gps/ang/rot1 -1 0 0 # aligns gps with positive z-axis
</pre>

The macro is called in <code>proton.mac</code> as follows:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Use macro to set properties of primaries
/control/execute gps.mac
</pre>

=== Reading primaries from phase space file ===
Primaries can be generated from an input phase space file using [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/FileReader.cc <code>FileReader</code>] in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PrimaryGeneratorAction.cc <code>PrimaryGeneratorAction</code>]. The input must be of the same format as the output files generated by the simulation and its first line is assumed to be a header. The path to the input file should be given in <code>proton.mac</code> and the position at which the particles should be generated:
<pre>
#-------------------------------------------------------------------------------
# Primary generator settings
#-------------------------------------------------------------------------------
...
## Reading from phase space file
/primarygenerator/input path/to/file

# Set position in z particles are generated at
/primarygenerator/generateAt 0
#/primarygenerator/generateAt 81
#/primarygenerator/generateAt 357
#/primarygenerator/generateAt 1700
</pre>
Specifying where particles should be generated in useful when running staged simulations as it allows particles to be generated at the same position they were initially recorded at.

= Gathering data =
== Tracking ==
=== Accumulating hits over a run ===
Several quantities can be monitored at different stages in the beamline using the <code>SensitiveDetector</code> defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhaseSpaceSD.cc <code>PhaseSpaceSD.cc</code>] (<code>PhaseSpaceSD</code> extends [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VSensitiveDetector.html <code>G4VSensitiveDetector</code>]). When a particle hits a boundary on which a <code>PhaseSpaceSD</code> is defined, <code>PhaseSpaceSD::ProcessHits</code> is called and generates a <code>PhaseSpaceHit</code> (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VHit.html <code>G4VHit</code>]) that is then stored in the hits collection of the sensitive detector. A unique ID belonging to the volume the hit was registered on is passed to the hit along with any required quantity. Hits are accumulated in the sensitive detector's hits collection over an event (an event corresponds to the full simulation of one primary).

The [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/RunAccumulator.cc <code>RunAccumulator</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4Run.html <code>G4Run</code>]) accumulates hits over a run (the collection of all events). It is constructed by <code>RunAction::GenerateRun</code> at the start of a run. At the end of an event, <code>RunAccumulator::RecordEvent</code> stores all hits that were registered during the event in the run hits vector and increments the event counter. Once that counter reaches the buffer, the hits are written to their corresponding output file. The buffer size can be set using the following line in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>]:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
# Set buffer size
/user/run/buffer 1000
</pre>

=== Recording at regular intervals using parallel world ===
To characterise the beam, the evolution of several properties (energy spectrum, emittance, spatial profile, etc.) must be monitored at regular intervals, or at least at several positions, along the beamline. To achieve this, many volumes would need to be defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction</code>] to be used as sensitive detectors. However, volumes overlapping or sharing boundaries can lead to errors in the simulation and must therefore be avoided. One approach in achieving this would be to carefully construct each detector volume such that it fits in, or around, a given region. An example illustrating why this may be challenging is given by the following; suppose detection was required inside and outside of the first aluminium box on a plane transverse to the direction of the beam. Separate detector volumes would be required on the inside and outside of the box just to detect particles incident on the same plane. This does not yet address the issue of how to handle the boundaries between the box and the detectors as particles would be likely to pass through the gaps and hence go undetected. Even for this simple example, the geometry of the detector volumes would be rather complex and hence not easily reusable.

To avoid these problems, the main geometry (called the ''mass geometry'') can be overlaid with another parallel geometry that does not interact physically with particles in the mass geometry, as described [http://geant4.web.cern.ch/geant4/workAreaUserDocKA/Backup/Docbook_UsersGuides_beta/ForApplicationDeveloper/html/ch04s07.html here]. Different volumes can then be defined at arbitrary positions and have sensitive detectors assigned to them in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldConstruction.cc <code>ParallelWorldConstruction</code>]. To track particles at regular intervals along the beamline, an array of thin boxes is defined by passing the parametrisation [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/ParallelWorldParam.cc <code>ParallelWorldParam</code>] (extension of [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4VPVParameterisation.html <code>G4VPVParametrisation</code>]) to the parametrised volume constructor [http://www.apc.univ-paris7.fr/~franco/g4doxy/html/classG4PVParameterised.html <code>G4PVParametrised</code>]. This creates boxes of a given size at given intervals spanning a given distance from the source. The size of the box can be adjusted in <code>ParallelWorldConstruction</code>, while the spacing and total distance spanned can be set in the <code>proton.mac</code> macro using the following settings:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## If using 'all'
/parallel/detector/spacing 25 mm # default is 25 mm
/parallel/detector/distance 1900 mm # default is 1900 mm

# Pass spacing and distance to run action and set buffer size
/user/run/detector/spacing 25
/user/run/detector/distance 1900
</pre>

=== Recording in a single position for staged simulations ===
Alternatively, particles can be recorded in a single position by choosing one of the components and uncommenting accordingly. The available components are
* scatterfoil1 at 81 mm
*tube1 at 357 mm
*nozzle at 1700 mm
*outside at 1768 m
Components can be added to the components map in <code>ParallelWorldConstruction</code>. Only recording in one position can be useful when the beam has been characterised and the simulation is know to be running as intended. Writing to output less often improves the simulation's performance significantly. This can be used to speed up the simulation further by performing simulations in a staged manner. Sections upstream, where no changes to the geometry are made (e.g. from the source to after the first collimator where the particle count reduces by about 90%), then only need to be simulated once and the output from that simulation can be used as the input to the rest of the simulation.

For example, if the simulation should only run up to the first scatter foil (situated after the first collimator), the commands would look as follows:
<pre>
#-------------------------------------------------------------------------------
# Detector settings
#-------------------------------------------------------------------------------
...
## Choose where particles are recorded
#/parallel/detector all
/parallel/detector scatterfoil1
#/parallel/detector tube1
#/parallel/detector nozzle
#/parallel/detector outside
...
## If using a component, set dump mode and detector position
/user/run/dump/single true
/user/run/detector/position 81
#/user/run/detector/position 357
#/user/run/detector/position 1700
#/user/run/detector/position 1768

...

#-------------------------------------------------------------------------------
# Stepping action settings
#-------------------------------------------------------------------------------
...
## Kill particles 1 mm after being recorded
/steppingAction/kill 82
#/steppingAction/kill 358
#/steppingAction/kill 1701
#/steppingAction/kill 1769
</pre>

The command <code>/user/run/dump/single</code> sets a flag used in <code>RunAccumulator::InitialiseOutputFiles</code>.

== Scoring ==
Scorers are defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>] and the quantities they record are dumped to files as defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>].

=== Longitudinal scoring mesh ===
A scoring mesh is ''longitudinal'' in the sense that it records data along the direction of the beamline, e.g. the energy deposited at different positions in z. Hence, for a mesh to be longitudinal it must have bins defined along its z-axis. The scorer defined on the mesh <code>detEnergyLon</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh detEnergyLon
/score/mesh/boxSize 20. 20. 20. mm
/score/mesh/nBin 1 1 200
/score/mesh/translate/xyz 0. 0. -2340 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

On the third line in the snippet above you can see that there is only a single bin in both the x and y directions but 200 in the z-direction.

=== Lateral scoring mesh ===
Similarly to the longitudinal meshes, a mesh is said to be ''lateral'' when it records data transverse to the direction of travel of the protons. For a mesh to be lateral it must have bins defined perpendicularly to its z-axis. The scorer defined on the mesh <code>braggEnergyLat</code> is an example of such a mesh. It is defined as follows:

<pre>
/score/create/boxMesh braggEnergyLat
/score/mesh/boxSize 20. 20. 0.5 mm
/score/mesh/nBin 200 1 1
/score/mesh/translate/xyz 0. 0. -2329.2 mm
/score/quantity/energyDeposit energyDeposit
/score/close
</pre>

Again, you can see that there is only a single bin in the y and z directions but 200 in the x-direction.

= Running simulations =

=== Build the simulation ===
To access a Linux desktop PC running Scientific Linux 6, [http://www.hep.ucl.ac.uk/pbt/wiki/Software/Geant4/UCL_HEP_Cluster follow the instructions here]. Once access has been established, follow the instructions below to copy the simulation files to your directory:

<pre>
[username@pc1XX ~]$ mkdir sim
[username@pc1XX ~]$ cd sim
[username@pc1XX sim]$ cp -r /unix/pbt/users/mhentz/Clatterbridge_sim/source .
[username@pc1XX sim]$ mkdir build
[username@pc1XX sim]$ cd build
[username@pc1XX build]$ /unix/pbt/software/scripts/pbt.sh
[username@pc1XX build]$ cmake -DGeant4_DIR=/unix/pbt/software/dev ../source
[username@pc1XX build]$ make -j4
</pre>

=== Running the simulation in batch mode with macro ===

This will run the simulation and produce the required output files in the <code>data</code> subdirectory created.

<pre>
[username@pc1XX build]$ mkdir data
[username@pc1XX build]$ ./clatterbridgeSim proton.mac
</pre>

=== Running simulations in parallel ===
A simulation comprising of a large number of particles can take a long time to run as events run sequentially (about 20 hours for 1e6 events). This can be reduced considerably by running a number of independent simulations with fewer events "in parallel" and then merging the output files once they have completed. The main limitation lies in the maximum number of simulations that can be run simultaneously–which is 120. Splitting 1e6 events into 100 simulations of 10,000 events reduces the runtime to a much more bearable 20 minutes.

For split simulations, the parameters are set in the macro [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>]. The main difference to [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] is the lower number of events and the variable seeds
<pre>
/random/setSeeds ${seed1} ${seed2}
</pre>
which are set in the PBS job submission script <code>clatterbridge_X.sh</code> described below (X is an integer). This ensures that all simulations are independent (since they have different seeds) but as the same set of seeds is used each time, the simulations remain repeatable.

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/run.sh <code>run.sh</code>] (located in the <code>source</code> directory but should automatically be copied to <code>build</code> by cmake) creates a timestamped directory along with a few useful subdirectories and submits a specified number of simulations to be executed there. It is currently set to submit 100 jobs to the medium queue through the [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] script. This can be changed by supplying different arguments on line 22 (other queues available are <code>short</code> and <code>long</code>):
<pre>
../submit.sh 100 medium
</pre>

The script [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/submit.sh <code>submit.sh</code>] assigns values to the variables in the template [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridge_split.sh <code>clatterbridge_split.sh</code>] and writes the result to a job submission script <code>clatterbridge_X.sh</code> for all 100 simulations. This is done on line 31:
<pre>
cat ../clatterbridge_split.sh | sed -e "s/\${i}/$i/" >> clatterbridge_$i.sh
</pre>

Besides setting vital parameters and navigating to the correct folder, the submission script sets the seeds in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac_split <code>proton.mac_split</code>] and writes the result to a new macro <code>proton.mac_simX</code>. It then runs the simulation with the corresponding macro.

Follow the instructions in "[http://www.hep.ucl.ac.uk/pbt/pbtWiki/index.php?title=Clatterbridge&action=submit#Make_the_simulation Make the simulation] " unless you have already done so. Split simulations are run as follows:
<pre>
[username@pc1XX build]$ ./run.sh
</pre>

The status of the simulations can be tracked with the following command (can be used in conjunction with the <code>watch</code> command):
<pre>
[username@pc1XX build]$ qstat -u <username>
</pre>

If something goes wrong and you wish to stop all the jobs simultaneously you can use the command
<pre>
[username@pc1XX build]$ qselect -u <username> | xargs qdel
</pre>

The resulting data can be combined by running the scripts [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_tracking.py <code>combine_tracking.py</code>] and [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/combine_scorers.py <code>combine_scorers.py</code>] as follows:
<pre>
[username@pc1XX build]$ cd timestamp-dir
[username@pc1XX timestamp-dir]$ cp ../../source/combine_*.py .
[username@pc1XX timestamp-dir]$ ./combine_tracking.py
[username@pc1XX timestamp-dir]$ ./combine_scorers.py
</pre>

= Output files =
=== Output used for beam emittance and energy spectra ===
The output files <code>output_zX.txt</code> are written to the subdirectory <code>data/</code> in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] (<code>X</code> is the position in z where the protons were recorded). Other particles can be recorded by changing [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>] accordingly. The output files contain the parent ID which is 0 for a primary particle and non-zero for all other particles, the xyz-coordinates in mm, the angle subtended by the projection of the momentum vector onto the x-axis and the z-axis in mrad (used to plot the beam emittance) and the kinetic energy in MeV. The z-coordinates are given relative to the position of the particle source and the z-axis lies along the beamline with its positive end pointing towards the origin. An example is shown below (<code>output_z1759.txt</code>):

<pre>
# zposition: 1759
# parentID, x, y, z [mm], theta [mrad], ke [MeV]
0 -9.78358 4.39357 1759.0 -6.01461 60.32370
0 -7.12684 -10.49640 1758.9 -4.95641 60.12080
0 8.83137 -2.33413 1759.0 10.60610 60.07960
0 5.92333 -3.29109 1758.9 7.89279 60.20930
0 -2.21463 6.24370 1759.0 -6.14550 60.04930
...
</pre>

=== Scorer outputs ===
The files described below are all written using the <code>/score/dumpQuantityToFile</code> commands in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_dump.mac <code>score_dump.mac</code>]. The data are recorded by the scorers defined in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>].

==== Lateral energy deposition at the Bragg peak in the detector ====
The file <code>BraggEnergyDepLat.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>braggEnergyLat</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition at the Bragg peak in the plane perpendicular to the direction of incidence of the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV (as the mesh is only binned in x, it consists of strips parallel to the y-axis):
<pre>
# mesh name: braggEnergyLat
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,29.931796279684590
1,0,0,38.256791245912446
2,0,0,115.051837974658625
3,0,0,185.831154261095321
...
</pre>
This file can be used to plot the lateral energy deposition profile at the Bragg peak.

==== Longitudinal energy deposition throughout the detector ====
The file <code>DetEnergyDepLon.txt</code> contains data recorded by the <code>energyDeposit</code> scorer defined on the <code>detEnergyLon</code> mesh in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/score_init.mac <code>score_init.mac</code>]. It records the energy deposition in the detector in bins along the beam. The columns represent the bin number in the x, y and z directions and the energy deposition in MeV:
<pre>
# mesh name: detEnergyLon
# primitive scorer name: energyDeposit
# iX, iY, iZ, value [MeV]
0,0,0,59390.147050145809771
0,0,1,59016.142052385039278
0,0,2,59209.373600437116693
0,0,3,59578.920521325460868
...
</pre>
This file can be used to plot the energy deposition profile through the detector. The Bragg peak can be identified from such a plot.

= Data Analysis =
=== Beam profile, emittance and energy spectrum ===
The script <code>analysis.py</code> produces several different plots from the <code>output_zX.txt</code> files. It plots the beam profile, the projection of the beam profile onto the x-axis, the beam emittance and the energy spectrum at the position in z where the protons were recorded. This script requires the <code>SciPy</code> module which is not available on the Linux SL6 PCs. Hence, it must currently be run locally. It is run for a given position along the beamline as follows (400 mm as an example, the script must be run in the directory containing the <code>data</code> subdirectory):

<pre>
[user@hostname ~]$ ./analysis.py 400
</pre>

This runs the script using the file <code>output_z400.txt</code> and produces the corresponding plots shown below. At this stage, the beam has undergone dual scattering and is starting to spread out again to produce a uniform beam.

[[File:Tiles_400.jpg|centre|750px]]

=== Longitudinal energy deposition profile and Bragg peak ===
The script <code>bragg.py</code> plots the longitudinal energy deposition profile through the detector from <code>DetEnergyDepLon.txt</code>. It is executed using Python 2.7 in the same directory as the data file as follows:

<pre>
[user@hostname ~]$ ./bragg.py
</pre>

This produces the following plot;
[[File:Lon_energy_deposition_bragg.png|centre|600px]]

=== Lateral energy deposition profile at the Bragg peak ===
The script <code>lateral.py</code> plots the lateral energy deposition at the Bragg peak from <code>BraggEnergyDepLat.txt</code> described above. It is executed on Python 2.7 as follows:

<pre>
[user@hostname ~]$ ./lateral.py BraggEnergyDepLat.txt
</pre>

This produces the following plot;
[[File:Lat_energy_deposition_bragg.png|centre|600px]]

== Changing parameters ==

=== Beam parameters ===
These can be modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/proton.mac <code>proton.mac</code>] under the header "particle gun settings".

==== Energy ====
The GPS gives the initial protons a range of energies following a Gaussian distribution.

<pre>
/gps/ene/type Gauss
/gps/ene/mono 62.5 MeV
/gps/ene/sigma 0.082 MeV
</pre>

==== Source position ====
The proton source is positioned at <code>z = -4200 mm</code> relative to the centre of the inner room, which corresponds to a position against the wall of the inner room.

<pre>
/gps/position 0.0 0.0 -4200 mm
</pre>

=== Scoring meshes ===
Filters can be applied to scoring meshes so as to only record a given type of particle using the command <code>/score/filter/particle protonFilter proton</code> in the definition of the scoring mesh. This can be useful as, for example, one may want to monitor the flux of different particles along the beamline. This would require a large mesh positioned over the whole beamline as follows:

<pre>
/score/create/boxMesh beamlineLongitudinal
/score/mesh/boxSize 50. 50. 1000. mm
/score/mesh/nBin 1 1 400
/score/mesh/translate/xyz 0. 0. -3200 mm
/score/quantity/flatSurfaceFlux protonFlux
/score/filter/particle protonFilter proton
/score/quantity/flatSurfaceFlux neutronFlux
/score/filter/particle neutronFilter neutron
/score/close
</pre>

This mesh records two separate quantities, the proton flux and the neutron flux, which can then be written to separate output files.

=== Beamline components ===
Components of the beamline can be added, removed or modified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/DetectorConstruction.cc <code>DetectorConstruction.cc</code>]. For example, the position of the second dose monitor can be changed by changing the z-value of the <code>G4ThreeVector</code> on the line below. As the dose monitor is placed within the dosimetry box <code>lAlBox2</code> it's position is given relative to the centre of that box.

<pre>
pDoseMonitor2 = new G4PVPlacement(0, G4ThreeVector(0, 0, -169.5*CLHEP::mm), lDoseMonitor1, "DoseMonitor2", lAlBox2, false, 0, checkOverlaps);
</pre>

'''After any changes made to the source or header files (ending in <code>.cc</code> or <code>.hh</code>), the code will need to be recompiled. In the build directory, write:'''

<pre>
[username@pc1XX build]$ make
</pre>

=== Physics list ===
A user-defined physics list can be specified in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/PhysicsList.cc <code>PhysicsList.cc</code>] and then selected in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/clatterbridgeSim.cc <code>clatterbridgeSim.cc</code>] using the line

<pre>
runManager->SetUserInitialization(new PhysicsList);
</pre>

=== Tracking positions ===
As described in the section "[http://www.hep.ucl.ac.uk/pbt/wiki/Clatterbridge#Tracking Tracking] ", protons are tracked in [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.cc <code>SteppingAction.cc</code>]. This class contains a vector <code>trackingPosVec</code> which contains the positions of interest where protons are to be tracked. Positions can be changed by changing <code>trackingPosFiller</code> in the header file [http://www.hep.ucl.ac.uk/pbt/wikiData/code/Clatterbridge_sim/source/src/SteppingAction.hh <code>SteppingAction.hh</code>] on the following line:

<pre>
trackingPosFiller = { 0, 45, 79, 81, 100, 150, ..., 1759, 1800, 1839, 1881 };
</pre>

'''Don't forget to recompile after any changes made.'''

= Files =

[[Clatterbridge files|List of files for Clatterbridge simulation]].