PBTWiki - User contributions [en]

Proton Calorimetry/Inventory

2025-08-27T09:53:34Z

SoniaEscribano:

Inventory of things

* Scintillators

* Photodiodes

* DDC Boards

* USB-C cables

* FPGA

* Raspberry pi

* Tools

ELogs/SoniaEscribano

2025-08-05T14:16:26Z

SoniaEscribano: /* To Do */

Electronic Log for Sonia Escribano

== To Do ==

'''Data Analysis'''
* Analysis of QuARC as a beam monitor
* Compare beam monitor results with PPTC
* Create a git repository with latest analysis codes

'''Raspberry pi 5 + GUI'''
* Modify live visualisation and calibration codes to include facility
* Add more info to error messages on the status box
* Add saturation error message?
* Replay codes?
* Duplicate latest version of code to new SD cards

'''Lab'''
* Organise all cables
* Create a spare set up for Trento Bem Test
*

== GUI ==

All the repositories can. be found here: https://github.com/UCL-Proton-Beam-Therapy/GUI

'''Version 2.1'''

This version displays the data from a file selected via a dropdown menu (using php), the Kelleter and Bortfeld curves, and the energy and range of the beam as a legend.
[https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/ https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/]

* Create a file with Manchester data
* Fix cache busting in Google Chrome
* Overlap between users in different browsers
* Determine number of points per PD for files
* Responsivity in different devices is not considered

'''Version 2.06'''

Version 2.06 has a unique start/stop button and displays live data when running a shell script via terminal.
* Zooming issue: PDdata and fitted data do not scale at the same rate, when zooming there is a shifting between the two -> Use linear scales for both ✓
* When clicking stop freeze the graph but do not make it dissapear ✓
* Modify code to give - instead of 0 when fit not performed ✓
* Plot data with 20 points ✓

'''Version 2.05'''

Version 2.05 has two separate start and stop button

* Upload files to new directory ✓
* Modify position of error bars ✓
* x-axis zooming problem: only some numbers work ✓
* Fix misalignment between PDdata and FITdata ✓
* Test access from NUC PC ✓

== Knowledge Transfer ==
- Future work in QuARC: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work]

- Summary : [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information]

== Replay + Fit ==
Two different codes are used for the data analysis: replay.py and fit.cpp, their functionalities are described below.

replay.py code calibrates and subtracts the background of each measurement, generating an output text file with header information, all the individual calibrated measurements and the average of them all. An example file with data from Clatterbridge Hospital of what the generated file looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_calibrated.txt], every individual run fit takes ~3.3s on a MacBook to be completed. All of them can be processed together by running the shell script ./replayshell.sh. To run this code we need the following arguments:
python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

fit.cpp averages the data at a user specified frequency (or generic 25 Hz) and then performs Bortfeld and Kelleter fits to the data. After the fitting, the beam energy and range are extracted for each of those averaged measurements. All of them can be processed together by running the shell script ./fitshell.sh An example of what the output of fit.cpp looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_fitted_25Hz.txt]. This code needs to be compiled, and then run with the appropriate arguments.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3
./fit reverse energy facility folder run (frequency)

To show these data in the GUI a shell script that iterates through the lines of the fitted output file is necessary.

Previously both of the codes were developed in cpp. The replay.cpp replays the DDC232 acquisition by either stepping through each measurement or averaging measurements at specified frame rate, while fit.cpp performs the fit on calibrated and background subtracted data.

== Done ==

* PARTREC pictures uploaded to: /unix/pbt/data/partrec/20221125/Pictures
* Cut Mylar foil sheets for Manchester experiment
* Talk to Derek about Mylar foil.
* Probation meeting with Simon (fill form)
* Set up new MacBook Pro: might need dual booting (talk to Tony Hoare about Windows).
* Set up Geant4 simulations of detector → copied and running
* Port Saad's FPGA data replay code to Python → copied
* Duplicate functionality of Fern's GUI in D3.

== Quick-start ==
* Send a copy of pasport/ID so they can set up a contract (Bonita Carboo). Then a videocall needs to be arranged to check passport.
* Proof of right to work (share code via gov.uk)
* Sign online contract. Your employee number is here.
* Once you have an employee number you can check your user ID and email account [https://myaccount.ucl.ac.uk/new-staff]
* Welcome email with infromation for new staff member, probation and guidance [https://www.ucl.ac.uk/human-resources/ucl-induction-and-probation-policy#Policy]
* Check and complete the induction checklist (received on the welcome email)
* Get ID card: book appointment online ✓
* Mandatory safety training courses: Safety Induction + Fire Safety ✓
* Book departamental safety induction with Lee Bebbington ✓
* Mandatory training courses: GDPR, information security, freedom of information, DSE, diversity, unconscious bias, ✓
* Create a wiki account ✓
* Register for an account on the HEP Linux cluster (email support@hep.ucl.ac.uk) ✓
* Install Geant4 (instructions: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Detector_Simulation]) ✓
* Set up bank details online on myHR ✓

== Important Dates ==

{| class="wikitable"
! style="text-align: center;" | Date
! style="text-align: center;" | Event
|-
|26th-27th October 2023|| Ion Imaging Workshop London
|-
|11th-13th October 2023|| ACCMED course Pavia
|-
|25th-28th July 2023|| FPGA Course RAL
|-
|3rd-7th July 2023|| HIT Online course
|-
|10th-16th June 2023|| PTCOG Conference Madrid
|-
|18th May 2023|| Clatterbridge beam test
|-
|5th Apr 2023|| PGI Seminar @Queen Mary
|-
|8th Feb 2023 || QuARC Seminar @Bristol
|-
|20th Jan 2023 || PhD Graduation York
|-
|30th Nov - 2nd Dec 2022 || FRPT Conference Barcelona
|-
|24-25th Nov 2022 || Groningen beam test
|-
|25-26th Oct 2022 || Manchester beam test
|-
|21st Oct 2022 || HEP Seminar
|-
|20th Oct 2022 || Prague
|-
|23rd Sep 2022 || VIVA York
|}

ELogs/SoniaEscribano

2025-08-05T14:15:47Z

SoniaEscribano: /* GUI */

ELogs/SoniaEscribano

2025-08-05T14:11:16Z

SoniaEscribano: /* To Do */

Electronic Log for Sonia Escribano

== To Do ==

'''Data Analysis'''
* Analysis of QuARC as a beam monitor
* Compare beam monitor results with PPTC
* Create a git repository with latest analysis codes

'''Raspberry pi 5 + GUI'''
* Modify live visualisation and calibration codes to include facility
* Add more info to error messages on the status box
* Add saturation error message?
* Replay codes?
* Duplicate latest version of code to new SD cards

'''Lab'''
* Organise all cables
* Create a spare set up for Trento Bem Test
*

== GUI ==
'''Version 2.1'''

This version displays the data from a file selected via a dropdown menu (using php), the Kelleter and Bortfeld curves, and the energy and range of the beam as a legend.
[https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/ https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/]

* Create a file with Manchester data
* Fix cache busting in Google Chrome
* Overlap between users in different browsers
* Determine number of points per PD for files
* Responsivity in different devices is not considered

'''Version 2.06'''

Version 2.06 has a unique start/stop button and displays live data when running a shell script via terminal.
* Zooming issue: PDdata and fitted data do not scale at the same rate, when zooming there is a shifting between the two -> Use linear scales for both ✓
* When clicking stop freeze the graph but do not make it dissapear ✓
* Modify code to give - instead of 0 when fit not performed ✓
* Plot data with 20 points ✓

'''Version 2.05'''

Version 2.05 has two separate start and stop button

* Upload files to new directory ✓
* Modify position of error bars ✓
* x-axis zooming problem: only some numbers work ✓
* Fix misalignment between PDdata and FITdata ✓
* Test access from NUC PC ✓

== Knowledge Transfer ==
- Future work in QuARC: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work]

- Summary : [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information]

== Replay + Fit ==
Two different codes are used for the data analysis: replay.py and fit.cpp, their functionalities are described below.

replay.py code calibrates and subtracts the background of each measurement, generating an output text file with header information, all the individual calibrated measurements and the average of them all. An example file with data from Clatterbridge Hospital of what the generated file looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_calibrated.txt], every individual run fit takes ~3.3s on a MacBook to be completed. All of them can be processed together by running the shell script ./replayshell.sh. To run this code we need the following arguments:
python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

fit.cpp averages the data at a user specified frequency (or generic 25 Hz) and then performs Bortfeld and Kelleter fits to the data. After the fitting, the beam energy and range are extracted for each of those averaged measurements. All of them can be processed together by running the shell script ./fitshell.sh An example of what the output of fit.cpp looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_fitted_25Hz.txt]. This code needs to be compiled, and then run with the appropriate arguments.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3
./fit reverse energy facility folder run (frequency)

To show these data in the GUI a shell script that iterates through the lines of the fitted output file is necessary.

Previously both of the codes were developed in cpp. The replay.cpp replays the DDC232 acquisition by either stepping through each measurement or averaging measurements at specified frame rate, while fit.cpp performs the fit on calibrated and background subtracted data.

== Done ==

* PARTREC pictures uploaded to: /unix/pbt/data/partrec/20221125/Pictures
* Cut Mylar foil sheets for Manchester experiment
* Talk to Derek about Mylar foil.
* Probation meeting with Simon (fill form)
* Set up new MacBook Pro: might need dual booting (talk to Tony Hoare about Windows).
* Set up Geant4 simulations of detector → copied and running
* Port Saad's FPGA data replay code to Python → copied
* Duplicate functionality of Fern's GUI in D3.

== Quick-start ==
* Send a copy of pasport/ID so they can set up a contract (Bonita Carboo). Then a videocall needs to be arranged to check passport.
* Proof of right to work (share code via gov.uk)
* Sign online contract. Your employee number is here.
* Once you have an employee number you can check your user ID and email account [https://myaccount.ucl.ac.uk/new-staff]
* Welcome email with infromation for new staff member, probation and guidance [https://www.ucl.ac.uk/human-resources/ucl-induction-and-probation-policy#Policy]
* Check and complete the induction checklist (received on the welcome email)
* Get ID card: book appointment online ✓
* Mandatory safety training courses: Safety Induction + Fire Safety ✓
* Book departamental safety induction with Lee Bebbington ✓
* Mandatory training courses: GDPR, information security, freedom of information, DSE, diversity, unconscious bias, ✓
* Create a wiki account ✓
* Register for an account on the HEP Linux cluster (email support@hep.ucl.ac.uk) ✓
* Install Geant4 (instructions: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Detector_Simulation]) ✓
* Set up bank details online on myHR ✓

== Important Dates ==

{| class="wikitable"
! style="text-align: center;" | Date
! style="text-align: center;" | Event
|-
|26th-27th October 2023|| Ion Imaging Workshop London
|-
|11th-13th October 2023|| ACCMED course Pavia
|-
|25th-28th July 2023|| FPGA Course RAL
|-
|3rd-7th July 2023|| HIT Online course
|-
|10th-16th June 2023|| PTCOG Conference Madrid
|-
|18th May 2023|| Clatterbridge beam test
|-
|5th Apr 2023|| PGI Seminar @Queen Mary
|-
|8th Feb 2023 || QuARC Seminar @Bristol
|-
|20th Jan 2023 || PhD Graduation York
|-
|30th Nov - 2nd Dec 2022 || FRPT Conference Barcelona
|-
|24-25th Nov 2022 || Groningen beam test
|-
|25-26th Oct 2022 || Manchester beam test
|-
|21st Oct 2022 || HEP Seminar
|-
|20th Oct 2022 || Prague
|-
|23rd Sep 2022 || VIVA York
|}

ELogs/SoniaEscribano

2025-08-05T14:10:26Z

SoniaEscribano: /* To Do */

Electronic Log for Sonia Escribano

== To Do ==

'''Data Analysis'''
* Analysis of QuARC as a beam monitor
* Compare beam monitor results with PPTC
* Create a git repository with latest analysis codes

'''Raspberry pi 5'''
* Modify live visualisation and calibration codes to include facility
* Add more info to error messages on the status box
* Add saturation error message?
* Duplicate latest version of code to new SD cards

'''Lab'''
* Organise all cables
* Create a spare set up for Trento Bem Test
*

== GUI ==
'''Version 2.1'''

This version displays the data from a file selected via a dropdown menu (using php), the Kelleter and Bortfeld curves, and the energy and range of the beam as a legend.
[https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/ https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/]

* Create a file with Manchester data
* Fix cache busting in Google Chrome
* Overlap between users in different browsers
* Determine number of points per PD for files
* Responsivity in different devices is not considered

'''Version 2.06'''

Version 2.06 has a unique start/stop button and displays live data when running a shell script via terminal.
* Zooming issue: PDdata and fitted data do not scale at the same rate, when zooming there is a shifting between the two -> Use linear scales for both ✓
* When clicking stop freeze the graph but do not make it dissapear ✓
* Modify code to give - instead of 0 when fit not performed ✓
* Plot data with 20 points ✓

'''Version 2.05'''

Version 2.05 has two separate start and stop button

* Upload files to new directory ✓
* Modify position of error bars ✓
* x-axis zooming problem: only some numbers work ✓
* Fix misalignment between PDdata and FITdata ✓
* Test access from NUC PC ✓

== Knowledge Transfer ==
- Future work in QuARC: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work]

- Summary : [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information]

== Replay + Fit ==
Two different codes are used for the data analysis: replay.py and fit.cpp, their functionalities are described below.

replay.py code calibrates and subtracts the background of each measurement, generating an output text file with header information, all the individual calibrated measurements and the average of them all. An example file with data from Clatterbridge Hospital of what the generated file looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_calibrated.txt], every individual run fit takes ~3.3s on a MacBook to be completed. All of them can be processed together by running the shell script ./replayshell.sh. To run this code we need the following arguments:
python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

fit.cpp averages the data at a user specified frequency (or generic 25 Hz) and then performs Bortfeld and Kelleter fits to the data. After the fitting, the beam energy and range are extracted for each of those averaged measurements. All of them can be processed together by running the shell script ./fitshell.sh An example of what the output of fit.cpp looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_fitted_25Hz.txt]. This code needs to be compiled, and then run with the appropriate arguments.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3
./fit reverse energy facility folder run (frequency)

To show these data in the GUI a shell script that iterates through the lines of the fitted output file is necessary.

Previously both of the codes were developed in cpp. The replay.cpp replays the DDC232 acquisition by either stepping through each measurement or averaging measurements at specified frame rate, while fit.cpp performs the fit on calibrated and background subtracted data.

== Done ==

* PARTREC pictures uploaded to: /unix/pbt/data/partrec/20221125/Pictures
* Cut Mylar foil sheets for Manchester experiment
* Talk to Derek about Mylar foil.
* Probation meeting with Simon (fill form)
* Set up new MacBook Pro: might need dual booting (talk to Tony Hoare about Windows).
* Set up Geant4 simulations of detector → copied and running
* Port Saad's FPGA data replay code to Python → copied
* Duplicate functionality of Fern's GUI in D3.

== Quick-start ==
* Send a copy of pasport/ID so they can set up a contract (Bonita Carboo). Then a videocall needs to be arranged to check passport.
* Proof of right to work (share code via gov.uk)
* Sign online contract. Your employee number is here.
* Once you have an employee number you can check your user ID and email account [https://myaccount.ucl.ac.uk/new-staff]
* Welcome email with infromation for new staff member, probation and guidance [https://www.ucl.ac.uk/human-resources/ucl-induction-and-probation-policy#Policy]
* Check and complete the induction checklist (received on the welcome email)
* Get ID card: book appointment online ✓
* Mandatory safety training courses: Safety Induction + Fire Safety ✓
* Book departamental safety induction with Lee Bebbington ✓
* Mandatory training courses: GDPR, information security, freedom of information, DSE, diversity, unconscious bias, ✓
* Create a wiki account ✓
* Register for an account on the HEP Linux cluster (email support@hep.ucl.ac.uk) ✓
* Install Geant4 (instructions: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Detector_Simulation]) ✓
* Set up bank details online on myHR ✓

== Important Dates ==

{| class="wikitable"
! style="text-align: center;" | Date
! style="text-align: center;" | Event
|-
|26th-27th October 2023|| Ion Imaging Workshop London
|-
|11th-13th October 2023|| ACCMED course Pavia
|-
|25th-28th July 2023|| FPGA Course RAL
|-
|3rd-7th July 2023|| HIT Online course
|-
|10th-16th June 2023|| PTCOG Conference Madrid
|-
|18th May 2023|| Clatterbridge beam test
|-
|5th Apr 2023|| PGI Seminar @Queen Mary
|-
|8th Feb 2023 || QuARC Seminar @Bristol
|-
|20th Jan 2023 || PhD Graduation York
|-
|30th Nov - 2nd Dec 2022 || FRPT Conference Barcelona
|-
|24-25th Nov 2022 || Groningen beam test
|-
|25-26th Oct 2022 || Manchester beam test
|-
|21st Oct 2022 || HEP Seminar
|-
|20th Oct 2022 || Prague
|-
|23rd Sep 2022 || VIVA York
|}

ELogs/SoniaEscribano

2025-08-05T14:10:15Z

SoniaEscribano: /* To Do */

Electronic Log for Sonia Escribano

== To Do ==

'''Data Analysis'''
* Analysis of QuARC as a beam monitor
* Compare beam monitor results with PPTC
* Create a git repository with latest analysis codes

'''Raspberry pi 5'''
- Modify live visualisation and calibration codes to include facility
- Add more info to error messages on the status box
- Add saturation error message?
- Duplicate latest version of code to new SD cards

'''Lab'''
* Organise all cables
* Create a spare set up for Trento Bem Test
*

== GUI ==
'''Version 2.1'''

This version displays the data from a file selected via a dropdown menu (using php), the Kelleter and Bortfeld curves, and the energy and range of the beam as a legend.
[https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/ https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/]

* Create a file with Manchester data
* Fix cache busting in Google Chrome
* Overlap between users in different browsers
* Determine number of points per PD for files
* Responsivity in different devices is not considered

'''Version 2.06'''

Version 2.06 has a unique start/stop button and displays live data when running a shell script via terminal.
* Zooming issue: PDdata and fitted data do not scale at the same rate, when zooming there is a shifting between the two -> Use linear scales for both ✓
* When clicking stop freeze the graph but do not make it dissapear ✓
* Modify code to give - instead of 0 when fit not performed ✓
* Plot data with 20 points ✓

'''Version 2.05'''

Version 2.05 has two separate start and stop button

* Upload files to new directory ✓
* Modify position of error bars ✓
* x-axis zooming problem: only some numbers work ✓
* Fix misalignment between PDdata and FITdata ✓
* Test access from NUC PC ✓

== Knowledge Transfer ==
- Future work in QuARC: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work]

- Summary : [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information]

== Replay + Fit ==
Two different codes are used for the data analysis: replay.py and fit.cpp, their functionalities are described below.

replay.py code calibrates and subtracts the background of each measurement, generating an output text file with header information, all the individual calibrated measurements and the average of them all. An example file with data from Clatterbridge Hospital of what the generated file looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_calibrated.txt], every individual run fit takes ~3.3s on a MacBook to be completed. All of them can be processed together by running the shell script ./replayshell.sh. To run this code we need the following arguments:
python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

fit.cpp averages the data at a user specified frequency (or generic 25 Hz) and then performs Bortfeld and Kelleter fits to the data. After the fitting, the beam energy and range are extracted for each of those averaged measurements. All of them can be processed together by running the shell script ./fitshell.sh An example of what the output of fit.cpp looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_fitted_25Hz.txt]. This code needs to be compiled, and then run with the appropriate arguments.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3
./fit reverse energy facility folder run (frequency)

To show these data in the GUI a shell script that iterates through the lines of the fitted output file is necessary.

Previously both of the codes were developed in cpp. The replay.cpp replays the DDC232 acquisition by either stepping through each measurement or averaging measurements at specified frame rate, while fit.cpp performs the fit on calibrated and background subtracted data.

== Done ==

* PARTREC pictures uploaded to: /unix/pbt/data/partrec/20221125/Pictures
* Cut Mylar foil sheets for Manchester experiment
* Talk to Derek about Mylar foil.
* Probation meeting with Simon (fill form)
* Set up new MacBook Pro: might need dual booting (talk to Tony Hoare about Windows).
* Set up Geant4 simulations of detector → copied and running
* Port Saad's FPGA data replay code to Python → copied
* Duplicate functionality of Fern's GUI in D3.

== Quick-start ==
* Send a copy of pasport/ID so they can set up a contract (Bonita Carboo). Then a videocall needs to be arranged to check passport.
* Proof of right to work (share code via gov.uk)
* Sign online contract. Your employee number is here.
* Once you have an employee number you can check your user ID and email account [https://myaccount.ucl.ac.uk/new-staff]
* Welcome email with infromation for new staff member, probation and guidance [https://www.ucl.ac.uk/human-resources/ucl-induction-and-probation-policy#Policy]
* Check and complete the induction checklist (received on the welcome email)
* Get ID card: book appointment online ✓
* Mandatory safety training courses: Safety Induction + Fire Safety ✓
* Book departamental safety induction with Lee Bebbington ✓
* Mandatory training courses: GDPR, information security, freedom of information, DSE, diversity, unconscious bias, ✓
* Create a wiki account ✓
* Register for an account on the HEP Linux cluster (email support@hep.ucl.ac.uk) ✓
* Install Geant4 (instructions: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Detector_Simulation]) ✓
* Set up bank details online on myHR ✓

== Important Dates ==

{| class="wikitable"
! style="text-align: center;" | Date
! style="text-align: center;" | Event
|-
|26th-27th October 2023|| Ion Imaging Workshop London
|-
|11th-13th October 2023|| ACCMED course Pavia
|-
|25th-28th July 2023|| FPGA Course RAL
|-
|3rd-7th July 2023|| HIT Online course
|-
|10th-16th June 2023|| PTCOG Conference Madrid
|-
|18th May 2023|| Clatterbridge beam test
|-
|5th Apr 2023|| PGI Seminar @Queen Mary
|-
|8th Feb 2023 || QuARC Seminar @Bristol
|-
|20th Jan 2023 || PhD Graduation York
|-
|30th Nov - 2nd Dec 2022 || FRPT Conference Barcelona
|-
|24-25th Nov 2022 || Groningen beam test
|-
|25-26th Oct 2022 || Manchester beam test
|-
|21st Oct 2022 || HEP Seminar
|-
|20th Oct 2022 || Prague
|-
|23rd Sep 2022 || VIVA York
|}

ELogs/SoniaEscribano

2025-08-05T14:08:54Z

SoniaEscribano: /* To Do */

Electronic Log for Sonia Escribano

== To Do ==

* Data analysis:
- Analysis of QuARC as a beam monitor
- Compare beam monitor results with PPTC
- Create a git repository with latest analysis codes

* Raspberry pi 5:
- Modify live visualisation and calibration codes to include facility
- Add more info to error messages on the status box
- Add saturation error message?
- Duplicate latest version of code to new SD cards

== GUI ==
'''Version 2.1'''

This version displays the data from a file selected via a dropdown menu (using php), the Kelleter and Bortfeld curves, and the energy and range of the beam as a legend.
[https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/ https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.1/]

* Create a file with Manchester data
* Fix cache busting in Google Chrome
* Overlap between users in different browsers
* Determine number of points per PD for files
* Responsivity in different devices is not considered

'''Version 2.06'''

Version 2.06 has a unique start/stop button and displays live data when running a shell script via terminal.
* Zooming issue: PDdata and fitted data do not scale at the same rate, when zooming there is a shifting between the two -> Use linear scales for both ✓
* When clicking stop freeze the graph but do not make it dissapear ✓
* Modify code to give - instead of 0 when fit not performed ✓
* Plot data with 20 points ✓

'''Version 2.05'''

Version 2.05 has two separate start and stop button

* Upload files to new directory ✓
* Modify position of error bars ✓
* x-axis zooming problem: only some numbers work ✓
* Fix misalignment between PDdata and FITdata ✓
* Test access from NUC PC ✓

== Knowledge Transfer ==
- Future work in QuARC: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Future_Work]

- Summary : [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Equipment/QuARC_General_Information]

== Replay + Fit ==
Two different codes are used for the data analysis: replay.py and fit.cpp, their functionalities are described below.

replay.py code calibrates and subtracts the background of each measurement, generating an output text file with header information, all the individual calibrated measurements and the average of them all. An example file with data from Clatterbridge Hospital of what the generated file looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_calibrated.txt], every individual run fit takes ~3.3s on a MacBook to be completed. All of them can be processed together by running the shell script ./replayshell.sh. To run this code we need the following arguments:
python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

fit.cpp averages the data at a user specified frequency (or generic 25 Hz) and then performs Bortfeld and Kelleter fits to the data. After the fitting, the beam energy and range are extracted for each of those averaged measurements. All of them can be processed together by running the shell script ./fitshell.sh An example of what the output of fit.cpp looks like can be found here [https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/Run001_fitted_25Hz.txt]. This code needs to be compiled, and then run with the appropriate arguments.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3
./fit reverse energy facility folder run (frequency)

To show these data in the GUI a shell script that iterates through the lines of the fitted output file is necessary.

Previously both of the codes were developed in cpp. The replay.cpp replays the DDC232 acquisition by either stepping through each measurement or averaging measurements at specified frame rate, while fit.cpp performs the fit on calibrated and background subtracted data.

== Done ==

* PARTREC pictures uploaded to: /unix/pbt/data/partrec/20221125/Pictures
* Cut Mylar foil sheets for Manchester experiment
* Talk to Derek about Mylar foil.
* Probation meeting with Simon (fill form)
* Set up new MacBook Pro: might need dual booting (talk to Tony Hoare about Windows).
* Set up Geant4 simulations of detector → copied and running
* Port Saad's FPGA data replay code to Python → copied
* Duplicate functionality of Fern's GUI in D3.

== Quick-start ==
* Send a copy of pasport/ID so they can set up a contract (Bonita Carboo). Then a videocall needs to be arranged to check passport.
* Proof of right to work (share code via gov.uk)
* Sign online contract. Your employee number is here.
* Once you have an employee number you can check your user ID and email account [https://myaccount.ucl.ac.uk/new-staff]
* Welcome email with infromation for new staff member, probation and guidance [https://www.ucl.ac.uk/human-resources/ucl-induction-and-probation-policy#Policy]
* Check and complete the induction checklist (received on the welcome email)
* Get ID card: book appointment online ✓
* Mandatory safety training courses: Safety Induction + Fire Safety ✓
* Book departamental safety induction with Lee Bebbington ✓
* Mandatory training courses: GDPR, information security, freedom of information, DSE, diversity, unconscious bias, ✓
* Create a wiki account ✓
* Register for an account on the HEP Linux cluster (email support@hep.ucl.ac.uk) ✓
* Install Geant4 (instructions: [https://www.hep.ucl.ac.uk/pbt/wiki/Proton_Calorimetry/Detector_Simulation]) ✓
* Set up bank details online on myHR ✓

== Important Dates ==

{| class="wikitable"
! style="text-align: center;" | Date
! style="text-align: center;" | Event
|-
|26th-27th October 2023|| Ion Imaging Workshop London
|-
|11th-13th October 2023|| ACCMED course Pavia
|-
|25th-28th July 2023|| FPGA Course RAL
|-
|3rd-7th July 2023|| HIT Online course
|-
|10th-16th June 2023|| PTCOG Conference Madrid
|-
|18th May 2023|| Clatterbridge beam test
|-
|5th Apr 2023|| PGI Seminar @Queen Mary
|-
|8th Feb 2023 || QuARC Seminar @Bristol
|-
|20th Jan 2023 || PhD Graduation York
|-
|30th Nov - 2nd Dec 2022 || FRPT Conference Barcelona
|-
|24-25th Nov 2022 || Groningen beam test
|-
|25-26th Oct 2022 || Manchester beam test
|-
|21st Oct 2022 || HEP Seminar
|-
|20th Oct 2022 || Prague
|-
|23rd Sep 2022 || VIVA York
|}

Proton Calorimetry

2024-12-11T17:03:13Z

SoniaEscribano: /* Inventory */

This page contains information about the UCL Proton Calorimetry project.

== [[/Experimental Runs|Experimental Runs]] ==

Details of each of the experimental runs made with the Proton Calorimetry detector can be found on the [[/Experimental Runs|Experimental Runs]] page.

== [[/Equipment|Detector Equipment]] ==

Information on the hardware used for the Proton Calorimetry systems.

== [[/Inventory|Inventory]] ==
Inventory of QuARC thingsss

== [[/Detector Analysis|Detector Analysis]] ==

Information on the code used for the QuARC data Analysis.

== [[/User Interface|Graphical User Interface]] ==

Details of the setup and design of the QuARC GUI.

== [[/QuARC Data Files|QuARC Data File Format]] ==

Information on the structure of the data files produced by the QuARC.

== [[/Detector Simulation|Detector Simulation]] ==

Information on the Geant4 code used for Proton Calorimetry simulation.

== [[/Meetings|Meeting Minutes]] ==

For minutes of Proton Calorimetry Meetings, see the [[/Meetings|Meeting Minutes]] page.

== [[ELogs|Electronic Logs]] ==

ELogs for each person working on a PBT-related research project.

== [[/Future Work|Future Work]] ==

Future work for the QuARC

Proton Calorimetry/Inventory

2024-12-11T17:01:29Z

SoniaEscribano: Created page with "Inventory of things"

Inventory of things

Proton Calorimetry

2024-12-11T17:01:14Z

SoniaEscribano: /* Inventory */

This page contains information about the UCL Proton Calorimetry project.

== [[/Experimental Runs|Experimental Runs]] ==

Details of each of the experimental runs made with the Proton Calorimetry detector can be found on the [[/Experimental Runs|Experimental Runs]] page.

== [[/Equipment|Detector Equipment]] ==

Information on the hardware used for the Proton Calorimetry systems.

== [[/Inventory|Inventory]] ==
Inventory of QuARC things

== [[/Detector Analysis|Detector Analysis]] ==

Information on the code used for the QuARC data Analysis.

== [[/User Interface|Graphical User Interface]] ==

Details of the setup and design of the QuARC GUI.

== [[/QuARC Data Files|QuARC Data File Format]] ==

Information on the structure of the data files produced by the QuARC.

== [[/Detector Simulation|Detector Simulation]] ==

Information on the Geant4 code used for Proton Calorimetry simulation.

== [[/Meetings|Meeting Minutes]] ==

For minutes of Proton Calorimetry Meetings, see the [[/Meetings|Meeting Minutes]] page.

== [[ELogs|Electronic Logs]] ==

ELogs for each person working on a PBT-related research project.

== [[/Future Work|Future Work]] ==

Future work for the QuARC

Inventory

2024-12-11T17:00:33Z

SoniaEscribano: Created page with "Cables:"

Cables:

Proton Calorimetry

2024-12-11T17:00:24Z

SoniaEscribano: /* Inventory */

This page contains information about the UCL Proton Calorimetry project.

== [[/Experimental Runs|Experimental Runs]] ==

Details of each of the experimental runs made with the Proton Calorimetry detector can be found on the [[/Experimental Runs|Experimental Runs]] page.

== [[/Equipment|Detector Equipment]] ==

Information on the hardware used for the Proton Calorimetry systems.

== [[Inventory|Inventory]] ==
Inventory of QuARC things

== [[/Detector Analysis|Detector Analysis]] ==

Information on the code used for the QuARC data Analysis.

== [[/User Interface|Graphical User Interface]] ==

Details of the setup and design of the QuARC GUI.

== [[/QuARC Data Files|QuARC Data File Format]] ==

Information on the structure of the data files produced by the QuARC.

== [[/Detector Simulation|Detector Simulation]] ==

Information on the Geant4 code used for Proton Calorimetry simulation.

== [[/Meetings|Meeting Minutes]] ==

For minutes of Proton Calorimetry Meetings, see the [[/Meetings|Meeting Minutes]] page.

== [[ELogs|Electronic Logs]] ==

ELogs for each person working on a PBT-related research project.

== [[/Future Work|Future Work]] ==

Future work for the QuARC

Proton Calorimetry

2024-12-11T17:00:12Z

SoniaEscribano: /* Detector Equipment */

This page contains information about the UCL Proton Calorimetry project.

== [[/Experimental Runs|Experimental Runs]] ==

Details of each of the experimental runs made with the Proton Calorimetry detector can be found on the [[/Experimental Runs|Experimental Runs]] page.

== [[/Equipment|Detector Equipment]] ==

Information on the hardware used for the Proton Calorimetry systems.

== [[Inventory|Inventory]] ==

== [[/Detector Analysis|Detector Analysis]] ==

Information on the code used for the QuARC data Analysis.

== [[/User Interface|Graphical User Interface]] ==

Details of the setup and design of the QuARC GUI.

== [[/QuARC Data Files|QuARC Data File Format]] ==

Information on the structure of the data files produced by the QuARC.

== [[/Detector Simulation|Detector Simulation]] ==

Information on the Geant4 code used for Proton Calorimetry simulation.

== [[/Meetings|Meeting Minutes]] ==

For minutes of Proton Calorimetry Meetings, see the [[/Meetings|Meeting Minutes]] page.

== [[ELogs|Electronic Logs]] ==

ELogs for each person working on a PBT-related research project.

== [[/Future Work|Future Work]] ==

Future work for the QuARC

Proton Calorimetry/User Interface

2024-12-06T13:30:09Z

SoniaEscribano: /* To do list for GUI */

==To do list for GUI==
* v3.3:
- Upload versions to github

- Update wiki

- Check errors

- Check differences between live calibration and post calibration

* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php ✓

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.✓

- Reverse option: it does not work, always shows same direction.✓

- Fitting of final measurement.✓

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.1 and 3.2==
These two versions had some issues because of the FTDI code, so they are skipped.

==GUI v3.01b==

Used for UCLH experiment 2024

FTDI code: combination of Sonia's + Matt’s -> FTDI_GUI.cpp

Calibrate code: calibrate_fit.cpp

Fit code: sudo ./fit_average false 100 UCLH UCLH/20240418 Run022.txt 25

- DAQ.js + DAQ.html: show data live
- Replay.js + Replay.html: show data averaged at desired frequency.

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

Instead of killing process just execute it based on a word read from a text file. The js calls the button_message.php, that writes the message to a file called buttonStatus.txt

File will either be stopping or running, based on what DAQ is doing.

when live data is being acquired, the change in average.csv is so quick that calibration can’t cope with the speed and skips some measurements

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

- Automated File name: we want to start automatically with Run001, and increase it with every run. The increase will happen when the filename box is clicked, but still allow for name modifications if desired.

– Status Messages: adding a textbox where status messages will be display.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==GUI version 2.04==
NUC PC +

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-12-06T13:29:25Z

SoniaEscribano: /* To do list for GUI */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php ✓

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.✓

- Reverse option: it does not work, always shows same direction.✓

- Fitting of final measurement.✓

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.1 and 3.2==
These two versions had some issues because of the FTDI code, so they are skipped.

==GUI v3.01b==

Used for UCLH experiment 2024

FTDI code: combination of Sonia's + Matt’s -> FTDI_GUI.cpp

Calibrate code: calibrate_fit.cpp

Fit code: sudo ./fit_average false 100 UCLH UCLH/20240418 Run022.txt 25

- DAQ.js + DAQ.html: show data live
- Replay.js + Replay.html: show data averaged at desired frequency.

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

Instead of killing process just execute it based on a word read from a text file. The js calls the button_message.php, that writes the message to a file called buttonStatus.txt

File will either be stopping or running, based on what DAQ is doing.

when live data is being acquired, the change in average.csv is so quick that calibration can’t cope with the speed and skips some measurements

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

- Automated File name: we want to start automatically with Run001, and increase it with every run. The increase will happen when the filename box is clicked, but still allow for name modifications if desired.

– Status Messages: adding a textbox where status messages will be display.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==GUI version 2.04==
NUC PC +

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-12-06T11:03:23Z

SoniaEscribano: /* GUI version 2.05 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.1 and 3.2==
These two versions had some issues because of the FTDI code, so they are skipped.

==GUI v3.01b==

Used for UCLH experiment 2024

FTDI code: combination of Sonia's + Matt’s -> FTDI_GUI.cpp

Calibrate code: calibrate_fit.cpp

Fit code: sudo ./fit_average false 100 UCLH UCLH/20240418 Run022.txt 25

- DAQ.js + DAQ.html: show data live
- Replay.js + Replay.html: show data averaged at desired frequency.

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

Instead of killing process just execute it based on a word read from a text file. The js calls the button_message.php, that writes the message to a file called buttonStatus.txt

File will either be stopping or running, based on what DAQ is doing.

when live data is being acquired, the change in average.csv is so quick that calibration can’t cope with the speed and skips some measurements

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

- Automated File name: we want to start automatically with Run001, and increase it with every run. The increase will happen when the filename box is clicked, but still allow for name modifications if desired.

– Status Messages: adding a textbox where status messages will be display.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==GUI version 2.04==
NUC PC +

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

ELogs/MechanicalEngineering

2024-12-03T11:19:42Z

SoniaEscribano: /* Components inside the box */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the peli case===

:1. 4x detector modules:
::1.1 128 polished scintillator sheets of 100x100x3mm (32 x module)
::1.2 132 mylar foil sheets of 100x100x0.006mm (1 at the beginning of each module + 1 after each scintillator)
::1.3 4x scintillator holders

:2. Electronics:
::2.1 256 photodiodes (32 sheets x 4 modules x 2 sides)
::2.2 8 DDC232 revE boards
::2.3 1 x Zmod daughter board
::2.4 1 x FPGA USB104

:3. DAQ and cabling:
::3.1 Raspberry pi5 for DAQ
::3.2 8 usb C cables: 7 to daisy chain the scintillator modules + 1 for connection with Zmod
::3.3 (Not confirmed yet) 7.5V power supplier for Zmod board (we need to test if it can be powered directly by the FPGA from the Zmod connection or not)
::3.4 5V power supply cable for FPGA
::3.5 USB for FPGA-Raspberry pi connection
::3.6 Power cable for raspberry pi5
::3.7 Ethernet cable for pi5

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T11:16:57Z

SoniaEscribano: /* Components inside the box */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the box ===

:1. 4x detector modules:
::1.1 128 polished scintillator sheets of 100x100x3mm (32 x module)
::1.2 132 mylar foil sheets of 100x100x0.006mm (1 at the beginning of each module + 1 after each scintillator)
::1.3 4x scintillator holders

:2. Electronics:
::2.1 256 photodiodes (32 sheets x 4 modules x 2 sides)
::2.2 8 DDC232 revE boards
::2.3 1 x Zmod daughter board
::2.4 1 x FPGA USB104

:3. DAQ and cabling:
::3.1 Raspberry pi5 for DAQ
::3.2 8 usb C cables: 7 to daisy chain the scintillator modules + 1 for connection with Zmod
::3.3 (Not confirmed yet) 7.5V power supplier for Zmod board (we need to test if it can be powered directly by the FPGA from the Zmod connection or not)
::3.4 5V power supply cable for FPGA
::3.5 USB for FPGA-Raspberry pi connection
::3.6 Power cable for raspberry pi5
::3.7 Ethernet cable for pi5

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T11:15:41Z

SoniaEscribano: /* Components inside the box */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the box ===

:1. 4x detector modules:
::1.1 128 polished scintillator sheets of 100x100x3mm (32 x module)
::1.2 132 mylar foil sheets of 100x100x0.006mm (1 at the beginning of each module + 1 after each scintillator)
::1.3 4x scintillator holders

:2. Electronics:
::2.1 256 photodiodes (32 sheets x 4 modules x 2 sides)
::2.2 8 DDC232 revE boards
::2.3 1 x Zmod daughter board
::2.4 1 x FPGA USB104

:3. DAQ and cabling:
::3.1 Raspberry pi5 for DAQ
::3.2 8 usb C cables: 7 to daisy chain the scintillator modules + 1 for connection with Zmod
::3.3 ?? Potential 7.5V external power supplier for Zmod board (we need to test if it can be powered directly by the FPGA from the Zmod connection or not)
::3.4 5V power supply cable for FPGA
::3.5 USB for FPGA-Raspberry pi connection
::3.6 Power cable for raspberry pi5
::3.7 Ethernet cable for pi5

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T11:08:23Z

SoniaEscribano: /* Components inside the box */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the box ===
#1. 4x detector modules:

#* 1.1 128 polished scintillator sheets of 100x100x3mm (32 x module)

1.2 132 mylar foil sheets of 100x100x0.006mm (1 at the beginning of each module + 1 after each scintillator)

1.3 4x scintillator holders

2. Electronics:
2.1 256 photodiodes (32 sheets x 4 modules x 2 sides)
2.2 8 DDC232 revE boards
2.3 1 x Zmod daughter board
2.4 1 x FPGA USB104

3. DAQ and cabling:
3.1 Raspberry pi5 for DAQ
3.2 8 usb C cables: 7 to daisy chain the scintillator modules + 1 for connection with Zmod
3.3 ?? Potential 7.5V external power supplier for Zmod board (we need to test if it can be powered directly by the FPGA from the Zmod connection or not)
3.4 5V power supply cable for FPGA
3.5 USB for FPGA-Raspberry pi connection
3.6 Power cable for raspberry pi5
3.7 Ethernet cable for pi5

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T11:08:13Z

SoniaEscribano: /* Components inside the box */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the box ===
1. 4x detector modules:

#* 1.1 128 polished scintillator sheets of 100x100x3mm (32 x module)

1.2 132 mylar foil sheets of 100x100x0.006mm (1 at the beginning of each module + 1 after each scintillator)

1.3 4x scintillator holders

2. Electronics:
2.1 256 photodiodes (32 sheets x 4 modules x 2 sides)
2.2 8 DDC232 revE boards
2.3 1 x Zmod daughter board
2.4 1 x FPGA USB104

3. DAQ and cabling:
3.1 Raspberry pi5 for DAQ
3.2 8 usb C cables: 7 to daisy chain the scintillator modules + 1 for connection with Zmod
3.3 ?? Potential 7.5V external power supplier for Zmod board (we need to test if it can be powered directly by the FPGA from the Zmod connection or not)
3.4 5V power supply cable for FPGA
3.5 USB for FPGA-Raspberry pi connection
3.6 Power cable for raspberry pi5
3.7 Ethernet cable for pi5

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T11:07:52Z

SoniaEscribano: /* Components inside the box */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the box ===
1. 4x detector modules:

1.1 128 polished scintillator sheets of 100x100x3mm (32 x module)

1.2 132 mylar foil sheets of 100x100x0.006mm (1 at the beginning of each module + 1 after each scintillator)

1.3 4x scintillator holders

2. Electronics:
2.1 256 photodiodes (32 sheets x 4 modules x 2 sides)
2.2 8 DDC232 revE boards
2.3 1 x Zmod daughter board
2.4 1 x FPGA USB104

3. DAQ and cabling:
3.1 Raspberry pi5 for DAQ
3.2 8 usb C cables: 7 to daisy chain the scintillator modules + 1 for connection with Zmod
3.3 ?? Potential 7.5V external power supplier for Zmod board (we need to test if it can be powered directly by the FPGA from the Zmod connection or not)
3.4 5V power supply cable for FPGA
3.5 USB for FPGA-Raspberry pi connection
3.6 Power cable for raspberry pi5
3.7 Ethernet cable for pi5

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T11:07:42Z

SoniaEscribano:

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the box ===
1. 4x detector modules:
1.1 128 polished scintillator sheets of 100x100x3mm (32 x module)
1.2 132 mylar foil sheets of 100x100x0.006mm (1 at the beginning of each module + 1 after each scintillator)
1.3 4x scintillator holders

2. Electronics:
2.1 256 photodiodes (32 sheets x 4 modules x 2 sides)
2.2 8 DDC232 revE boards
2.3 1 x Zmod daughter board
2.4 1 x FPGA USB104

3. DAQ and cabling:
3.1 Raspberry pi5 for DAQ
3.2 8 usb C cables: 7 to daisy chain the scintillator modules + 1 for connection with Zmod
3.3 ?? Potential 7.5V external power supplier for Zmod board (we need to test if it can be powered directly by the FPGA from the Zmod connection or not)
3.4 5V power supply cable for FPGA
3.5 USB for FPGA-Raspberry pi connection
3.6 Power cable for raspberry pi5
3.7 Ethernet cable for pi5

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T10:52:52Z

SoniaEscribano: /* Proton Therapy */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Components inside the box ===

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

ELogs/MechanicalEngineering

2024-12-03T10:52:30Z

SoniaEscribano: /* To Do */

Electronic Log for Mechanical Engineering

== 2025 Experiment ==

=== Proton Therapy ===

== To Do ==

=== Proton Therapy ===

==== Preparation for UCLH beam tests 29-30th April ====

# '''New scintillator stack holders:'''
#* Stack holders 3D printed: remake by hand if time allows.
#* Mounting plate to screw to Thorlabs optical breadboards in Peli cases:
#** Rough version already 3D printed.
#** Needs modifying to fix stack holder in place with M14(?) nuts.
#** Must be able to mount upside down.
#* New frame needed for patch panel with 50mm stand off to allow deeper connectors not to interfere with scintillator stacks.
# '''Holder for NPL Transmission Calorimeter:'''
#* TC needs to mount into Peli Case in front of stack holders.
#* CAD STEP-files already received from Sam Flynn (NPL) for existing holder.
#* If making new holder, exit pipe for electronics cabling needs to be shortened to fit in small Peli case.
#* If making holder for holder, must mount to Thorlabs optical breadboard (M6 holes on 25mm pitch?).
#* Centre of holder must be '''71 mm''' from baseplate (scintillator stacks are 50mm radius, plus 21mm offset from base).
# '''Holder for Birmingham Pixel Sensor:'''
#* '''Tony Price''' (Birmingham) has provided dimensions to '''HB'''.
#* Holder needed to support pixel sensor between TC and scintillators.
#* Peli case needs 100x10mm slot cut to left of side handle to allow sensor ribbon cable to pass through.
# '''Nozzle mount for detector enclosure:'''
#* Small Peli case needs to be nozzle-mounted for beam tests.
#* Dimensions already obtained from UCLH measurements:
#** '''SJ''' has contacted UCLH to check whether we can bypass light curtain.
#** Assume we can't and design to bypass mechanically...
#* 3 parts needed:
#** Nozzle mounting plate that sits inside nozzle using range modulator plate mounting slots.
#** U-bracket that slots in to this mounting plate — bypassing the light curtain — with a 40 x 30 cm hole for the beam to pass through.
#** Peli Case support that fixes to the U-bracket and holds the Peli Case whilst gantry is rotating.

==== May onwards ====

# '''Bari scintillating fibre holder:'''
#* Bari are designing the scintillating fibre assembly for the '''QuADProBe'''.
#* Prototype fibre array holder machined from aluminium.
#* Clinical prototype ideally needs a version in plastic.
#* Review Bari CAD drawings and recommend suitable materials.
# '''Scintillator machining:'''
#* Promising results from Datron: chase up potential workshops who could machine scintillator for us.
#* Liaise with '''Martin Blackman''' in main workshop to assess quality of his scintillator machining.
#* Write case and obtain quote for us to purchase Datron Neo.

==== Full detector scintillator holder ====

* We need to go from the prototype version bolted to the optical plate to something completely standalone.
* We can probably put it in another Peli case — I have loads — initially with an optical breadboard but eventually without.
* It needs a support structure that is not rigidly bolted to the case but will align to any windows we cut: that means clinical staff can whack the detector and it won’t break or go out of alignment. So something internally sprung.

== Completed ==

=== 2023 ===

; January-May : Enclosure design for small-scale Clatterbridge detector prototype (with Simon).
:* Includes feedthroughs, mounts for NUC PC, Raspberry Pi and USB104 FPGA.
:* Nozzle mounted

=== 2022 ===

; November-December : Design and 3D print external frame with alignment marks and alignment plates for small Peli case.
: Duplicate alignment bar from Zarges case with circular studs to allow alignment with treatment couch.

; October : Modified medium-sized Peli case ready for beam tests:
:* Install Thorlabs optical breadboard in base of case.
:* Cut out window at each end of case for mylar window installation.
:* Cut out 2 windows for 2-gang patch panels, 1 per side.
:* Cut out 2 windows for 1-gang patch panels, 1 per end.
:* 3D print internal frames for mylar windows with bolt holes in each corner. Drill bolt holes in Peli case.
:* 3D print external frame with alignment marks matching centre line of optical breadboard (vertical) and 71mm above optical breadboard (horizontal). Match bolt holes to mylar window frames.
:* 3D print alignment plates that are strong enough to double as external cover plates when shipping enclosure.

; September : Machine a patch panel complete with 10 smaller panels that can be swapped between on the main patch panel to allow for various electrical feedthroughs

; August : Created CAD cross hair for Peli Case to be printed. Also created small printed windows to clamp Mylar smooth over the beam entry point.

; July : Created CAD models for internal components including:
:* Scintillator holder.
:* 2x Side Vice faces.
:* 1x Front Vice face.
:* Cover for beam entry point.
: These were printed soon after.
: Beam entry point was cut out of the case, piece was sent for testing with Adam Gibson.
: Machined two special 1/4-20 UNC screws for camera mount.
: Tapped all holes on Scintillator holder to take m5 nylon screws to allow vice faces to travel.

; June : Initiated design of large Peli Case. Received Lab scissor jack used to mount camera.

; May : CAD Prototype a sliding system to mount multiple scintillator holders into a case

; February : Create CAD model for small Peli Case so ideas on how to install the holder into the case can be prototyped

=== 2021 ===

; November : Design initial prototype of a new holder that will clamp scintillator sheets together. Will supersede previous adhesive idea
; June : Prepare scintillator sheets to be adhered together using standard primer

Proton Calorimetry/User Interface

2024-11-26T14:59:22Z

SoniaEscribano: /* GUI v3.01b */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.1 and 3.2==
These two versions had some issues because of the FTDI code, so they are skipped.

==GUI v3.01b==

Used for UCLH experiment 2024

FTDI code: combination of Sonia's + Matt’s -> FTDI_GUI.cpp

Calibrate code: calibrate_fit.cpp

Fit code: sudo ./fit_average false 100 UCLH UCLH/20240418 Run022.txt 25

- DAQ.js + DAQ.html: show data live
- Replay.js + Replay.html: show data averaged at desired frequency.

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

Instead of killing process just execute it based on a word read from a text file. The js calls the button_message.php, that writes the message to a file called buttonStatus.txt

File will either be stopping or running, based on what DAQ is doing.

when live data is being acquired, the change in average.csv is so quick that calibration can’t cope with the speed and skips some measurements

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

- Automated File name: we want to start automatically with Run001, and increase it with every run. The increase will happen when the filename box is clicked, but still allow for name modifications if desired.

– Status Messages: adding a textbox where status messages will be display.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T14:53:41Z

SoniaEscribano: /* GUI v3.01b */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==

Used for UCLH experiment 2024

FTDI code: combination of Sonia's + Matt’s -> FTDI_GUI.cpp

Calibrate code: calibrate_fit.cpp

Fit code: sudo ./fit_average false 100 UCLH UCLH/20240418 Run022.txt 25

- DAQ.js + DAQ.html: show data live
- Replay.js + Replay.html: show data averaged at desired frequency.

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

Instead of killing process just execute it based on a word read from a text file. The js calls the button_message.php, that writes the message to a file called buttonStatus.txt

File will either be stopping or running, based on what DAQ is doing.

when live data is being acquired, the change in average.csv is so quick that calibration can’t cope with the speed and skips some measurements

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

- Automated File name: we want to start automatically with Run001, and increase it with every run. The increase will happen when the filename box is clicked, but still allow for name modifications if desired.

– Status Messages: adding a textbox where status messages will be display.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T14:29:02Z

SoniaEscribano: /* GUI v3.01 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

Instead of killing process just execute it based on a word read from a text file. The js calls the button_message.php, that writes the message to a file called buttonStatus.txt

File will either be stopping or running, based on what DAQ is doing.

when live data is being acquired, the change in average.csv is so quick that calibration can’t cope with the speed and skips some measurements

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

- Automated File name: we want to start automatically with Run001, and increase it with every run. The increase will happen when the filename box is clicked, but still allow for name modifications if desired.

– Status Messages: adding a textbox where status messages will be display.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T14:03:47Z

SoniaEscribano: /* GUI v3.01 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

Instead of killing process just execute it based on a word read from a text file. The js calls the button_message.php, that writes the message to a file called buttonStatus.txt

File will either be stopping or running, based on what DAQ is doing.

when live data is being acquired, the change in average.csv is so quick that calibration can’t cope with the speed and skips some measurements

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T13:59:19Z

SoniaEscribano: /* GUI v3.01 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:

- Starting acquisition

- Stopping when acquisition has finished

- Stop graph when acquiring data

It needs to:

4. Read if detector is not connected and show an error message.

5. Kill the execution of shell script when stop is clicked.

User input values:

$selected_value1 = RunName = Run0xy

$selected_value2 = FSR = 12.5, 350…

$selected_value3 = tINT

$selected_value4 = num_measurements

$selected_value5 = reverse

$selected_value6 = frequency

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T13:59:00Z

SoniaEscribano: /* GUI v3.01 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

DAQ.js calls executeDAQ.php calls executeFTDI.sh and FTDI2

Need to change the autoclick button: instead of clicking after a certain time it has passed, we now print measurement number + word running to a new file with each measurement, and with last one we print the stopping. If the word stopping is found in the file, the stop button will be automatically clicked. When detector not connected: show an error message to the user (very basic styling for now) + click button to stop acquisition.

Now Measure/Stop button is capable of:
- Starting acquisition
- Stopping when acquisition has finished
- Stop graph when acquiring data

It needs to:
4. Read if detector is not connected and show an error message.
5. Kill the execution of shell script when stop is clicked.

User input values:
$selected_value1 = RunName = Run0xy
$selected_value2 = FSR = 12.5, 350…
$selected_value3 = tINT
$selected_value4 = num_measurements
$selected_value5 = reverse
$selected_value6 = frequency

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T13:54:13Z

SoniaEscribano: /* GUI v3.3 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).
This version was last modified by Sonia on the 26th of November, 2024.

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T13:53:51Z

SoniaEscribano: /* GUI v3.3 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages: one for data acquisition (DAQ), one for calibrating data at a desired frequency (Calibrated) and one for the display of previosuly fitted data (Replay).

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T13:52:12Z

SoniaEscribano: /* REPLAY */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Replay code is a visualisation tool only. It reads the fitted data, selected from a dropdown menu, and plots every measurement with its fits and values.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T13:35:12Z

SoniaEscribano: /* CALIBRATED */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.
The graph shows the file output_GUI_Calibrated.txt, that is updated in real time at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Visualisation tool only.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T13:34:30Z

SoniaEscribano: /* CALIBRATED */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency, selected from a dropdown menu. It calls calibrateGUI.php, that then calls executeGUI.sh. The shell script calls the cpp code calibrate_rate, that reads the raw data file, calibrates the data and the averages it at the desired frequency.

sudo ./calibrate_rate true "$dataDir" "$shortFileName" revB "$frequency"

=== REPLAY ===
Visualisation tool only.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:49:31Z

SoniaEscribano: /* GUI v3.3 */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

=== CALIBRATED ===
Performs calibration at a desired frequency

=== REPLAY ===
Visualisation tool only.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:47:55Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt The actual code work as it follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:47:35Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt.

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt
The actual workflow is as follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:38:12Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt. The actual workflow is as follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php sends the word running or stopping to buttonStatus.txt, based on the number of clicks the start/stop button has received.

uttonStatus.txt??

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:34:44Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt. The actual workflow is as follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages. IF the file includes the word stopping, acquisition has finished and the start/stop button will be clicked automatically.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php

uttonStatus.txt??

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:33:36Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt. The actual workflow is as follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh. The shell script acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php

uttonStatus.txt??

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:28:47Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt. The actual workflow is as follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh.

executeFTDI.sh acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages.
sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php

uttonStatus.txt??

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:28:33Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

To run the GUI we need DAQ.js and DAQ.html. The graph in this webpage displays output_GUI.txt. The actual workflow is as follows:

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh.

executeFTDI.sh acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages.

sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php

uttonStatus.txt??

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:27:18Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

DAQ.js: . Plots live output_GUI.txt

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh.

executeFTDI.sh acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages.

sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php

uttonStatus.txt??

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:27:06Z

SoniaEscribano: /* DAQ */

==To do list for GUI==
* v3.1:

-MaxY = 1.5*Photodiode ✓

- Status message of running situation for users ✓

- Start/Stop button to finish when clicked, not after tinterval ✓

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

==GUI v3.3==
This version of the GUI has three separate webpages:

=== DAQ ===

This section handles the data acquisition of the QuARC. The arguments needed for the different codes to run correctly can be directly input by the user in the different textboxes and dropdown menus.
Single start/stop button that handles the raw acquisition, the live calibration and the postprocessing of the data (fitting it at 25 Hz).

DAQ.js: . Plots live output_GUI.txt

The start/stop button calls executeDAQ.php, that then sends the input valus to the shell script executeFTDI.sh.

executeFTDI.sh acquires data live, calibrates it and then fits the data in three separate steps.

1. Data Acquisition: FTDI_GUI.cpp, the code responsible for communicating the FPGA over FTDI that creates the raw file and the date folder. This code also creates the subfolders Raw, Calibrated and Fitted.
Three important files, that the GUI will need, are created with this code: first one is GUI/average.csv, that simply writes live raw data in decimal format that will be used for calibration. The second file is runningStatus.txt, this shows the message created by FTDI code: whether the detector is or not connected, the number of measurements that are acquired and/or any other error messages.

sudo ./FTDI_GUI usb104 1 "$selected_value2" "$selected_value3" "$selected_value4" 50 "$selected_value1.txt" revB 2 &

2. Calibration: calibrate_fit_GUI.cpp and ./calibrate_singlefit.cpp. The first code, calibrate_fit_GUI.cpp, reads from GUI/average.csv and subtracts background and applies ST calibration. It generates a live file, output_GUI.txt, that is updated with the calibrated data live for GUI visualisation. IT also generates a file with all the calibrated data, but due to the acquisition speed some lines are skipped, that's why we need the second code.

calibrate_singlefit.cpp: reads the raw data file, calibrates it and creates a new file with appropiate header information. At the end of the file, the average of all the measurements as well as a fit applied to this average will be generated.

sudo ./calibrate_fit_GUI "$selected_value1.txt" 1 "$selected_value2" "$selected_value4" revB "$selected_value5" "$selected_value6" "$selected_value3" calibration background.txt shootBack.txt shootFront.txt 2>&1

sudo ./calibrate_singlefit "$date" "$selected_value1.txt" revB "$selected_value3" "$selected_value5" background.txt shootBack.txt shootFront.txt

3. Fitting: once the calibrated file has been generated, with header info, one line per measurement and the averaged value we can proceed to do more fittings. This code averages the data at 25Hz and performs an individual fit for every set of data averaged at 25Hz. The output file generated is Calibrated/YYmmdd_Run0xy_fitted.txt.

sudo ./fit_average "$selected_value5" 100 UCLH "$date" "$selected_value1.txt" "$selected_value6" & #Value 100 is energy, currently not in use.

button_message.php

uttonStatus.txt??

File structure: there is an entry textbox, that when clicked assigns the name Run0xy. Three files are produced: YYmmdd_Run0xy_raw.txt, YYmmdd_Run0xy_calibrated.txt and YYmmdd_Run0xy_fitted.txt

==GUI v3.01b==
Used for UCLH experiment 2024

==GUI v3.01==
This version has been developed on raspberry pi5. The goal of this version is to fix the pending issues of v2.3, specially the behaviour of the Start/Stop button as this has to know when the DAQ is running and how to kill it.

As this version is the most stable one with php and sh scripts, below is a detailed explanation of how it works.

==GUI v2.3==
This version has been developed on raspberry pi5.
The code has been split in two main functions: one that creates the graph and one that updates it, to keep up to speed with the acquisition. A summary of the changes can be found on the slides created on the 20th March 2024 (SER_240320_GUI-v2.3.pptx): https://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2024/

Main changes:

- Single start/stop button for DAQ.

- t_INT has a minimum value of 170us. If user inputs a value smaller than this, an error message is displayed and tINT automatically modified to 170us.

- Run time entry box added. This correlates to the number of measurements considering the t_INT decided by the user.

- Auto-adjustment of the y-axis and the x-axis based on the data that is being parsed.

Pending:

- Status message of running situation for users

- Start/Stop button to finish when clicked, not after tinterval

- Kill shell script from php

- Error Bars: check how values are obtained in FTDI

- Cache issue, situation has improved but there are some crashes.

- Reverse option: it does not work, always shows same direction.

- Fitting of final measurement.

(This version has accidentally overwritten v2.2 and therefore is saved on pi5 as GUI v2.2)

==GUI v2.2==

This version keeps the data display from previous runs that was implemented in version 2.1, and adds elements to perform the data acquisition directly from the GUI.
Initial test was to design a simple execute DAQ button that reads the dropdown menus and text boxes with the values defined by the user and uses a shell script to write them to a file called boardvalues.txt.

This version of the GUI can be found on: https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.2/

'''GUI v2.2 on pi5'''
The newly acquired raspberry pi5 presents a much better reading time than previous version pi4. The GUI is going to be developed directly in this device, as it can be attached to the detector with the intention of acquiring and displaying data live. To be able to execute the FTDI code from a shell script we need to add a passwordless rule in sudoers for the script (/etc/sudoers.d/FTDI_rule) and this allows execution.

We want to acquire data and calibrate it when the button is clicked: new code, calibrateSonia.cpp does the calibration and generates the files that are read by the GUI.

- The arguments required for .FTDI that do not change are hardcoded into the shell script.
- The arguments that vary (number of measurements) are read from the textboxes/dropdown menus.

Acquisition from local host using a remote laptop has been successful. This version of the GUI has many changes in comparison to cluster v2.2, so a new version number (2.3) is given and will be described in more detail in a separate section.

==GUI version 2.1.b==
This version has fixed all of the issues we had in previous versions. Design is the same but the code has been rewritten completely. The key improvements are:

* Zooming feature: Shifting between PDdata and curves when zooming in: two different axis were used for PDdata and fitted data. This has been modified to have only one xScale, defined as a linear scale. This means that the x-axis now utilises numerical datasets instead of categorical, and therefore the zooming of both sets of data is performed equally at the same time.
var xScale = d3.scaleLinear()

.domain([0, setWET])

.range([margins.left, width - margins.right ]);

* Bandwidth separation: Previous version assumed that the separation between PDdata was uniform, but this did not correspond to reality as the PD thicknesses varies. The bandwidth needs to be redefined based on the separation between each PD and the previous one.

bandwidths.push(xPDposition[i+1] - xPDposition[i]);

''This version has been developed for Clatterbridge data (12PD) - If using for larger data sets we need to modify the numer of setPhotodiodes and setWET in GUI.js ''

==GUI version 2.06==
Located on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.06/ </code>

Improved version based on v 2.05. The start and stop buttons have been changed by a single button that performs both actions, keeping in screen the results from the last data plotted instead of deleting the graph and beam parameters.

Live update + fitting curves.

==GUI version 2.05==
This version of the GUI can be seen on: <code> https://www.hep.ucl.ac.uk/pbt/PDdisplay/escribano/v2.05/ </code>

'''General information'''

Developed in JavaScript with d3 and papaparse libraries. This version consists of a single tab, where the user can adjust WET number (mm) and max Y (pC) and select which elements are plotted on the graph, via checkboxes.

Data used for testing GUI: 245 MeV, acquired at The Christie in October 2022

Disclaimer!: The error bars from this version do not correspond to the real error value, as the magnitude has been increased by 5pC for visualisation purposes (defined in the code as ''extraerror'').

'''Live update'''

This version provides a live visualisation of the data file. In order to generate a dynamic set of data this two scripts have been developed in Python: GUIrandom.py and runauto.py.

- GUIrandom.py: calculates a random number and multiplies the PD data and the curves by this number. The random number is shown in the terminal.

- runauto.py: runs GUIrandom.py at the desired time interval (initially 0.02s, to have a refresh rate of 50Hz)

==Files for GUI visualisation==

The experimental data acquired needs to be calibrated with the acquired shoot-through measurements (back and front) and the background needs to be subtracted. Then, each of the measurements can be fitted to the models, extracting the range and the energy of the beam. There are two different codes to perform this.

'''Calibrated data: '''
Development of a python code (replay.py) that generates a file with header info followed by the 30k measurements calibrated and with background subtracted. If our initial file is called Run0xy.txt the output of this will be Run0xy_calibrated.txt

python3 replay.py t_INT framerate skipahead reverse folder run background backST frontST

'''Fitted data: '''
Once the data has been calibrated, we need to perform the fittings and extract the parameters. This code (fit.cpp) utilises Run_0xy.txt and outputs Run0xy_fitted_frequency.txt, where the frequency value is 25Hz by default but can be changed by adding an extra argument to the code line. This takes about 15 second per run for Clatterbridge data.
g++ -o fit fit.cpp `root-config --cflags --glibs` -O3

./fit reverse energy facility folder run (frequency)

Both, fitted and calibrated data for all the runs acquired at Clatterbridge experiment in 2023 are in <code> https://www.hep.ucl.ac.uk/pbt/wikiData/data/GUI-Clatterbridge/ </code>

'''Live Update: '''
The webpage is reading from file outputGUI.csv, located in the same directory than the GUI js and html files. To visualise it live, we create a shell script that goes through each measurement at a certain time interval, writing the output to the outputGUI.csv file. To execute this, simply we cd into ReplayFinal and then execute the script.
./shellGUI.sh

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: <code> cd /Path/To/Data </code>
**Generate the photodiode data with: <code> root -l "writeCSV.cpp" </code>
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: <code> tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv' </code>
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: <code> cd /unix/www/html/pbt/PDdisplay/ </code>
**Run the following script in that directory: <code> ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02 </code>
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the <code>-v</code> flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

Proton Calorimetry/User Interface

2024-11-26T11:21:49Z

SoniaEscribano: /* DAQ */

Proton Calorimetry/User Interface

2024-11-26T11:21:04Z

SoniaEscribano: /* DAQ */

Proton Calorimetry/User Interface

2024-11-22T15:08:36Z

SoniaEscribano: /* GUI v3.03 */

Proton Calorimetry/User Interface

2024-11-22T15:05:55Z

SoniaEscribano: /* GUI v3.01 */