Hi! I am running a Geant4 application using an HPC system for the first time. I have the code set-up to output sensitive detector hits in CSV format, which works when I run it locally on my laptop, but I am not getting any CSV output on the cluster. Is there any extra definition I am missing for cluster running? Thank you!
- Are you running G4 “sequential” (single thread, not master-worker) or G4 multithreaded (master and worker threads)?
- Are you opening and writing to the CSV file yourself, or are you using the G4analysis interface to write to a file? If you’re doing the I/O yourself, by hand, then you need to make sure the file is only opened once (typically by the master thread), and that writing to the file is done in a global singleton protected by mutexes.
I’m currently running single-threaded as the Geant4 installed on the HPC system is set-up only for this at the moment. I have the analysis manager opening and writing the CSV file.
You’re using the same G4 version on both your local laptop and the cluster? Same macro file? Does the job run on the cluster give any kind of error messages?
Is the job really large? Are you perhaps exceeding your disk quota on the HPC cluster?
A couple other things to check:
Does your HPC system require your files to be in a specific location? Our system requires all input and output files to be located in (or automatically migrated to) the system scratch directory.
Check the runtime environment for the case when you run on the cluster. You might not have the same environment variables. And specifically, does the geant4.sh script get sourced? Since the cluster is probably not an interactive shell, it may not be running with the same environment. One way I do this is to run a script that calls
env > environment.txt before running the GEANT4 program.