Merge and save data during run

Recording, Visualizing and Persisting Data
1

Hello,

I created a MT application that during the simulation save some information for each simulated event.
At the end of the run, the Master calls the method “Merge” and then I can save all the data in a file.

Here is my question:

If I simulate too many events, the amount of RAM needed to store all the information is too high.
I’d like to know if it is possible to perform a “Merge”, save the data on disk and reset the variables that store the data in the Workers, every X simulated events.

So, for example, if I execute /run/beamOn 1000000, every 100 k events, the Merge method is called, the master saves the data and the RAM is freed, ready for the next 100 k events.

It would be possible?

Thanks for any suggestion

Andrea

2

I suggest you simply save the data at end of each event. You have to be careful to lock the writing process when multithreading.

#ifdef G4MULTITHREADED
#include "G4Threading.hh"
#include "G4AutoLock.hh"
#endif
#include "G4SystemOfUnits.hh"
...
  // collect energy deposited in this step
  G4double eDep = step->GetTotalEnergyDeposit();
  fEventAction->AddEdep(eDep);
#ifdef G4MULTITHREADED
  static G4Mutex stuffMutex = G4MUTEX_INITIALIZER;
  G4AutoLock al(&stuffMutex);
#endif
  static std::ofstream stuff("stuff.csv");
  static bool first = true;
  if (first) {
    first = false;
    stuff << "#,eDep/keV,..." << std::endl;
  }
  stuff << ',' << eDep/keV /* ... */ << std::endl;
3

Thank you.

I thought about using the mutex, but I fear that since all worker have to access the same file on disk so frequently, there would be a queue, and thus the simulation speed would be affected.