In our simulation, we have a framework to attached SDs to any of the volumes in the geometry, and we are also able to define arbitrary “scoring volumes” (not the built-in Geant4 scorers) in a parallel world, in order to record the same kind of complex Hit objects as we used normally.
The use of those “scoring volumes” is somewhat limited in our simulations, so we only recently noticed that they do not appear to function correctly with MT builds (Geant4 10.06.p03-MT in particular).
With verbose printout, I see that our SD is properly created and attached to the parallel-world “scoring volume”, when our ParallelWorld::ConstructSD() function is called.
With /tracking/verbose, I see that steps are properly limited at the boundary of one of our “scoring volumes”.
However, the “scoring volume” SD::Hit() function is not being called at those scoring-volume boundaries. In fact, it never gets called at all, neither for boundaries at the parallel-world volume nor for any other steps.
Is this a known problem, where SDs in the parallel world are just not active? Is there an extra action that we need to take with the parallel world to “turn on” SDs associated with the parallel word?
I don’t believe this is a problem with our application code. Building the identical executables, and using the same macro, with a non-MT (sequential) build of G4 shows that the same SDs work properly, and we get the expected output.
I certainly spoke too soon. The problem is precisely in our application code. Our ParallelWorld builder collects a list of all the LVs which need to have SDs attached to them. In ConstructSD() it iterates through this list, attaches an SD to the volume, then removes the volume from the list. When the list is empty, the loop terminates.
That’s fine in sequential mode, but it’s very very bad in MT. ConstructSD() is first called in the master thread, which consequently empties the list. So when the worker threads call it, they find nothing to do, so there are no SDs and no hits get recorded. D’Oh. But even if the master thread hadn’t been called, all the worker threads would be sitting there emptying a shared container simultaneously, without mutexes, and life would be very bad.
I’ll mark this as Solved, and fix our application code accordingly.