In our experiment, we have a number of custom G4VHit subclasses, each with their own allocator type, as well as a corresponding hits collection.
I’m running into severe memory usage problems. One of my hit types is just 48 bytes (two ints and five doubles), but we can make millions of them in a single event. A long while back I validated the size and memory usage on small events: 1.06 million hits used about 45 MB of memorty (including the detector geomery) in a job with one worker thread.
I’m now generating much larger events (up to about 20 million of these hits per event), and the memory has exploded. I’m seeing 25-30 GB per thread (where I should see closer to 1 GB per thread), which of course blows away my 20-thread jobs. I can’t even run a 10-thread job without having it killed OOM.
Is there some non-code documentation for how both the G4AllocatorPool and the G4VHitsCollection increase their memory usage? I’m concerned that one or the other may be doing what std::vector does, and doubling in size on re-allocation. If that is the case, then I would really like to preallocate a size that I can compute based on the job configuration (source, detector voltage, etc.).
Hi Mike, the use of G4Allocator is in general suggested for objects of small fixed size that gets frequently allocated and deallocated. In such case, the pool grows by chunks of 1Kb, taking into account memory alignment. For large objects the chunk size is set to be 10 times the object’s size. I would certainly not recommend its use in your case, as it is not supposed to be used that way, if your objects are large and have a relatively long lifetime. You may need to implement your own way to store/allocate the hits, or eventually handle them and flush the pool from time to time.
Thanks, Gabriele! My objects are small (48 bytes each), and have a lifetime of one event (subclass of G4VHit, put into a hits collection). My problem is that we have millions of them in each event. It might be more efficient for me to try making use of G4Allocator::IncreasePageSize() in my initialization to reduce fragmentation.
From your description, the pool grows linearly with need, not exponentially, which is what I was mainly worried about.