Visualisation runs in its own thread receiving events one by one from the multiple worker threads, and holds up the worker threads if the queue for drawing gets full. Did you get any warning messages such as?
G4WT2 > WARNING: The number of events in the visualisation queue has exceeded the maximum, 100. If, during a multithreaded run, the simulation gets ahead of the visualisation by more than this maximum, the simulation is delayed until the vis sub-thread has drawn a few more events and removed them from the queue. You may change this maximum number of events with "/vis/multithreading/maxEventQueueSize ", where N is the maximum number you wish to allow. N Alternatively you may choose to discard events for drawing by setting "/vis/multithreading/actionOnEventQueueFull discard". To avoid visualisation altogether: "/vis/disable". And maybe "/tracking/storeTrajectories 0".
So either /vis/disable or /vis/multithreading/actionOnEventQueueFull discard before running lots of events.