Ntuple does not exist

Sorry even though I can run similar code in other examples I am getting this error and can’t tell what the difference may be.

-------- WWWW ------- G4Exception-START -------- WWWW -------
*** G4Exception : Analysis_W001
      issued by : G4RootPNtupleManager::GetActivation
ntuple description id= 0 does not exist.
*** This is just a warning message. ***
-------- WWWW -------- G4Exception-END --------- WWWW -------

I even expressly activate the ntuple with this

analysisManager->SetNtupleActivation(0,1);

Both in booking and filling.

Any help to identify the issue would be much appreciated.

_Geant4 Version:_11.3
Operating System:
Compiler/Version:
CMake Version:


Hello,

Ntuples are active by default, so the problem must be somewhere else.

Can you let us know at where (Class::Method) you create your ntuple(s) and where you fill them ? Also, it can be useful to know, which Geant4 example did you get inspired in your use case.

Best regards,

Hello,

It’s a modified neutron source example. I create the ntuple in the histomanager and fill from stepping manager via a pointer to the class. The booking member function of histomanager has an activation function so thats why while I initially had the ntuple creation in there and manually activated as I mentioned previously I moved it outside of that function and created another one just for the ntuples though I am not sure if the activation parameter still applies, either way I still use the activation function mentioned.

Can you post your code where you create your ntuple(s):

void HistoManager::Book()
{
// ... skipped
your code ?
}

And the code where you fill them ?

void SteppingAction::UserSteppingAction(const G4Step* aStep)
{
//... skipped
your code ?
}

You can also try to increase the verbose level to 4 in your HistoManager::Book():

  analysisManager->SetVerboseLevel(4);

This will produce a detailed information about all created analysis objects. As it prints information with each Fill, reduce your number of
events to 1, and eventually also the number of primaries, if it is large.

Best regards,

I am getting this with verbosity 4

G4WT5 > ... going to create ntuple booking : Secondary Generation 
G4WT5 > ... create ntuple booking : Secondary Generation ntupleId 0 - done
G4WT5 > ... going to create ntuple T column :  ntupleId 0 
G4WT5 > ... create ntuple T column : ParticleName ntupleId 0 - done
G4WT5 > ... going to create ntuple T column :  ntupleId 0 
G4WT5 > ... create ntuple T column : ProcessGen ntupleId 0 - done
G4WT5 > ... going to create ntuple T column :  ntupleId 0 
G4WT5 > ... create ntuple T column : PosX ntupleId 0 - done
G4WT5 > ... going to create ntuple T column :  ntupleId 0 
G4WT5 > ... create ntuple T column : PosY ntupleId 0 - done
G4WT5 > ... going to create ntuple T column :  ntupleId 0 
G4WT5 > ... create ntuple T column : PosZ ntupleId 0 - done
G4WT26 > Ntuple of id 0 was made 
G4WT26 > Booking ntuple activation 1
G4WT26 > Booking number of ntuple 1
G4WT13 > Histograms are booked

It continues posting messages about the activated histograms being filled but then ends with this error with this level of verbosity.

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)

This is my function to create the ntuple.

void HistoManager::SecondaryGenerationPositionNtuple()
{
  G4AnalysisManager* analysisManager = G4AnalysisManager::Instance();
  // Create ntuples.
  // Ntuples ids are generated automatically starting from 0.
  // The start value can be changed by:
  //analysisManager->SetFirstNtupleId(1);

  // Create 1st ntuple (id = 0)
  // create an ntuple of secondary Tritium
//  if (volumeReps > 0){
    //for (G4int k=1; k<=volumeReps; k++){
      analysisManager->CreateNtuple("Secondary Generation", "Secondary Coordinates");
      analysisManager->CreateNtupleSColumn("ParticleName");//column id 0
      analysisManager->CreateNtupleSColumn("ProcessGen");  //column id 1
      analysisManager->CreateNtupleDColumn("PosX");        //column id 2
      analysisManager->CreateNtupleDColumn("PosY");        //column id 3
      analysisManager->CreateNtupleDColumn("PosZ");        //column id 4
      analysisManager->FinishNtuple(); 
    //}
//  }
  //analysisManager->SetNtupleFileName(0,"TritiumLocation");
  analysisManager->SetNtupleActivation(0,1);
  G4cout << "Ntuple of id " << analysisManager->GetFirstNtupleId() << " was made " << G4endl;
  G4cout << "Booking ntuple activation " << analysisManager->GetNtupleActivation(0) << G4endl;
  G4cout << "Booking number of ntuple " << analysisManager->GetNofNtuples() << G4endl;
}

called here

HistoManager::HistoManager()
{
  Book();
  SecondaryGenerationPositionNtuple();
}

and filled here

void HistoManager::FillSecondaryNtuple(G4int fSampleRepetitions, G4String fParticleName, G4String fProcGen, G4double fPosX, G4double fPosY, G4double fPosZ)
{
  G4AnalysisManager* analysisManager = G4AnalysisManager::Instance();
  // Fill 1st ntuple ( id = 0)
  /*for (G4int volumeReps=1; volumeReps<=fSampleRepetitions; volumeReps++)
  {*/
  analysisManager->SetNtupleActivation(0,1);
    G4cout << "filling ntuple " << G4endl;
    G4cout << "Filling ntuple ID is " << analysisManager->GetFirstNtupleId() << G4endl;
    G4cout << "ntuple directory name is " << analysisManager->GetNtupleDirectoryName() << G4endl;
    G4cout << "ntuple activation " << analysisManager->GetNtupleActivation(0) << G4endl;
    analysisManager->FillNtupleSColumn(0, fParticleName);
    analysisManager->FillNtupleSColumn(1, fProcGen);
    analysisManager->FillNtupleDColumn(2, fPosX);
    analysisManager->FillNtupleDColumn(3, fPosY);
    analysisManager->FillNtupleDColumn(4, fPosZ);
    analysisManager->AddNtupleRow(0);
  // Fill 2nd ntuple ( id = 1)
  // Lets try and do this per sample Volume
  //analysisManager->FillNtupleDColumn(1, 0, trackLAbs);
  //analysisManager->FillNtupleDColumn(1, 1, trackLGap);
  //analysisManager->AddNtupleRow(1);
  //}
}

from stepping action as follows:

  fSecondary = steppingManager -> GetfSecondary();

  // Iterate across the size of this vector for all secondaries?
  for(size_t lp1=0;lp1<(*fSecondary).size(); lp1++)
  { 
    // Retrieve the info about the generation of secondary particles 
    G4int copyNumber = (*fSecondary)[lp1] -> GetVolume() -> GetCopyNo(); // copy of volume where secondary was generated 
    G4String volumeName = (*fSecondary)[lp1] -> GetVolume() -> GetName(); // volume where secondary was generated 
    G4String secondaryParticleName =  (*fSecondary)[lp1]->GetDefinition() -> GetParticleName();  // name of the secondary
    G4String process = (*fSecondary)[lp1]-> GetCreatorProcess()-> GetProcessName();   // process creating it
    G4double charge = (*fSecondary)[lp1] -> GetDynamicParticle() -> GetDefinition() -> GetPDGCharge();
    G4int AA = (*fSecondary)[lp1] -> GetDynamicParticle() -> GetDefinition() -> GetBaryonNumber();
    //
    // Secondary generation location      
    G4ThreeVector posXYZ = (*fSecondary)[lp1]->GetVertexPosition();

    //Testing with larger volume!
    //if (volumeName == "DiaVol_phys")
	  //	 G4cout << "Particle in = " << volumeName << G4endl;
	    if (/*(secondaryParticleName == "proton") ||
               (secondaryParticleName == "neutron")||
               (secondaryParticleName == "alpha") ||
               (secondaryParticleName == "deuton") ||*/ 
               (secondaryParticleName == "triton") /*|| 
               (secondaryParticleName == "He3") || 
	       (secondaryParticleName =="GenericIon")*/)
      {
        G4cout << "the copy number is " <<copyNumber << G4endl;
        fHistoManager->FillSecondaryNtuple(copyNumber, secondaryParticleName, process, posXYZ.getX() / mm, posXYZ.getY() / mm, posXYZ.getZ() / mm ); 	
      }
  }

using this pointer

HistoManager* HistoManager::GetPointer()
{
  if (nullptr == fManager) {
    static HistoManager analysisManager;
    fManager = &analysisManager;
  }
  return fManager;
}

I have some data from a previous example filled from stacking action


Using basically the same code so I don’t really know whats wrong.

Also can csv ntuples be merged?

I think that the problem is in your HistoManager::GetPointer() function; as the HistoManager is instantiated per thread. You need to add G4ThreadLocal to your static declaration.

And no, merging csv ntuples is not supported.

1 Like

I think this instantiation per thread is the problem. I used the detector construction messenger which is called as I see from your slides on the master thread only, therefore only once to amend the number of ntuples and histograms created but when the histomanager is run again on each worker thread the number of histograms to be created is reset so I think thats where the errors are coming from. Do you have a tip on how to address this? fixed this by making the int a static variable but its updated after the histograms are booked in run action.

Is there a way to call run action after detector construction? I fixed this by splitting these histograms and ntuples to be generated only in BeginOfRunAction. Is this a good way to go about this?

Additionally I am accessing this G4ThreadLocal pointer of the histomanager from stepping action. It wasn’t clear to me from the multithreading notes if this will incur a serious performance penalty.
Thanks for your help.

PS: Any chance you could also record the materials from your course?

Update 2: The warning message issue is unresolved. It just seems like it randomly goes back to inactivated state for some threads giving this warnings but the ntuple is filled normally, maybe from other threads.

Is it possible one thread is trying to write into an ntuple that hasn’t been booked yet in another thread?
Update 3: If I ignore all worker threads then this message disappears…

I think this instantiation per thread is the problem.

The instantiation of G4AnalysisManager per thread is required by design. In case of ROOT output, the instances per thread send data to master and data are merged on fly. In case of other output types, the data are written in output files in parallel.

I used the detector construction messenger which is called as I see from your slides on the master thread only

Could you, please, point me which slide do you mean ?

I fixed this by splitting these histograms and ntuples to be generated only in BeginOfRunAction. Is this a good way to go about this?

It is recommended to create histograms and ntuples in the RunAction constructor, as they are not destroyed with the end of run.

PS: Any chance you could also record the materials from your course?

We do not record the speakers during the Orsay tutorial.

Update 2: The warning message issue is unresolved. It just seems like it randomly goes back to inactivated state for some threads giving this warnings but the ntuple is filled normally, maybe from other threads.

Can you limit the output to one thread only:

/control/cout/ignoreThreadsExcept 0

an upload the complete output (after zipping the file) ?

Is it possible one thread is trying to write into an ntuple that hasn’t been booked yet in another thread?

This should not happen, each thread books ntuples first, then proceed with their filling.

Update 3: If I ignore all worker threads then this message disappears…

How do you do this ?

Can you, please, also post your Geant4 version ? Thank you.

I understand this and I looked into the the class source and I found the mutexs and singletons that make the instance threadsafe from what I gather. I was concerned that since I am using a function to fill the ntuples from HistoManager that i wrote myself that I may not have implemented it in a thread safe way and it may be writing to incorrect threads.

This is the slide:


This was pertaining to my comment on the messenger. If I instantiate my HistoManager in the constructor then detector construction seems to not have been run yet, or perhaps my macro inputs haven’t been executed yet? As such the number of ntuples and histograms I want to generate as a function of the number of samples are not.

That is unfortunate but I do appreciate the slides you publish. It would be nice if you could record also
output.txt (32.1 KB)
even if it its just audio.
It won’t let me upload a zip file. Either way here is the output in txt form. The error doesn’t always appear on this thread so it takes a few times to get it. The program also segfaults if I increase the events to 100000.

If you ignore threads greater than the number of available cores then all worker threads get ignored.
I am running G4.11.3

Thank you for your information, I believe that the problem is in your HistoManager class implementation.
In your output we see that the ntuple is not created. I suggest to drop your HistoManager and move the code to RunAction and SteppingAction:

  • the code where you create your histograms & ntuples in RunAction constructor
  • the code where you fill the ntuple in SteppingAction::UserSteppingAction()

Best regards,

I see. Is there some advice on making a threadsafe class and accessing it? Im sure there will be other occasions for this also. TestEM8 and TestEM9 seem to be able to do this and I can’t really see what I am doing wrong.
From TestEM8 TestParameters.hh

public:
  // With description

  static TestParameters* GetPointer();


private:

  static TestParameters* fManager;

TestParameters.cc implementation

TestParameters* TestParameters::fManager = nullptr;

TestParameters* TestParameters::GetPointer()
{
  if(!fManager) {
    fManager = new TestParameters();
  }
  return fManager;
}

My modified HistoManager class header:

public:
  static HistoManager* GetPointer();

private:
  static G4ThreadLocal HistoManager* fHistoManager;

Implementation

G4ThreadLocal HistoManager* HistoManager::fHistoManager = nullptr;

HistoManager* HistoManager::GetPointer()
{
  if (!fHistoManager) {
    fHistoManager = new HistoManager();
  }
  return fHistoManager;
}

To me this line that I used to check the ntuple activation found in the function that fills the ntuple:

 G4cout << "ntuple activation of id  " << fSampleRepetitions << " is " 
    << analysisManager->GetNtupleActivation(fSampleRepetitions) << G4endl;

Suggests with the output:

G4WT20 > ntuple activation of id  1 is 
-------- WWWW ------- G4Exception-START -------- WWWW -------
*** G4Exception : Analysis_W001
      issued by : G4RootPNtupleManager::GetActivation
ntuple description id= 1 does not exist.
*** This is just a warning message. ***
-------- WWWW -------- G4Exception-END --------- WWWW -------

G4WT20 > 0

That for some threads the ntuple has not being activated by the time it comes to filling it, but I do not understand how that is possible as I book it in BeginOfRunAction

Finally how do we resolve the question of dynamic ntuple generation if histograms are booked in run actions constructor prior to detector messenger instantiation?

Parallelism in Geant4: multi-threading capabilities — Geant4 Documentation 11.3 documentation along with links in there to TWiki pages on specific topics.

In the CDMS experiment, we did exactly the kind of thing you’re trying to do (a singleton, mutex-protected ROOT manager) so that we could utilize ROOT TTrees directly.

I suspect the “big picture” problem is that the G4AnalysisManagers are already designed to take care of thread safety and multithread merging in their own way. Your HistoManager is conflicting with that. If you want to use G4Analysis, you’ve got to play by it’s rules. If you want to do your own output management, you sure can, but then you have to make your own design choices.

1 Like

Thank you for your reply! Is this available in the examples? Perhaps in the phonon example?

Filling from stepping action as below makes no difference. Looks like its something to do with booking in begin of run action even when run on a single thread (different threads get activated in system resources. Not sure if this matters).

But if it is the booking and I need to shift that to the RunAction constructor as Ivana suggested, how should I get the number of detectors? It looks like that section of code gets executed before the DetectorConstruction even if I directly try to book the ntuples there without an additional class.

  analysisManager->FillNtupleSColumn(copyNumber,0,secondaryParticleName);
  analysisManager->FillNtupleSColumn(copyNumber,1,process);
  analysisManager->FillNtupleDColumn(copyNumber,2,posXYZ.getX()/mm);
  analysisManager->FillNtupleDColumn(copyNumber,3,posXYZ.getY()/mm);
  analysisManager->FillNtupleDColumn(copyNumber,4,posXYZ.getZ()/mm);

I also generate and fill some 3d histograms in the same way and they don’t complain.

Note: I have some questions about g4cmp and how to add data for new materials but it will probably confuse this thread.