Error with python.hpp for MacOS (clang/anaconda3)

Hi,
I am new to Geant4 and wish to install its python G4py version. I tried to follow the instructions that I found online for installing both on MacOS High Sierra (clang and anaconda3).
However, I came across an issue when I <make> G4py, which starts with:

[  0%] Building CXX object source/intercoms/CMakeFiles/pyG4intercoms.dir/pyG4UIcommand.cc.o
/Users/diana/Documents/opt/geant4/environments/g4py/source/intercoms/pyG4UIcommand.cc/Users/diana/Documents/opt/geant4/environments/g4py/source/interface/pyG4UIterminal.cc::3131::1010::  fatal errorfatal error: : 'boost/python.hpp' file not found
'boost/python.hpp' file not found
#include <boost/python.hpp>
         ^~~~~~~~~~~~~~~~~~
#include <boost/python.hpp>
         ^~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [source/intercoms/CMakeFiles/pyG4intercoms.dir/pyG4UIcommand.cc.o] Error 1

Attached is a text file with the commands I used to install all the dependencies for Geant4 and G4py Install_log.txt (1.4 KB)

Have you ever encountered this issue before? Do you have any recommendation on how to fix it?
Thank you for your time and sorry in advance in case I missed this issue being reported elsewhere,
Diana

Don’t have much experience with this part of geant4 but some things to investigate:

  • In the cmake command you use “Boost” for the include directory and “BOOST” for all the others is this correct?
  • If you go to the location of $BOOST_DIR/include can you see the python.hpp file?
  • Running make VERBOSE=1 will show you the full compiler command and you can see what values are actually being used for the include location.

Thank you for your reply.

  • Yes, in the cmake command used to compile g4py I used “Boost” for the include directory and “BOOST” for the root and lib directories:
cmake -DBOOST_ROOT=$BOOST_DIR -DBoost_INCLUDE_DIR=$BOOST_DIR/include -DBOOST_LIBRARY=$BOOST_DIR/lib -DXERCESC_INCLUDE_DIR=$XERCESC_DIR/include -DXERCESC_LIBRARY=$XERCESC_DIR/lib ..
  • There are python.hpp files in $BOOST_DIR/include/boost/mpi/python.hpp and $BOOST_DIR/include/boost/parameter/python.hpp. Should I copy one of them directly to $BOOST_DIR/include?

  • Thank you for the suggestion of using verbose, it really helps to see where make is looking for the file (which I think is $BOOST_DIR/include only and not its sub-directories mpi or parameter).

Do you think that the absence of a python.hpp file in the $BOOST_DIR/include directory mean that the Geant4 installation was not successful?

The file /anaconda3/share/cmake-3.15/Modules/FindBoost.cmake is looking for the python.hpp file in $Boost_INCLUDE_DIR/boost.
I tried to change it to point to the mpi directory location by changing line 1619 to:
set(_Boost_PYTHON_HEADERS "boost/mpi/python.hpp")
But I still get the same error.

It looks like your conda install of boost may be missing a file, or you need an additional package that provides the necessary file. I used my systems package mananger to do a full install of boost and see the following instances of a python.hpp file

$ locate "/python.hpp"
/usr/include/boost/mpi/python.hpp
/usr/include/boost/parameter/python.hpp
/usr/include/boost/python.hpp
$ ll /usr/include/boost/mpi/python.hpp /usr/include/boost/parameter/python.hpp /usr/include/boost/python.hpp
-rw-r--r-- 1 root root  2803 Feb  4  2019 /usr/include/boost/mpi/python.hpp
-rw-r--r-- 1 root root 19092 Feb  4  2019 /usr/include/boost/parameter/python.hpp
-rw-r--r-- 1 root root  2950 Feb  4  2019 /usr/include/boost/python.hpp

They are all different sizes so I don’t think copying one of the ones you currently have into $BOOST_DIR/include will work.

This will not solve what is currently going wrong, but you could try installing boost globally on you system then rerunning the cmake command without any of the boost options.

What is the output of the cmake command? Does it recognise that you have boost installed? I get the following:

$ cmake -DCMAKE_INSTALL_PREFIX=$GEANT4_INSTALL/g4py  ../../geant4/environments/g4py/
-- The C compiler identification is GNU 8.3.0
-- The CXX compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Geant4: /usr/local/share/geant4/10.5.1/install/debug/bin/.. (10.5.1)
-- Found PythonInterp: /usr/bin/python (found version "2.7.16") 
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found version "2.7.16") 
-- Boost version: 1.67.0
-- Found XercesC: /usr/lib/x86_64-linux-gnu/libxerces-c.so  
-- NOT Found ROOT.
-- --------------------------------------------------------
-- Parameters for building
-- CMAKE_BUILD_TYPE: RelWithDebInfo
-- CMAKE_CXX_COMPILER: /usr/bin/c++
-- CMAKE_INSTALL_PREFIX: /usr/local/share/geant4/10.5.1/install/debug/g4py
-- --------------------------------------------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/share/geant4/10.5.1/g4py_build

and this compiles.

Hi! Thanks for the feedback. In that case my problem is that Geant4 itself is compiling without creating that python.hpp file.

It finds boost, just not the necessary file. The output of my cmake command is:

cmake -DCMAKE_INSTALL_PREFIX=$GEANT4_INSTALL -DBOOST_ROOT=$BOOST_DIR -DBoost_INCLUDE_DIR=$BOOST_DIR/include -DBOOST_LIBRARY=$BOOST_DIR/lib -DXERCESC_INCLUDE_DIR=$XERCESC_DIR/include -DXERCESC_LIBRARY=$XERCESC_DIR/lib ..
-- The C compiler identification is AppleClang 10.0.0.10001044
-- The CXX compiler identification is AppleClang 10.0.0.10001044
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Geant4: /usr/local/bin/.. (10.6.0)
-- Found PythonInterp: //anaconda3/bin/python (found version "3.7.5") 
-- Found PythonLibs: //anaconda3/lib/libpython3.7m.dylib (found version "3.7.5") 
-- Found Boost: /anaconda3/pkgs/libboost-1.67.0-hebc422b_4/include (found version "1.67.0")  
-- Found XercesC: /anaconda3/pkgs/xerces-c-3.2.2-h4db8090_1003/lib  
-- Found ROOT 6.19/01 in /usr/local
-- --------------------------------------------------------
-- Parameters for building
-- CMAKE_BUILD_TYPE: RelWithDebInfo
-- CMAKE_CXX_COMPILER: /Library/Developer/CommandLineTools/usr/bin/c++
-- CMAKE_INSTALL_PREFIX: /Users/diana/Documents/opt/geant4/geant4-install
-- --------------------------------------------------------
-- Configuring done
CMake Warning (dev):
  Policy CMP0068 is not set: RPATH settings on macOS do not affect
  install_name.  Run "cmake --help-policy CMP0068" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.

  For compatibility with older versions of CMake, the install_name fields for
  the following targets are still affected by RPATH settings:

   EMSTDpl
   ExN01geom
   ExN01pl
   ExN03geom
   MedicalBeam
   NISTmaterials
   ParticleGun
   Qgeom
   Qmaterials
   ezgeom
   pyG4digits_hits
   pyG4event
   pyG4gdml
   pyG4geometry
   pyG4global
   pyG4graphics_reps
   pyG4intercoms
   pyG4interface
   pyG4materials
   pyG4particles
   pyG4physicslists
   pyG4processes
   pyG4run
   pyG4track
   pyG4tracking
   pyG4visualization

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    BOOST_LIBRARY


-- Build files have been written to: /Users/diana/Documents/opt/geant4/environments/g4py/build

Could you share how you installed Geant4, please?
I might have done something wrong there. My commands were:

# Geant4
git clone https://gitlab.cern.ch/geant4/geant4.git
cd geant4
GEANT4_DIR=/Users/diana/Documents/opt/geant4
GEANT4_INSTALL=$GEANT4_DIR/geant4-install
mkdir $GEANT4_INSTALL
GEANT4_BUILD=$GEANT4_DIR/geant4-build
mkdir $GEANT4_BUILD
cd $GEANT4_BUILD
cmake -DCMAKE_INSTALL_PREFIX=$GEANT4_INSTALL -DGEANT4_USE_OPENGL_X11=ON $GEANT4_BUILD
# We can not activate -DGEANT4_BUILD_MULTITHREADED=ON for the python interface
make -j 4
make install

But maybe I needed to add some flag like USE_PYTHON=ON.

Thanks

I install almost the same as you, but create parallel installs for each new version so have spearate directories for data, source code and builds/versions

e.g. when I installed 10.5.1

[/usr/local/share/geant4/] $ ls
10.1.2  10.2.0  10.2.1  10.2.2  10.3.0  10.3.1  10.3.2  10.4.0  10.4.1  10.4.2  10.5.0  data  geant4
# Checkout required geant4 version
$ cd geant
$ git checkout v10.5.1
# Create directory to build and install in
$ cd ../
$ mkdir 10.5.1 && cd 10.5.1
$ mkdir -p build install/debug
$ cd build
$ cmake -DCMAKE_INSTALL_PREFIX=/usr/local/share/geant4/10.5.1/install/debug -DGEANT4_INSTALL_DATADIR=/usr/local/share/geant4/data -DGEANT4_BUILD_MULTITHREADED=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo -DGEANT4_USE_QT=ON -DGEANT4_USE_GDML=ON -DGEANT4_USE_OPENGL_X11=ON -DGEANT4_USE_RAYTRACER_X11=ON /usr/local/share/geant4/geant4
$ make -j4
$ make install

I then add the necessary lines to ~/.bashrc so environments are sourced and can build g4py with

$ cd /usr/local/share/geant4/10.5.1
$ mkdir g4py_build
$ cd g4py_build
$ GEANT4_INSTALL=/usr/local/share/geant4/10.5.1/install/debug
$ cmake -DCMAKE_INSTALL_PREFIX=$GEANT4_INSTALL/g4py  ../../geant4/environments/g4py/
$ make -j4
$ make install

Geant does not create the python.hpp file that you are missing, that comes as part of the boost install. Are there any other boost packages you could install via conda?

Thanks for the reply!
That is really interesting that for you setting a prefix works.
I found out that making the cmake commands without prefix is necessary in my computer to have the correct link to the boost python.hpp file.

My build was now very straightforward (although not nice for my computer):

git clone https://gitlab.cern.ch/geant4/geant4.git
cd geant4
mkdir build
cd build
cmake  -DGEANT4_USE_OPENGL_X11=ON $GEANT4_BUILD
make -j 4
make install

cd ../environments/g4py/
mkdir build
cd build
cmake ..
make -j 4
(this is where a new error occurred)

Now I got a very different error message, with Hep3Vector call operator missing and I reported it here:
https://bugzilla-geant4.kek.jp/show_bug.cgi?id=2201
The error message starts with:

In file included from /anaconda3/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:47:
/anaconda3/include/boost/python/detail/invoke.hpp:86:12: error: type 'const boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<const
     CLHEP::Hep3Vector>' does not provide a call operator
   return rc( (tc().*f)(BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT)) );
          ^~
/anaconda3/include/boost/python/detail/caller.hpp:216:40: note: in instantiation of function template specialization

Have you ever seen it before? I will update here any news I get from that other issue post.

Thanks for all your help and ideas!

ps: What were the necessary lines that you added to ~/.bashrc so that the environments were sourced and you could build g4py?

Not seem that error before, although the g4py README file - environments/g4py/README.md - does say that python3 is experimental.

Nothing special in the .bashrc just sourcing the file that sets environments

export G4VERSION=10.5.1
. /usr/local/share/geant4/${G4VERSION}/install/debug/bin/geant4.sh

Thank you again for the super quick reply!
I will try to do the same sourcing and see.
That is a good point, maybe it only works with python2, I will test with it.
Thanks