Julia @ IN2P3-FR Computing center (CCINP3)

Hello,

We are working on setting up a Julia environment at the French IN2P3 computing center.

Note: These resources are available to HEP physicists from French institutes with an account at the center.

What is available now:

  • Julia v1.7.0. To enable it, use the command module add julia. (A more recent version can be installed on-demand)

Note 1: to prevent the error when adding a package, set the environment variable JULIA_DEPOT_PATH to $HOME/.julia:/pbs/software/centos-7-x86_64/julia/1.7.0/share .

export JULIA_DEPOT_PATH=$HOME/.julia:/pbs/software/centos-7-x86_64/julia/1.7.0/share

or

setenv JULIA_DEPOT_PATH $HOME/.julia:/pbs/software/centos-7-x86_64/julia/1.7.0/share

depending on your shell.

Note 2: Julia is also available from cvmfs by sourcing /cvmfs/sft.cern.ch/lcg/views/LCG_101swan/x86_64-centos7-gcc10-opt/setup.sh:

source /cvmfs/sft.cern.ch/lcg/views/LCG_101swan/x86_64-centos7-gcc10-opt/setup.sh

or

source /cvmfs/sft.cern.ch/lcg/views/LCG_101swan/x86_64-centos7-gcc10-opt/setup.csh

What we are working on:

  • Adding Julia in Jupypter
  • Installing wrapit and use it to interface Julia with ROOT

This topic will be used for technical discussion on this effort and any other topic of interest for the Julia IN2P3/Irfu community. The goal of the CCIN2P3 team is to provide us with the best possible environment for our needs.

Patrice Lebrun (IN2P3, COMET), Wilfrid Da Silva (IN2P3, COMET), Sébastien Gadrat (IN2P3, CMS/CC-INP3) and Philippe Gras (Irfu, CMS and CUPID)

5 Likes

Hi,

About testing WrapIt, I have installed Clang (together with LLVM v15.0.5, Clang being a sub-project of LLVM), compiled with gcc 9.3.1 (so fully compatible with C++17).

Environment setup can be done using module, as follow:
$ module load Libraries/llvm/15.0.5

I did not try to compile code yet, so it may be necessary to add some PATH to properly find all the required includes. I will do some tests and update the modulefile (to set up the environment) as required.

Cheers.

Excellent. Thanks Sébastien. I confirm I can compile wrapit. A copy is in /pbs/home/p/pgras/public/bin/

Recipe to compile wrapit at CCINP3:

  • Run module add Libraries/llvm/15.0.5 and module add cmake
  • Include the -DClang_DIR=/pbs/software/centos-7-x86_64/llvm/15.0.5/lib/cmake/clang option in the cmake command line.

We can eventually discuss if we want the wrapit executable installed centrally.

I have some troubles with the ROOT example of wrapit usage that does not behave as in the other places I’ve tested it, I will investigate.

Philippe.

Hi Philippe,

Glad it worked!

As you are several people interested in that software, I see no reason to not have it installed and available centrally. I can have a look and install it in the software area, and make a modulefile so it can be used using modules.

About ROOT, which version of it are you using? Is it a ROOT version installed here? I think only the latest ROOT version installed here, i.e. 6.26.10 is C++17 compliant (but the Clang version is not that new, i.e. 9.0.1 if I remember correctly).

Off topic, but great to hear about julia users in the Lyon area!

1 Like

Thank you very much Sebastien an Philippe,

I just took 3 seconds to follow the procedure but I got the following errors.
I suppose, I did not have the correct compiler.

cmake -DClang_DIR=/pbs/software/centos-7-x86_64/llvm/15.0.5/lib/cmake/clang …
– The C compiler identification is GNU 4.8.5
– The CXX compiler identification is GNU 4.8.5
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: /usr/bin/gcc - skipped
– Detecting C compile features
– Detecting C compile features - done
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/c++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Performing Test Terminfo_LINKABLE
– Performing Test Terminfo_LINKABLE - Success
– Found Terminfo: /usr/lib64/libtinfo.so
– Found ZLIB: /usr/lib64/libz.so (found version “1.2.7”)
– Found LibXml2: /usr/lib64/libxml2.so (found version “2.9.1”)
– Configuring done
CMake Error in CMakeLists.txt:
Target “wrapit” requires the language dialect “CXX17” (with compiler
extensions). But the current compiler “GNU” does not support this, or
CMake does not know the flags to enable it.

– Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.

Patrice.

Hello Patrice,

The command module add Libraries/llvm/15.0.5 should switch the compiler to 9.3.1. It is possible that cmake kept in cache the compiler it found in a previous run. Try to delete your build directory and start from a new one.

Philippe.

Hello Sebastien,

The problem I mentioned before turns out to be linked to the LLVM version. Would it be possible to install CLANG+LLVM version 11.0.1, for which the code has been validated? That’s great we can install wrapit. Let’s do it, once I’ve validated the problem is solved.

Thanks,

Philippe.

Hi Philippe,

For my previous test I have deleted the build directory. To be sure after deleting build again I redo this test and I got the same error (no surprise).

Here is an output of printenv. I suppose some directories are not in the right place in PATH

printenv | grep -i CC
__MODULES_LMPREREQ=Libraries/llvm/15.0.5&Compilers/gcc/9.3.1
HOSTNAME=cca003
XROOTD_VMP=ccxroot:1999:/hpss/=/hpss/
__MODULES_LMTAG=Compilers/gcc/9.3.1&auto-loaded
MODULECONTACT=https://support.cc.in2p3.fr/
PATH=/pbs/software/centos-7-x86_64/cmake/3.25.0/bin:/pbs/software/centos-7-x86_64/llvm/15.0.5/bin:/opt/rh/devtoolset-9/root/usr/bin:/pbs/software/centos-7-x86_64/julia/1.7.0/bin:/usr/local/bin/:/usr/bin:/usr/local/root/pro/bin:/opt/bin:/opt/software/treqs2-client//prod/bin:/opt/software/rfio-hpss/prod/bin:/pbs/software/centos-7-x86_64/xrootd/4.8.1/bin:/pbs/software/centos-7-x86_64/oracle/12.2.0/instantclient/bin:/pbs/software/centos-7-x86_64/logon/decommission:/pbs/software/centos-7-x86_64/fs4/prod/bin:/usr/lib64/qt-3.3/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/pbs/software/centos-7-x86_64/suptools/prod/bin:/opt/ccin2p3/bin:/opt/dell/srvadmin/bin:/pbs/home/l/lebrun/bin://bin/:/pbs/throng/comet/bin:.
LMFILES=/pbs/software/centos-7-x86_64/modules/modulefiles/Compilers/gcc/9.3.1:/pbs/software/centos-7-x86_64/modules/modulefiles/Libraries/llvm/15.0.5:/pbs/software/centos-7-x86_64/modules/modulefiles/Production/cmake/3.25.0
LOADEDMODULES=Compilers/gcc/9.3.1:Libraries/llvm/15.0.5:Production/cmake/3.25.0
ENVIRONMENT=ACCESS
__MODULES_LMALTNAME=Compilers/gcc/9.3.1&Compilers/gcc/rh-devtoolset-9&al|rh-devtoolset:Libraries/llvm/15.0.5&as|Libraries/llvm/default&as|Libraries/llvm/latest:Production/cmake/3.25.0&Production/cmake/latest&al|cmake&as|Production/cmake/default
CC=/usr/bin/gcc
BASH_FUNC_ccenv()=() { eval (/opt/ccenv/bin/ccenv @)

Patrice.

Then:

  • Check that which gcc shows /opt/rh/devtoolset-9/root/usr/bin/gcc
  • Add -DCMAKE_CXX_COMPILER=`which g++` -DCMAKE_C_COMPILER=`which gcc` on you cmake command line. (`: backquote)

You can also run ccmake instead of cmake and edit the compiler path in the user interface.

Philippe.

It’s fine with these two options:

cmake -DClang_DIR=/pbs/software/centos-7-x86_64/llvm/15.0.5/lib/cmake/clang -DCMAKE_CXX_COMPILER=which g++ -DCMAKE_C_COMPILER=which gcc
– Configuring done
You have changed variables that require your cache to be deleted.
Configure will be re-run and you may have to reset some variables.
The following variables have changed:
CMAKE_C_COMPILER= /opt/rh/devtoolset-9/root/usr/bin/gcc
CMAKE_CXX_COMPILER= /opt/rh/devtoolset-9/root/usr/bin/g++
CMAKE_CXX_COMPILER= /opt/rh/devtoolset-9/root/usr/bin/g++

– The C compiler identification is GNU 9.3.1
– The CXX compiler identification is GNU 9.3.1
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: /opt/rh/devtoolset-9/root/usr/bin/gcc - skipped
– Detecting C compile features
– Detecting C compile features - done
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /opt/rh/devtoolset-9/root/usr/bin/g++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Performing Test Terminfo_LINKABLE
– Performing Test Terminfo_LINKABLE - Success
– Found Terminfo: /usr/lib64/libtinfo.so
– Found ZLIB: /usr/lib64/libz.so (found version “1.2.7”)
– Found LibXml2: /usr/lib64/libxml2.so (found version “2.9.1”)
– Configuring done
– Generating done
– Build files have been written to: /pbs/home/l/lebrun/comet/wrapit/build

More informations about gcc:
(without -DCMAKE_CXX_COMPILER= which g++` -DCMAKE_C_COMPILER= ``which gcc ` cmake does not work)

/pbs/home/l/lebrun/comet/wrapit/build(0)>gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/lto-wrapper
Target: x86_64-redhat-linux
Configured with: …/configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-9/root/usr --mandir=/opt/rh/devtoolset-9/root/usr/share/man --infodir=/opt/rh/devtoolset-9/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-9.3.1-20200408/obj-x86_64-redhat-linux/isl-install --disable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)

At stage make, I have this error :

make
.
.
.
[ 14%] Linking CXX executable wrapit
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/wrapit.dir/src/libclang-ext.cpp.o: in function hasDefaultConstructor(CXCursor)': libclang-ext.cpp:(.text+0x32): undefined reference to typeinfo for clang::CXXRecordDecl’
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: libclang-ext.cpp:(.text+0x37): undefined reference to typeinfo for clang::Decl' /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/wrapit.dir/src/libclang-ext.cpp.o: in function fully_qualified_name(CXCursor)‘:
libclang-ext.cpp:(.text+0xbb): undefined reference to typeinfo for clang::NamedDecl' /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: libclang-ext.cpp:(.text+0xc0): undefined reference to typeinfo for clang::Decl’
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/wrapit.dir/src/libclang-ext.cpp.o: in function get_template_parameters(CXCursor)': libclang-ext.cpp:(.text+0xef8): undefined reference to typeinfo for clang::TemplateDecl’
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: libclang-ext.cpp:(.text+0xefd): undefined reference to `typeinfo for clang::Decl’
collect2: error: ld returned 1 exit status
make[2]: *** [wrapit] Error 1
make[1]: *** [CMakeFiles/wrapit.dir/all] Error 2
make: *** [all] Error 2

I think the latest ROOT version (i.e. 6.26.10) builtin LLVM version is 9… So not recent enough.

What should maybe be done is to turn off the build of the builtin version, and use an external version. Let me check whether this is possible…

Ok, let le see if I can install WrapIt centrally.

That would probably be the best way to go.