Installing Extrae
You can download and install binary versions, but I prefer to build my MPI-capable programs from source, since you often need to use the same MPI to build something as you do to run it. So some prerequisites:
- PAPI
- dyninst
- libdwarf
- binutils (with development headers)
- an MPI
- CUDA (if you want to do CUDA Profiling)
I am probably missing some, but the installation process took some massaging to get it working on my machine.
Without recompiling
You can use LD_PRELOAD to insert Extrae's profiled symbols in place of standard symbols so you can collect statistics (like MPI communication overhead) from a program without having to recompile.
To do this, you need to set the following environment variables
export EXTRAE_CONFIG_FLAGS=/usr/local/extrae/3.0.1/share/example/MPI/detailed_trace_basic.xml
export EXTRAE_HOME=/usr/local/extrae/3.0.1
export LD_PRELOAD=/usr/local/extrae/3.0.1/lib/libmpitracef.so
Note, the paths depend on where you installed Extrae (and libmpitracef.so is used instead of libmpitrace.so for me because I'm profiling a Fortran-MPI program). They have examples for different kinds of profiling in $EXTRAE_HOME/share/example, with tracing.sh scripts that show you what environment variables to set up
Next, just run your program like normal, and you will get a Paraver trace file out of it!