[Experimental] Application profiling using MAQAO

In a step toward better result interpretation, HPCW can profile applications using MAQAO. This tool works on both x86_64 and aarch64 systems, enabling comparison across architectures. The resulting profiling takes the form of HTML, which can be easily copied and read over almost any machine. Integrating MAQAO into HPCW can be done in the job launcher, where we can interleave MAQAO between the batch scheduler commands and the application.

What is MAQAO ?

‍MAQAO (Modular Assembly Quality Analyzer and Optimizer) is a performance analysis and optimization framework operating at binary level with a focus on core performance. Its main goal of is to guide application developpers along the optimization process through synthetic reports and hints.

MAQAO mixes both dynamic and static analyses based on its ability to reconstruct high level structures such as functions and loops from an application binary. Since MAQAO operates at binary level, it is agnostic with regard to the language used in the source code and does not require recompiling the application to perform analyses. MAQAO has also been designed to concurrently support multiple architectures.

(source : https://maqao.org/)

Refer to the official documentation for details on how to use this tool. The following section focuses on how we integrated the tool in HPCW.

Integrate MAQAO in a HPCW Job Launcher

Eviden has integrated MAQAO in its job-launcher.sbatch. The key points are the following, depicted in the code snippet below :

• MAQAO expects a full path to the binary ;
• Decompose the call into the MPI Command and application launch ;
• MAQAO will override the results directory --replace ;
• --exit-after-profiling tells MAQAO to not generate HTML reports. See below for explanations.

Refer to the official doc for other arguments.

launcher=direct-timed-maqao
 
# MAQAO expects full path to the binary
binary=$(type -f -p $1)
shift
 
MPI_CMD="srun -n $NPROCESS"
 
[ -n ${OMP_NUM_THREADS:-} ] && MAQAO_EXTRA_ARGS+=" --envv_OMP_NUM_THREADS=$OMP_NUM_THREADS"
 
HPCW_MAQAO="maqao oneview                   \
    --create-report=one                     \
    --exit-after-profiling                  \
    --replace                               \
    ${MAQAO_EXTRA_ARGS}                     \
    --lprof-params=--sampling-rate=high     \
    -xp=${log_dir}/maqao-logs-${name}       \
    --show-program-output=on                \
    -WT                                     \
    -mpi-command=\"$MPI_CMD\"               \
    -- $binary $@"

Also note that the launcher (direct-timed-maqao) is specified in order for the $HPCW_MAQAO command to be called.

The current implementation relies on the variable ENABLE_MAQAO, that needs to be exported before calling HPCW (either via CMake or using the build wrapper). If MAQAO is not provided by the system (along with -DUSE_SYSTEM_maqao=ON), then the MAQAO project has to be enabled manually using -DENABLE_maqao=${ENABLE_MAQAO:-OFF} to have MAQAO installed my HPCW.

Results are stored in the log folder, under maqao-logs-${name} with ${name} taking the value of the test case. Because MAQAO result analysis can be long, we chose to only run the data sampling along with the jobs (using MAQAO flag --exit-after-profiling). The MAQAO reports need to be generated a posteriori by running command:

maqao oneview --create-report=one -xp=${log_dir}/maqao-logs-${name}