Add Toolchain

To add a new project or run HPCW on a new system you may also want to create a specific toolchain environment, if necessary. This script may contain specific environment variables to export and modules to load and you will need to source it before the compilation:

toolchains/<system>/<hpc_center>-<toolchain_name>.env.sh

To configure CMake's behavior, set compilers to use or additional compilation flags for the toolchain, create the following CMake file:

toolchains/<system>/<hpc_center>-<toolchain_name>.cmake

When compiling, you can use this toolchain file with the following cmake option:

cmake -DCMAKE_TOOLCHAIN_FILE=$HPCW_SOURCE_DIR/toolchains/<system>/<hpc_center>-<toolchain_name>.cmake

Changing Flags

Some toolchains (especially experimental or exotic toolchains) require specific build flags. It is possible to pass down build flags on a per-project basic to each of the CMake projects managed by HPCW.

HPCW will look for variables for each project, following these naming schemes:

• Controlling the configure command:
  • CMake: <project_name>_cmake_args
  • configure: <project_name>_configure_args
• Controlling the build command: <project_name>_build_args

Please note that some projects use non standard configure and/or build commands. For instance, NEMO uses a custom makenemo command with arguments that are not compatible with GNU Make. All the variables are listed in the beginning of every project definition (.cmake file). For example, below is the variable declaration for NEMO:

set(nemo_cmake_args "" CACHE STRING "Custom CMake arguments for NEMO")
set(nemo_build_args "" CACHE STRING "Custom makenemo arguments for NEMO")

In the toolchain, one can then add custom flags with the following syntax:

# content of the toolchain.cmake file
# note the separator ';' as this is a CMake list
set(<project_name>_cmake_args     "-Wno-dev;-DENABLE_XXX=ON" CACHE STRING "")
set(<project_name>_configure_args "--enable-single"          CACHE STRING "")
set(<project_name>_build_args     "-j1;VERBOSE=1"            CACHE STRING "")

:warning: Due to the structure of the CMake files and the version requirement (3.19), the variables in the toolchain file need to be set as cache variables. This CMake policy update, introduced in 3.21, would allow the usage of normal variables in this scenario instead, but presently, we are using the OLD behaviour of this policy throughout the project.

Alternatively, one can pass down custom flags via the .env.sh toolchain file.

# content of the toolchain.env.sh file
# [...]
 
<project_name>_cmake_args="-Wno-dev;-DENABLE_XXX=ON"
<project_name>_configure_args="--enable-single"
<project_name>_build_args="-j1;VERBOSE=1"
 
cmakeFlags+=" -D<project_name>_cmake_args=\"$(printf "%q" "$<project_name>_cmake_args")\""
cmakeFlags+=" -D<project_name>_configure_args=\"$(printf "%q" "$<project_name>_configure_args")\""
cmakeFlags+=" -D<project_name>_build_args=\"$(printf "%q" "$<project_name>_build_args")\""

This syntax is similar to the one used by the build wrapper (see How-Tos section).

Example - ecTrans

To enable a verbose build for the ectrans project, one would add the following in the toolchain .cmake file:

set(ectrans_build_args VERBOSE=1 CACHE STRING "")
set(ectrans_cmake_args
  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
  CACHE STRING ""
  )

Note that this can also be achieved by setting the CMake option HPCW_VERBOSE.

Another example, to go for -O3 and target native would look like this:

# No need to update the build command
set(ectrans_cmake_args
  -DCMAKE_C_FLAGS="-O3 -march=native -mtune=native"
  -DCMAKE_Fortran_FLAGS="-O3 -march=native -mtune=native -heap-arrays 32"
  -DCMAKE_BUILD_TYPE=None
  CACHE STRING ""
  )

:warning: Please note the last CMake flag: This tells CMake to not use any build configuration. Without this, CMake will generate the default build configuration flags after the user flags. In some situation, this can lead to unexpected behavior, compiler flag conflicts, override, etc. In our example above, this would lead to a binary produced with the following flags:

c compiler      : Intel 19.1.3.20200925
  flags         : -O3 -march=native -mtune=native -O2 -g -DNDEBUG
fortran compiler: Intel 19.1.3.20200925
  flags         : -O3 -march=native -mtune=native -heap-arrays 32 -O2 -g -DNDEBUG -heap-arrays 32

Example - FFTW

Enabling FMA and disabling documentation :

set(fftw_configure_args
  --disable-doc
  --enable-fma
  CACHE STRING ""
  )
set(fftw_build_args VERBOSE=0 CACHE STRING "")