LMOD¶
A module system is software that helps manage environment variables, paths, and software packages. It allows users to switch between different software versions easily, set up specific configurations, and manage dependencies. The environmental modules add, modify, and remove environmental variables in a coherent way. For a an extensive guide check the project documentation
Basic operations¶
In this tutorial, we explore a module system's essential features and legacy usage. We will focus on the MPI module in the ORFEO environment.
The command module avail lists all available modules in the cluster. In the login node with a fresh session, the command output will look something like the following (Press q to exit from available modules list):
$ module avail
---------------------------- /orfeo/opt/modules/tools ----------------------------
R/4.2.3 fastqc/0.12.1 gatk/4.3.0 java/1.8.0
foldseek/5-53465f0 hwloc/2.8.0 java/18.0.2.1 java/19.0.1 (D)
---------------------------- /orfeo/opt/modules/mpi ------------------------------
openMPI/4.1.4/gnu/12.2.1 openMPI/4.1.5/icc/2021.7.1
---------------------------- /orfeo/opt/modules/libraries ------------------------
cuda/12.1 (D) mkl/latest mkl/2022.2.1 (D) openBLAS/0.3.23 (D)
Where software is logically grouped into sections (tools, mpi, libraries), the section tile is the folder where the moudle files are stored.
The command to load a module is module load {{ module name }}. In the particular case of the openMPI module, we have different versions. To load a specific one, like 4.1.4 with GNU Compiler Collection version 12.2.1, we must issue: module load openMPI/4.1.4/gnu/12.2.1. This operation changes our environment (that we can explore with the env command) by modifying all the variables needed for the software contained in the module to work.
# List of modified variables
$ env | grep MPI
MPI_LIB=/opt/area/shared/programs//AMD/openMPI/4.1.4/lib
OMPI_MCA_btl=^openib
__LMOD_REF_COUNT__LMFILES_=/opt/area/shared/modules_v2/profiles/architecture/AMD.lua:1;/opt/area/shared/modules_v2/AMD/mpi/openMPI/4.1.4/gnu/12.2.1.lua:1
MANPATH=/opt/area/shared/programs/AMD/openMPI/4.1.4/share/man:/usr/share/lmod/lmod/share/man::
MPI_INCLUDE=/opt/area/shared/programs//AMD/openMPI/4.1.4/include
MPI_HOME=/opt/area/shared/programs//AMD/openMPI/4.1.4
...
PATH=/opt/area/shared/programs/AMD/openMPI/4.1.4/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
...
It is now possible to use commands such as mpicc and mpirun, but why? Because the path to the folder containing the mpicc and mpirun executables is now in the environment variable 'PATH.'
To keep track of the loaded modules module list show all currently loaded modules. If for some reason the user need to unload some software, the command module unload [modulename] will do the trick. If a lot of modules are loaded or a refresh of the environment is necessary, module purge unload all currently loaded modules.
Equivalent and conflicting modules¶
Remaining on our openmpi module. What happens if the openMPI module is loaded and the user tries to load the IntelMPI implementation? In this case, the module system automatically replaces the openMPI module with the Intel module, and the environment is always modified coherently, avoiding software clashes as much as possible.
$ module load openMPI/4.1.4/gnu
$ module load intelMPI/2021.7.1
Loading mpi version 2021.7.1
Lmod is automatically replacing "openMPI/4.1.4/gnu/12.2.1" with "intelMPI/2021.7.1".
The same behavior is observed when we change the compiler wrapped into mpicc.
$ module load openMPI/4.1.4/gnu/12.2.1
$ module load openMPI/4.1.4/icx/2022.2.1
Loading compiler version 2022.2.1
Loading tbb version 2021.7.1
Loading compiler-rt version 2022.2.1
Loading oclfpga version 2022.2.1
Load "debugger" to debug DPC++ applications with the gdb-oneapi debugger.
Load "dpl" for additional DPC++ APIs: https://github.com/oneapi-src/oneDPL
Lmod is automatically replacing "openMPI/4.1.4/gnu/12.2.1" with "openMPI/4.1.4/icx/2022.2.1".
Creating new modules¶
As you lack permissions to install new software using dnf, apt, or equivalent package managers, you may need to compile and install to your local storage. The module system comprises a series of .lua files and folders organized in a manner that enables us to structure the software based on versioning and its intended purpose, loading it only when required.
This structure proves helpful in managing different versions of software such as Java or R. As such, here we will outline the steps for installing an additional Java version alongside those already in the ORFEO module system as an example. Typically, the installation of user-compiled software involves placing files in your home directory's .local folder. In this case, we will increase the complexity of the procedure in exchange for flexibility. To start the creation of your module, create inside the $HOME/.local folder two other directories named modules and programs:
Then download the binaries and place the entire Java folder inside .local/programs. In this case, the software of choice is Java; however, you can do the same with any software you can compile by setting the correct variables at compile time.
Now you can generate a new module file to configure our environment for running the newly installed Java binaries. Writing this file for the Java software is simple and requires only adding the location of Java binaries to the PATH environment variable and the libraries to the LD_LIBRARY_PATH. A draft of the module file is like the one that follows; place it inside the modules folder with a notation like modules/{{ software_name }}/{{ version }}.lua. We suggest you keep track of the partition where the software has been compiled by adding an extra folder.
-- -*- lua -*-
local name = "java"
local version = "11.0"
whatis("Name : " .. name)
whatis("Version : " .. version)
family("java")
--In this section all dependencies will be specified
--depends_on("openBLAS/0.3.23")
--conflict()
home= "{{ my home }}/.local/programs/java-11/"
prepend_path("PATH", home .. "/bin")
prepend_path("LD_LIBRARY_PATH", home .."/lib")
prepend_path("MANPATH", home .."/man")
The resulting file tree will look similar to the following one:
$ tree .local/
.local/
├── bin
├── etc
├── lib
├── modules
│ └── java
│ └── 11.lua
├── programs
│ └── java-11
│ ├── bin
│ ├── conf
│ ├── include
│ ├── jmods
│ ├── lib
│ └── man
└── share
Lastly, inform the module system that your custom modules are in the .local/modules folder by executing the command: module use $HOME/.local/modules/. To ensure the changes persist, add the command to your bashrc: echo "module use $HOME/.local/modules/" >> .bashrc.
At the end, the module avail command should also show you the new modules:
For a more in-depth guide, have a look at the official LMOD guide; have a sneak peek at our modules by exploring the /orfeo/opt folder or have a look at our repository to discover how do we do them (WIP)
Other software
- More complex software may require compilation; the indicated PREFIX during the configuration phase will reside within
$HOME/.local/programs/[software]/[version] - Please ensure to build the software on computational nodes, and for your own sake, keep track of the partition where that binary works