Using software in modules¶
To get your work done on the cluster, the software you require must be available on the cluster. Much common software is already installed, but if you need some special program it has to be installed. As a user you have the possibility to install software in your home directory. If you need support on installation please send a support request.
To organise much of the software installed on the cluster we use the Environment Module System https://en.wikipedia.org/wiki/Environment_Modules_(software). It allows you to list available software and dynamically load software modules in your command line session. The command used is simply called module, see man module for detailed usage information.
We offer a module system with a lot of pre-installed software. If your software isn't available, we offer several compilers for compiling the software yourself. Many languages also have systems in place for user local installations, see Python, Perl below. If it's a common software, we can do a system installation.
You can also use libraries, build tools, and compilers from the module system to build your own software.
We use EasyBuild to build all software.
Modules¶
A module system is used to allow the user to specify which application/software they want to use on the cluster. The modules package allows dynamical modification of a user's environment (PATH etc.), which makes the program available from the shell. Modules can be loaded and unloaded dynamically.
The module
command¶
The module command (or the ml
shortcut) controls how to change the software you have available. For the complete manual, please use
man module
To search for a module names:
module spider softwarename
ml spider softwarename
To search for keywords in a module (such as extensions in bundles):
module keyword foo
ml keyword foo
To view available software possible to load directly:
module avail
ml avail
To see what modules are loaded at the moment use:
module list
ml
To load/unload a module:
module load <modulename>
module unload <modulename>
module purge # Unloads all currently loaded modules`
ml <modulename>
ml unload <modulename>
ml purge
To see what a module does, use
module show <modulename>
ml show <modulename>
This will display what environment variables the module sets or modifies.
Toolchains¶
A toolchain is a set of modules that makes up the necessary components to build most software.
Most commercial (closed source) tools, do not require a toolchain, e.g. Mathematica, MATLAB. These can be loaded directly;
module load MATLAB/2021a
other software is build for a particular toolchain, predominatly foss
(GCC, OpenMPI, OpenBLAS), and intel
.
Note: starting from 2021, the cuda
toolchains are simply part of foss
and intel
.
Mixing dependencies between different toolchain version is likely to break the software (and the module system prevents you from doing this to some extent).
Flat module tree¶
Note: This is the only option on Alvis. On Vera the hierarchical module
tree is still default but it is possible to switch with the bash function
flat_modules
. You can put this in your .bashrc
file to enable it by
default.
You can use module avail
or module spider
to find all the available modules.
The specific compiler or CUDA version used will be part of the version string when you load the modules, e.g:
module load TensorFlow/2.5.0-fosscuda-2020b
module list # shows all the loaded dependencies
Hierarchical module tree¶
Note: This is default on Vera.
Using the hierarchical module tree you first need to load the toolchain you want to use before any software compiled with it is available to load;
module load fosscuda/2020b
module load TensorFlow/2.5.0
Hierarchical module tree: Finding software¶
To search for all available software independently of the particular toolchain you can use the module spider <software_name>
command. If you are a new user on our system, please take the time and search for the particular software you want to use in your research. Chances are that it is installed an ready to go!
For example, if you want to use Python/NumPy/SciPy have a look at,
module spider scipy
You can also view the modules for your currently loaded toolchain (including other toolchains) by running:
module avail
However, when you first log in to our systems this won't show you most software, since no toolchain is loaded by default. Instead, you have to pick a toolchain, in order for module avail
to show more modules. For example
module load intel
module avail
will show all modules compiled under the intel
toolchain.
Please note that some software might only be available in some toolchains, including specific versions of a toolchain, e.g.
module load foss/2016b
module avail
Hierarchical module tree: Finding the right toolchain for a module¶
Searching for a specific version gives you additional information about the module, e.g:
module spider Python/2.7.12
------------------------------------------------------------------------------
Python: Python/2.7.12
------------------------------------------------------------------------------
Description:
Python is a programming language that lets you work more quickly and
integrate your systems more effectively. - Homepage: http://python.org/
Other possible modules matches:
Biopython
This module can only be loaded through the following modules:
GCC/5.4.0-2.26 OpenMPI/1.10.3
icc/2016.3.210-GCC-5.4.0-2.26 impi/5.1.3.181
ifort/2016.3.210-GCC-5.4.0-2.26 impi/5.1.3.181
Help:
Python is a programming language that lets you work more quickly and integrate your systems
more effectively. - Homepage: http://python.org/
------------------------------------------------------------------------------
To find other possible module matches do:
module -r spider '.*Python/2.7.12.*'
Here, it tells us that that we can do either
module load GCC/5.4.0-2.26 OpenMPI/1.10.3
module load Python/2.7.12
or
module load icc/2016.3.210-GCC-5.4.0-2.26 impi/5.1.3.181
module load Python/2.7.12
or, as GCC + OpenMPI are part of the foss/2016b
, and icc/ifort + impi are part of the intel/2016b
toolchain, we could also do
module load foss/2016b
module load Python/2.7.12
or
module load intel/2016b
module load Python/2.7.12