Singularity

Singularity is a container system for HPC that lets you define your own environment and makes your work portable and reproducible on any HPC that supports it. Singularity has extensive documentation and examples for helping you installing, setting up containers http://singularity.lbl.gov/

Note: creating and modifying containers must be done outside the cluster on your own linux machine where you have root access.

There are some images, and several examples recipes accessible from the clusters under /apps/Singularity/

GPU

To access the GPU, you can use the --nv option when running your container, e.g:

singularity exec --nv my_image.img  my_gpu_app

When running graphical applications that need 3D acceleration on the GUI machines, you need to combine this with VirtualGL:

singularity exec --nv my_image.img  vglrun my_gui_app

Using containers in jobs

Using the image in a job is straight forward, and requires no special steps:

#!/bin/bash
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 0:30:00
#SBATCH -A **your-project** -p hebbe

echo "Outside of singularity, host python version:"
python --version
singularity exec ~/ubuntu.img echo "This is from inside a singularity. Check python version:"
singularity exec ~/ubuntu.img python --version

Using modules inside your container

If you need to import additional paths into your container using the SINGULARITYENV_ prefix. This is in particular useful with the PATH and LD_LIBRARY_PATH which are for technical reasons cleared inside the container environment.

module load MATLAB
export SINGULARITYENV_PATH=$PATH
export SINGULARITYENV_LD_LIBRARY_PATH=$LD_LIBRARY_PATH
singularity exec ~/ubuntu.simg matlab -nodesktop -r "disp('hello world');"

However, note that is it very easy to break other software inside your container by importing the host's PATH and LD_LIBRARY_PATH into your container. In addition, any system library that the software depends on needs to be installed in your container. E.g. you can not start MATLAB if there is no X11 installed, which is typically not done when setting up a small, lean, Singularity image. Thus, if possible, strive to call modules from outside your container unless you a special need, e.g:

singularity exec ~/ubuntu.simg run_my_program simulation.inp
module load MATLAB
matlab < post_process_results.m