Singularity is a container system for HPC that lets you define your own environment and makes your work portable and reproducible on any HPC that supports it. Singularity has extensive documentation and examples for helping you installing, setting up containers http://singularity.lbl.gov/
Note: creating and modifying containers must be done outside the cluster on your own Linux machine where you have root access. Please see our build instructions for more information.
Our main repository of containers can be found on the clusters under
/apps/containers/, but many users will also want to build their own customized containers.
Deprecated: Some older images, and examples recipes are accessible under
To access the GPU, you can use the
--nv option when running your container, e.g:
singularity exec --nv my_image.img my_gpu_app
When running graphical applications that need 3D acceleration on the GUI machines, you need to combine this with VirtualGL:
singularity exec --nv my_image.img vglrun my_gui_app
Using containers in jobs
Using the image in a job is straight forward, and requires no special steps:
#!/bin/bash #SBATCH -n 1 #SBATCH -t 0:30:00 #SBATCH -A **your-project** -p hebbe echo "Outside of singularity, host python version:" python --version singularity exec ~/ubuntu.img echo "This is from inside a singularity. Check python version:" singularity exec ~/ubuntu.img python --version
Using modules inside your container
If you need to import additional paths into your container using the
This is in particular useful with the
LD_LIBRARY_PATH which are for technical reasons cleared inside the container environment.
module load MATLAB export SINGULARITYENV_PATH=$PATH export SINGULARITYENV_LD_LIBRARY_PATH=$LD_LIBRARY_PATH singularity exec ~/ubuntu.simg matlab -nodesktop -r "disp('hello world');"
However, note that is it very easy to break other software inside your container by importing the host's
LD_LIBRARY_PATH into your container.
In addition, any system library that the software depends on needs to be installed in your container.
E.g. you can not start MATLAB if there is no X11 installed, which is typically not done when setting up a small, lean, Singularity image.
Thus, if possible, strive to call modules from outside your container unless you a special need, e.g:
singularity exec ~/ubuntu.simg run_my_program simulation.inp module load MATLAB matlab < post_process_results.m
Using overlays for persistent storage
A singularity container is by default immutable, that is, the container will not allow internal modifications while running. If you try to update or add a file in the container image you will get a "Read-only file system" error or similar.
singularity shell myimage.sif Singularity> echo date >> /myapp.sh bash: /myapp.sh: Read-only file system
Prototyping and developing often requies one to install an additional package,
or make changes to files inside a container. In these cases the
--overlay option can be used. Once we know the changes we can later make them persistent
by rebuilding the SIF-image, or bind-mount the affected
files, or, if we see fit, continue to use the overlay.
An overlay is a directory or file system that "over lays" the immutable
filesystem inside your container. The overlay will receive and persist your
changes, thus avoiding changing the read-only parts of the container. Creating and
preparing the overlays is a bit tricky but using them is simple. At Alvis we
have prepared ready-to-go overlays of different sizes at
Example: Using the 1G overlay on myimage.sif.
cp /apps/containers/overlay_1G.img ~/dev_1G_overlay.img singularity shell --overlay ~/dev_1G_overlay.img myimage.sif Singularity> echo date >> /myapp.sh Singularity>
It is now possible to write to
You can also use an overlay to add additional software on top of an existing container. Example: Installing tensorflow into centrally installed anaconda container
cp /apps/containers/overlay_1G.img ~/tf_anaconda_2021.05.img singularity shell --overlay ~/tf_anaconda_2021.05.img /apps/containers/Conda/anaconda-2021.05.sif Singularity> conda install tensorflow
Note the such an overlay will likely only work in conjunction with the exact original singularity container and nothing else.
You can not use --overlay from multiple hosts at the same time
If you attempt to use
--overlay from multiple hosts at the same time you will
get a FATAL error as the overlay option does not support concurrent processes.
Only a single writer is supported at one time. If you want to run your
container with an overlay on several hosts (or in multiple separate processes
within a host) you need to make the overlay read-only by appending
ro to the
singularity exec --overlay ~/dev_1G_overlay.img:ro myimage.sif