Python

We install both version 2 and 3. The normal Python module also includes many common libraries, such as numpy, scipy, matplotlib, and ipython.

Installing local Python modules

With the pip command, you can install additional python libraries yourself. You must first choose a existing python module and load it.

Please direct larger installations to $SNIC_NOBACKUP:

pip install some_library --prefix $SNIC_NOBACKUP/my_python

You will need add the site-packages path to $PYTHONPATH, e.g:

export PYTHONPATH=$PYTHONPATH:$SNIC_NOBACKUP/my_python/lib/python3.6/site-packages/

Similarly, you may have to add

export PATH=$PATH:$SNIC_NOBACKUP/my_python/bin

to your $PATH and for some libraries, also extend $LD_LIBRARY_PATH to where various shared libraries may end up.

Virtual environments

The virtualenv command is included in the Python module. The first time, we create a new virtual environment (only done once)

virtualenv my_python

to use this environment, we must activate it (every time you log in)

source my_python/bin/activate

and then we can install modules locally

pip install some_module

For more information see 1

Matplotlib

Please note that when running Matplotlib, you might want to run

matplotlib.use('Agg')

after importing, to avoid Matplotlib trying to use the X Windows backend, which will fail if you didn't log in with X forwarding (which won't work in batch jobs).

Conda environments

NOTE The best way to create conda environments is with Singularity. There is an example recipe at

/apps/Singularity/conda-example.recipe

which can be used as e.g.

singularity exec /apps/Singularity/conda-example.simg python my_script.py

This drastically reduces the installation size, and the number of files (down to 1), making it drastically better for the centre storage. Many conda packages aren't build compatible with CentOS6 (or even 7). In these cases, you must use singularity, or build all of them manually. Please see our Singularity container for more details.

If you really can't use containers

If you really can't create a container, you can create conda environments (these become large! you must use $SNIC_NOBACKUP), e.g.

module load Anaconda3
cd $SNIC_NOBACKUP
conda create -p $SNIC_NOBACKUP/my_conda python=3

Activate the environment when you wish to use it:

source activate $SNIC_NOBACKUP/my_conda

(you might want to create an alias, or do this in your bashrc, as you will need to do this every time you run the program).

Install what you need, you can also add other channels, e.g:

conda config --add channels acellera
conda install htmd

conda will cache files in ~/.conda/pkgs/ which are necessary. These extra copies fill up your disk quota, and you can clear them out with the command:

conda clean --all