PyTorch

PyTorch is a popular machine learning (ML) framework, see https://pytorch.org/.

A common use case is to import PyTorch as a module in Python. It is then up to you as a user to write your particular ML application as a Python script using the torch Python module functionality.

At C3SE we provide precompiled optimized installations of both legacy and recent versions of PyTorch in our tree of software modules, see our introduction to software modules. It is also possible to run PyTorch using containers. For generic guid on using containers, see https://github.com/c3se/containers/blob/master/README.md and https://www.c3se.chalmers.se/documentation/applications/containers/.

In the software module tree we provide PyTorch versions both with CUDA GPU acceleration and versions using only the CPU. Which one you want to use depends on which part of our clusters you will run your jobs on. However, PyTorch is heavily optimized for GPU hardware so we recommend using the CUDA version and to run it on the compute nodes equipped with GPUs. How to do this is described in our guide to running jobs.

To list the available versions you can use the module spider pytorch command:

[hebbe@vera ~]$ module spider pytorch
...
     Versions:
        PyTorch/1.1.0-Python-3.7.2
        PyTorch/1.2.0-Python-3.7.2
        PyTorch/1.3.1-Python-3.7.4
        PyTorch/1.4.0-Python-3.7.4

To use the version PyTorch/1.4.0-Python-3.7.4 (i.e. PyTorch v1.4.0 with Python 3.7 bindings) we inspect that particular module with the module spider command:

[hebbe@vera ~]$ module spider PyTorch/1.4.0-Python-3.7.4

-------------------------------------------------------------------------------------------------
  PyTorch: PyTorch/1.4.0-Python-3.7.4
-------------------------------------------------------------------------------------------------
    Description:
      Tensors and Dynamic neural networks in Python with strong GPU acceleration. PyTorch is a
      deep learning framework that puts Python first.


    You will need to load all module(s) on any one of the lines below before the "PyTorch/1.4.0-Pytho
n-3.7.4" module is available to load.

      GCC/8.3.0  CUDA/10.1.243  OpenMPI/3.1.4
      GCC/8.3.0  OpenMPI/3.1.4
...

Here we see that the PyTorch module depends on a number of other software modules. All these modules have to be loaded before loading the PyTorch/1.4.0-Python-3.7.4 module.

If you want to run on CUDA accelerated GPU hardware, make sure to select the set of modules including the CUDA/10.1.243 package.

[hebbe@vera ~]$ module load GCC/8.3.0 CUDA/10.1.243 OpenMPI/3.1.4 PyTorch/1.4.0-Python-3.7.4

After loading the PyTorch/1.4.0-Python-3.7.4 module your environment is now configured to start calling PyTorch from Python. Here is a small test that prints the PyTorch version available in your environment:

[hebbe@vera ~]$ python -c "import torch; print(torch.__version__)"
1.4.0

If you intend to run your calculations on GPU hardware it can be useful to check that PyTorch detects the GPU hardware using the torch.cuda submodule. Here is an example from a node equipped with a Nvidia Quadro GPU.

[hebbe@vera ~]$ python -c "import torch; print('CUDA enabled:', torch.cuda.is_available())"
CUDA enabled: True
[hebbe@vera ~]$ python -c "import torch.cuda as tc; id = tc.current_device(); print('Device:', tc.get_device_name(id))"
Device: Quadro P2000