This is a guide aimed at you as a new user of a C3SE cluster, and guides you on how to get set and up and running.
By following this guide you will learn what software you need on your computer to communicate with the cluster, you'll learn the basic idea of cluster computing and how to run your calculations in a cluster environment. We will briefly discuss how to access pre-installed software and how to transfer files back and forth from the cluster file system.
This information is a supplement to the C3SE Introduction Seminar that are regularly given at Chalmers for new users. As a new user you are obliged to attend this seminar before using C3SE resources. You can view the C3SE Introduction Seminar-slides here: https://docs.google.com/presentation/d/10vNk4VGZH7Pk8DN4ZncQlbNnaaWDGLDksz-F74cc1gg/edit?usp=sharing
The cluster environment
When connecting to a computer cluster the actual computer you have connected to is only part of this system - the login node of the cluster. This is the computer in the cluster with which you mostly will work directly with. But it is NOT the computer that is doing the heavy calculations. Also connected to the cluster is a large amount of "compute nodes" that do the actual big computations. So, how do we get to use these compute nodes?
In order for the cluster to be efficiently used it is equipped with a "batch processing system". The basic idea is that all computation tasks are packed and specified in advance, and then submitted as a "job" to the job scheduler of the batch processing system. The job scheduler will then put the job in its "job queue" and when the requested of computer resources are available it will execute the job on a partition of all the compute nodes of the cluster.
The result of this approach is a more efficient use of all the compute nodes of the cluster as jobs are scheduled and executed automatically without any required user intervention. Thus the cluster is working day and night, weekends and holidays as long as there are jobs in the queue.
The clusters use Slurm. There are numerous commands you use to submit and check your jobs.
Step by step
- Before we can proceed, you must first get access to one of C3SE's resources.
- Connectting to the cluster is done via SSH described here.
- Running jobs
- Instructions for the filesystem and data transfer.
- How to access the software
If you encounter problem, please see the the support page