C3SE Storage

Storage infrastructure and storage resources

Storage infrastructure is storage that is expected to live "forever". Centre(wide) and national storage, explained in more detail below, are two such infrastructures. Storage resources on the other hand is storage that have a limited life-span. Cluster filesystems and node local disk are examples of such storage.

The storage hierarchy

Storage is available for different usage and different availability. Looking at it from the bottom up, we have:

• The node local disk: Available only to running jobs, automatically purged after the job finishes. Usually not shared between different nodes in the same job. Accessible to the job using $TMPDIR (the environment variable TMPDIR is automatically set to contain the correct directory path). The available size is different for different clusters.
• Cluster filesystems: Available for all jobs and also through the cluster front-end. Your files my stay here as long as you have an account on the cluster (maximum the lifetime of the cluster). See ex. Svea Filesystem, Beda Filesystem
• Centre(wide) storage: Available from all resources at the centre. For more details look below
• National storage: Available wherever you have your certificate loaded. Requires an (storage-) allocation through SNAC (a sub-committee of SNIC) for access. Based on working with files as opposed to the storage above which can be accessed on the block-layer (i.e. you can change a file)

Centre storage

The centre storage is available on the resources at the centre. It is divided into two parts, one with backup available at $SNIC_BACKUP and one part without backup available at $SNIC_NOBACKUP

The initial quotas are:

Storage location	Soft space quota	Hard space quota	Soft files quota	Hard files quota
$SNIC_BACKUP	25GiB	50GiB	60 000	120 000
$SNIC_NOBACKUP	200GiB	1024GiB	500 000	2 500 000

[SNIC_BACKUP and SNIC_NOBACKUP are environment variables. The $-notation can be used in batch scripts or in an interactive shell to retrieve the value (directory path) of the variable.]

Mounting of your specific filesystem is done via auotomount, i.e. looking in the directory above might not list your directory, but accessing it will make it visible (and usable).