Skip to content

Storage resources

Overview

ORFEO shared storage resources (available to all the cluster nodes) are organized under the /orfeo path.

/
|-- orfeo
|   |-- cephfs
|   |   |-- home
|   |   |-- opt
|   |   `-- scratch
|   |-- LTS
|   |-- fast -> /orfeo/cephfs/fast
|   |-- opt -> /orfeo/cephfs/opt
|   `-- scratch -> /orfeo/cephfs/scratch
`-- u -> /orfeo/cephfs/home/

Technology

The bulk of ORFEO shared storage is hosted on a Ceph storage cluster of 23 nodes for a total raw space of 4.8 PiBs.

The fast filesystem /orfeo/cephfs/fast is hosted on four Ceph nodes with SATA SSDs, providing a total raw capacity of 279 TiB.

Both the users' home directories (/orfeo/cephfs/home) and the scratch space (/orfeo/cephfs/scratch) are managed by 19 Ceph nodes equipped with mechanical HDDs.

The long term storage (LTS) instead is hosted on a DELL Powervault SAN appliance exposed via NFSv4 to the cluster nodes. The total raw space for LTS is 2.8 PiBs.

Where to store data

Each user has a dedicated 200 GiBs of storage space for their home directory ($HOME). It is intended to store the user's configuration files and personal data. The users may check their usage by issuing the myquota command.

The scratch area is a shared storage space intended as a temporary working area for ORFEO users. It is visibile to all the cluster nodes and it is partitioned in different directories: one for each group of users/collaboration project. It is hosted on SAS 10k RPMs HDDs.

The fast area is a shared storage space intended as a temporary working area for ORFEO users. It is visibile to all the cluster nodes and it is partitioned in different directories: one for each group of users/collaboration project. It is hosted on SATA SSDs.

The Long Term Storage area is a shared storage space allocated as multiple volumes (one for every group of users/collaboration project) on the Powervault SAN and accessed via NFSv4. It is inteded as an archive of data rather than a working area.

Quota Expandable Backup Retention
User home 200 GiB No No User account lifetime1
Scratch No2 No No Minimum 30 days2
Fast No2 No No Minimum 30 days2
LTS 1 TiB3 Yes
Up to 128 TiB3		 No Variable3

Notes:

  1. User account lifetime means that your data inside your home directory will be kept at minimum as long as you have a valid account to access to ORFEO. Your group leader shall communicate the end of the validity.
  2. Scratch and Fast have no quota enforced but in the event of a fill up ORFEO admins will delete data older than 30 days (mtime > 30). If a user/group is deliberately changing the mtime of their Scratch/Fast content to bypass the retention policy ORFEO admins reserve the right to terminate their account(s) and delete all their data from Scratch/Fast.
  3. LTS is hosted in separate volumes that should be explicitly demanded by the group/project leader sending and email to our contact mailbox. The Quota is the minimum amount that will be allocated to a group/project. Bigger groups/projects should contact us if they require more than 128 TiB of long term storage. Also the retention policy varies volume by volume and may be (re)negotiated once per year.

Local scratch

On each computational node there's a /local_scratch folder available as temporary fast storage for checkpointing and other small I/O operations that need to be fast. ORFEO admins do not guarantee that the contents of local_scratch will survive past the end of your slurm job.

Warning

Do not abuse the /local_scratch by dumping large amounts of data and/or by moving data from your home or your scratch space in order to bypass the quota and/or avoid the retention policy. In this cases ORFEO admins reserve the right to terminate your account and delete all the data in local_scratch.