Storage types

The DIFA-OPH cluster offers different storage areas with very different characteristics and limitations.

This is a storage area available from every node. Should be used for source files, compiling code, jobs that do not need a lot of space.

Temporary replacement for /fast (see below), useable to store work that doesn't fit $HOME.

This share is web-accessible (read-only) at https://apps.difa.unibo.it/files/people/Str957-cluster . Web access is read-only and it is not possible to create dynamic pages.

Used as work area for jobs that do not need very big datasets or need to have lots of files in a single directory (not recommended, might degrade performances for all users!).

IMPORTANT: DO NOT USE FOR LOTS OF SMALL FILES!!!: /scratch is a clustered filesystem that on one hand allows to store big files and to scale out capacity by adding more servers, but on the other hand introduces quite a lot of overhead (it must contact 3 servers to determine where to get a file from, for example) that won't get amortized with small files. Way worse are directory lookups, that nearly crawl to a halt when a directory contains more that about a thousand files. If you need some big files (say a dataset) and your codes do not allow for specifying a different path, just use symlinks from $HOME to the files in /scratch.

Quota: per-sector (see df -kh /scratch), with extra space allocated for specific projects or bought by individuals.

PLANNED ⇒ not useable (yet)
Parallel filesystem, for quick access to the data you're working on.
No quota, but files older than TBD¹⁾ days will be automatically deleted without further notice.
No cross-server redundancy, just local RAID: if (when) a server (or two disks in the same RAID) fails, all data becomes unavailable – always keep a copy of your important data archived elsewhere (maybe /scratch, but for very important data offsite is better)

¹⁾

Decision to be finalized by the board