Strumenti Utente

Strumenti Sito


oph:cluster:storage

Questa è una vecchia versione del documento!


Storage types

OPH cluster offers different storage areas with very different characteristics and limitations.

$HOME

This is a storage area available from every node.

Should be used for source files, compiling code, jobs that do not need a lot of space.

NFS-mount via Ethernet (1Gbps): not very fast but quite responsive.

Quota limit: 50GB (soft) / 100GB (hard)

/home/public

This share is web-accessible (read-only) at https://apps.difa.unibo.it/files/people/Str957-cluster . Web access is readonly and it's not possible to create dynamic pages.

/home/work

Used as work area for jobs that do not need very big datasets or need to have lots of files in a single directory (not recommended, might degrade performances for all users!).

/scratch

Main archive area. To be used for large files, big datasets and archives.

DO NOT USE FOR LOTS OF SMALL FILES: /scratch is a clustered filesystem that on one hand allows to store big files and to scale out capacity by adding more servers, but on the other hand introduces quite a lot of overhead (it must contact 3 servers to determine where to get a file from, for example) that won't get amortized with small files. Way worse are directory lookups, that nearly crawl to a halt when a directory contains more that about a thousand files.

If you need some big files (say a dataset) and your sw does not allow for specifying a different path, just use symlinks from $HOME to the files in /scratch.

Quota: per-sector, with extra space allocated for specific projects or bought by individuals.

oph/cluster/storage.1675316971.txt.gz · Ultima modifica: 2023/02/02 05:49 da diego.zuccato@unibo.it

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki