File Storage on Iceberg

  • Every user has a home area under the directory /home/username for 10GB.
  • Every user has access to a permanent data-storage area provided under /data/username for 100 GB.  
  • Every user also has access to a temporary, very large data storage area under /fastdata
  • To find out your current filestore quota allocation and usage type quota 

Whether you are working on ShARC or iceberg you have access to the same file storage under your /data and /home areas. That is to say ShARC and iceberg use the same /home and /data locations. Therefore there is no need to transfer files between ShARC and iceberg. 
However /fastdata areas are separate for ShARC and iceberg.

For further information please see HPC documents on file storage

If you exceed your file storage allocation

As soon as the quota is exceeded your account becomes frozen. In order to avoid this situation it is strongly recommended that you -

  • Use the quota command to check your usage regularly.
  • Copy files that you do not need to be backed up to the /data/username area.
    Example:  cp mylargefile  /data/${USER}

       If your account becomes frozen you can remove unwanted files by using the RM command (note upper case) and you will be automatically unfrozen.

Alternatively, if you have larger file storage requirements please request it by emailing research-it@sheffield.ac.uk .  


Recovering Accidentally deleted files 

If you notice the accidental deletion or corruption of an important file, it is advisable to attempt its recovery quickly.

CURRENT BACKUP POLICY

  • We take snapshots of /home and /data areas every 4 hours and keep the 10 most recent snapshots,
  • We also take nightly snapshots of /home areas and keep these on a separate storage area,
  • We keep 28 days of /home area snapshots and only 7 days of /data areas.

There are 7 days worth of snapshots available in your /home and /data directories in a hidden directory called .snapshot. You need to explicitly cd into this directory to get at the files:

cd /home/$USER/.snapshot
The files are read-only. This allows you to attempt to recover any files you might have accidentally deleted recently.

This does not apply for /fastdata for which we take no back-ups!