Page Comparison

• Anchor
  id3 id3

  Compute Data Transfer Policy

• Anchor
  id4 id4

  Storage Service Integration Points

• Anchor
  id5 id5

  Checking Storage Usage

  • Anchor
    id6 id6

    Storage Service Allocations

    • Anchor
      id7 id7

      SMB Inteface

    • Anchor
      id8 id8

      Cache Interface

  • Anchor
    id9 id9

    Compute Service Home Directories

  • Anchor
    id10 id10

    Compute Service Scratch Space

• Anchor
  id11 id11

  Staging Data

• Anchor
  id12 id12

  Addtional Notes

Compute Data Transfer Policy

Supported methods to transfer data into and out of the Scientific Compute Platform are:

1. Globus

2. Rclone

3. Submitting a job using a data transfer tool (e.g. rsync, wget, curl, scp)

   • Please do not use these data transfer tools directly on the compute1 client nodes. This type of activity can slow down the client node and negatively affect all users connected to the client nodes.

   • If you require assistance submitting jobs using these tools, please open a ticket at our Service Desk.

Storage Service Integration Points

In addition to the Storage Service SMB Interface to a given Allocation, the Compute Service reveals another three interfaces to the Storage Service. This brings the total number of different types of locations where RIS Storage Space can be consumed to four:

...

Each of these types of locations have different methods and policies for managing and inspecting usage. This helps balance the availability of space and performance with the capabilities of the resources that provide them.

Checking Storage Usage

Storage Service Allocations

An accurate report of a Storage Service Allocation’s space consumption can only be obtained through the Storage Service SMB Interface. This is a known limitation of the Storage Service.

SMB Inteface

The easiest way to do this is to mount the allocation to your desktop, right-click the mounted folder, and select the appropriate menu option for more information.

...

Telling us that this Allocation is using 2.14 TiB out of a 128 TiB limit.

Cache Interface

...

If you encounter a conflict, contact the RIS Service Desk for assistance.

Compute Service Home Directories

Every Compute Service User is assigned a limit of 9 GiB of home directory space on the Compute Platform. This space is restricted at the user level, and can only be checked with the appropriate mmlsquota command:

...

There is no SMB interface to this space, and df reports space for the entire device, which is shared among all home directories.

Compute Service Scratch Space

High-performance Scratch Space is typically allocated for each lab as it is onboarded to the Compute Service. This space is restricted at the group level, which should represent an eponymous lab. Because it is a shared device like that for home directories, this usage must also be inspected with the appropriate mmlsquota command, referencing a group name and group quota on the scratch device:

...

$ groups | grep -Po 'compute-\S+' | while read COMPUTE_GROUP
> do ls -ld "/scratch1/fs1/${COMPUTE_ALLOCATION}"
> mmlsquota -g "$COMPUTE_GROUP" --block-size auto scratch1-fs1
> done
drwxr-sr-x. 6 root compute-ris 4096 Aug 23 02:43 /scratch1/fs1/ris

Disk quotas for group compute-ris (gid 1208827):
                         Block Limits                                    |     File Limits
Filesystem type         blocks      quota      limit   in_doubt    grace |    files   quota    limit in_doubt    grace  Remarks
scratch1-fs1 GRP          2.168T         3T         4T          0     none | 33226772       0        0        0     none
drwx--S---. 2 root compute-corcoran.william.p 4096 Aug 23 02:24 /scratch1/fs1/corcoran.william.p

Disk quotas for group compute-corcoran.william.p (gid 1262586):
                         Block Limits                                    |     File Limits
Filesystem type         blocks      quota      limit   in_doubt    grace |    files   quota    limit in_doubt    grace  Remarks
scratch1-fs1 GRP               0        50T        50T          0     none |        3       0        0        0     none

Staging Data

The Compute Service home directories and Scratch Space are not accessible from outside of the Compute Platform. Data should be staged to these locations from a Storage Service Allocation, and computational result or job output data should then be staged back to a Storage Service Allocation.

Addtional Notes

...