/
Containers with SLURM Jobs: Pyxis/Enroot Integration

You can use our service desk portal for getting RIS support. RIS also offers 15 min. virtual office hours session Mon-Thru..

Containers with SLURM Jobs: Pyxis/Enroot Integration

Compute2 utilizes Enroot via Pyxis. Enroot is container runtime that allows users to launch containers from Docker images and to pull them from common container registries such as Docker Hub, Nvidia Container Registry, or Github Container Registry. Pyxis is a Slurm plugin that allows access to Enroot through srun and sbatch.

Basic Examples

  • The flag --container-image= will pass your desired image via Pyxis.

  • Launching a basic image with Pyxis (please replace compute2-groupname with the appropriate Slurm account):

MODULE_MOUNTS='/etc/profile.d,/etc/sysconfig/modules,/cm,/opt/thpc' srun -A compute2-account -p general-interactive \ --container-image='ghcr.io#washu-it-ris/ris-thpc:rocky9.2' \ --container-mounts=${MODULE_MOUNTS} \ --pty /bin/bash

  • Other flags are available when needed:

MODULE_MOUNTS='/etc/profile.d,/etc/sysconfig/modules,/cm,/opt/thpc' srun -A compute2-account -p general-interactive \ --container-image='ghcr.io#washu-it-ris/ris-thpc:rocky9.2' \ --containter-mounts=${MODULE_MOUNTS},/storage2/fs1/myuser/Active/data,~/outputs:/custom-outputs \ --container-env=MYVAR,OUTPUT_DIR \ --pty /bin/bash

More Container Options

  • --container-mounts='/path/example/a,/path/example/b:/dest/example/b': Mounts paths to be accessed inside the contain. Paths are comma separated and if the src:dest syntax is not used, it will be mounted to the same path as the source.

    Note: If launching a container based off of the ris-thpc image, the following mounts need to be included for lmod to function:
    MODULE_MOUNTS='/etc/profile.d,/etc/sysconfig/modules,/cm,/opt/thpc'

    More info: C2 THPC

  • --container-env=NAME: A list of environment variables that will be forwarded to the container.

  • --container-entrypoint: command to execute

  • --container-workdir: The initial directory to run the provided script out of.

  • Additional flags can be found on the Pyxis documentation.

Authentication for Private Repositories

To access private repositories, you can add your access credentials to ~/.config/enroot/.credentials.

  1. Verify that the ~/.config/enroot directory exists, as it may not be instantiated by default.

    mkdir -p ~/.config/enroot

  2. Open up the .credentials file.

    vim ~/.config/enroot/.credentials

  3. Add the following line to the .credentials file:

    machine <registry endpoint login <username> password <token>

    1. E.g.

      machine ghcr.io login your_username password <token>

  4. Now private images can be access from the added container registry. To add other registries, append additional lines to the .credentials file.

 

Multiple registries can be contained within the .credentials file separated by new lines:

machine ghcr.io login <username> password <token> machine nvcr.io login <username> password <token> machine auth.docker.io login <username> password <token>

Please make sure to replace <username> and <token> with your respective values after copying the details from here.