Managing Apptainer/Singularity Containers
- Mark Bober
The ENGR cluster supports Apptainer(Singularity) containers in job submissions.
Apptainer is generally able to take any Docker-based container and convert it. EIT is ready to help with any specific need in using an existing or developing a new container for your workloads.
EIT provides a specific host for users to build and manage their own containers if required. SSH, from the VPN or another ENGR host, to build.engr.wustl.edu.
Upon login, you’ll receive a new home directory, not connected to your research or academic home, where you can work with containers using ‘sudo’ commands. RIS storage is available in the usual locations.
Apptainer and Docker can be installed on Linux, and also Windows and Mac desktops with some extra effort. See the Apptainer instructions at https://apptainer.org/docs/admin/main/installation.html , and Docker instructions at https://docs.docker.com/desktop/ . Do note that Macs with Apple silicon (ARM architecture) require you to utilize emulation to develop x86 compatible containers, and functionality and stability may be limited.
Creating Singularity Containers
As most existing projects utilize Docker Hub for providing images, instructions here will include getting those images into Apptainer format.
sudo apptainer pull docker://rockylinux:8.5 rockylinux-8.5.sif
The above command pulls, from Docker Hub, the listed container and saves it as a SIF file that can be copied elsewhere for use.
sudo apptainer build --sandbox rocky-8.5/ docker://rockylinux:8.5
The above does the same thing, but it brings the container down as a sandbox - an unpacked directory tree that is the contents of the container - so that the container can be edited.
In this form the container can be changed. If you need to insert or change files, you can simply do that in-place. If you need to perform installations of packages, you can enter the container in a writable mode:
sudo apptainer shell --writable rocky-8.5/
This places you inside the container, as if it was running during a job, in a writable mode. From here you can install packages with yum or other software as you might on a local Linux desktop or VM.
Before you exit out of the container, if you’ve made changes, clean up temporary files to save space. For yum or apt, clean their cache:
yum clean all
or
apt-get clean
when working with RPM or DPKG based containers, respectively.
To package the container back up:
sudo apptainer build my-rocky-8.5.sif rocky-8.5/
and from there you can copy the SIF file to a usable location.