Moving Data From CHPC with smbclient

Owned by RIS-SVC-conf-publish
Last updated: Jun 06, 2023

Overview

This document serves as a guide for current CHPC users who wish to transfer data to RIS storage1.

Prerequisites

  1. Meet the assumptions listed for accessing storage1.

  2. A CHPC account with access to the data node.

  3. A user account for RIS storage1 and compute1 services.

Transferring data from CHPC to storage1

This guide will provide a demo transfer of a 1 GB file from the CHPC scratch folder to RIS storage1.

The file in question is named file_1.bin and is located in the CHPC scratch folder /scratch/stahan.

The goal is to transfer the file to a folder in the RIS storage1 Active directory named chpc_data.

I. Connecting to the CHPC Data Node

  • Open a terminal session and connect to the CHPC Data Node.

ssh USERNAME@dtn01.chpc.wustl.edu

Where USERNAME is your CHPC username. For more information on connecting via the terminal, including required software, please reference the compute quick start page.

II. Connect to storage1 using smbclient

a. Create a file to store smbclient credentials in your home directory.

touch ~/.smb_creds

b. Enter the following into the ~/.smb_creds file using your text editor of choice, substituting WUSTL KEY ID and WUSTL KEY PASSWORD with your own WUSTL Key credentials.

username = WUSTL KEY ID
password = WUSTL KEY PASSWORD
domain = ACCOUNTS.AD.WUSTL.EDU

c. Use smbclient to connect to your storage1 allocation replacing WUSTL_KEY with the WUSTL Key that the storage allocation belongs to (e.g., a PI).

smbclient -A ~/.smb_creds -m SMB3 //storage1.ris.wustl.edu/WUSTL_KEY

You will be greeted with the smbclient prompt

II. Transfer files of interest to storage1

In the smbclient prompt, navigate to the local directory on CHPC in which your file of interest resides using lcd.

lcd /scratch/stahan

Navigate to the destination folder using the cd command.

cd Active/chpc_data

The chpc_data folder is currently empty. This can be verified with the ls command.

Using the put command, The file can be ‘put’ from CHPC to RIS. The syntax for the put command is put <local file name> [remote file name]

put file_1.bin file_1.bin

This command will transfer file_1.bin from the CHPC scratch folder to RIS storage1 and save the file as file_1.bin

The 1 GB file transferred at ~ 80 MB/s and is now located in RIS storage1. This can be verified again using the ls command.

References