Versions Compared

Version Old Version 168 New Version 169
Changes made by

RIS-SVC-conf-publish

RIS-SVC-conf-publish

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Video - An Example in Using Globus

Html bobswift
<iframe width="560" height="315" src="https://www.youtube.com/embed/8_M7pPT1-so" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

What is Globus?

From https://www.globus.org/what-we-do:

Globus is a non-profit service for secure, reliable research data management.

With Globus, subscribers can move, share, & discover data via a singleinterface single
interface – whether your files live on a supercomputer, lab cluster, tapearchivetape
archive, public cloud or your laptop, you can manage this data from anywhere,
using your existing identities, via just a web browser.

Developers can also use Globus to build applications and gatewaysleveraging gateways
leveraging our advanced identity management, single sign-on, search,
authorization, and automation capabilities.

RIS and Wash U IT have a partnership with Globus to provide an integrated solution for data movement.

Quick Start

1. Log in with an existing identity

  • Visit https://app.globus.org and click “Login” at the top of the page. On the Globus login page, choose an organization you’re already registered with, such as your school or your employer.

  • Select “Washington University in St. Louis” in the Organization widget and click “Continue”:

...

  • Once you’ve logged in with your organization, Globus will ask if you’d like to link to an existing account. If this is your first time logging in to Globus, click “Continue.” If you’ve already used another account with Globus, you can choose “Link to an existing account.”

  • You may be prompted to provide additional information such as your organization and whether or not Globus will be used for commercial purposes. Complete the form and click “Continue.”

  • Finally, you need to give Globus permission to use your identity to access information and perform actions (like file transfers) on your behalf.

2. The File Manager

  • After you’ve signed up and logged in to Globus, you’ll begin at the File Manager.

...

Info

A collection is a named location containing data you can access with Globus. Collections can be hosted on many different kinds of systems, including campus storage, HPC clusters, laptops, Amazon S3 buckets, Google Drive, and scientific instruments. When you use Globus, you don’t need to know a physical location or details about storage. You only need a collection name. A collection allows authorized Globus users to browse and transfer files. Collections can also be used for sharing data with others and for enabling discovery by other Globus users. Globus Connect is used to host collections.

3. Access a collection

  • Click in the Collection field at the top of the File Manager page and type “globus tutorial end”. Globus will list collections with matching names.

...

  • WashU RIS has two endpoints within Globus. Wash U RIS storage1 dtn1 and Wash U RIS storage1 dtn2.

  • Once you’ve selected one of the storage endpoints, you can see the data by entering /storage1/fs1/${STORAGE_ALLOCATION}/Active/ where allocation_name is the name of your or your group’s storage allocation.

...

4. Request a file transfer

  • Click Transfer or Sync to… in the command panel on the right side of the page. A new collection panel will open, with a “Transfer or Sync to” field at the top of the panel.

...

Info

  • After you request a file transfer, Globus takes over and does the work on your behalf. You can navigate away from the File Manager, close the browser window, and even logout.

  • Globus will optimize the transfer for performance, monitor the transfer for completion and correctness, and recover from network errors and collection downtime.

  • The Globus service routinely achieves high availability, providing nearly uninterrupted oversight of data transfers taking place on much less reliable networks and collection hosts.

  • When a problem is encountered part-way through the transfer, Globus resumes from the point of failure and does not retransmit all of the data specified in the original request.

  • Globus can handle extremely large data transfers, even those that don’t complete within the authentication expiration period of a collection (which is controlled by the collection administrator).

  • If your credentials expire before the transfer completes, Globus will notify you to re-authenticate on the collection, after which Globus will continue the transfer from where it was paused.

  • These wide-ranging capabilities make data transfer with Globus truly “fire-and-forget.”

5. Confirm transfer completion

  • You can confirm the transfer is complete in the Activity page. You will also be emailed about the status of your transfer once it is complete, or if it errors.

...

Sharing Data (“Collections”)

  • With Globus, you can easily share research data with your collaborators. You don’t need to create accounts on the server(s) where your data is stored.

  • You can share data with anyone using their identity or their email address.

  • To share data, you’ll create a guest collection and grant your collaborators access as described in the instructions below.

Info

Sharing with guest collections is available on Globus endpoints managed by an active subscription. You can create guest collections on Globus Connect Personal endpoints (your laptop, for example) if you are a Globus Plus user. Please contact your research computing center or IT manager to upgrade your Globus account to Plus. If your institution does not have a subscription, please contact us to discuss your needs.

1. Log Into Globus and Navigate to the File Manager

  • Select the collection that has the files/folders you wish to share and, if necessary, activate the collection.

  • Highlight the folder that you would like to share and Click Share in the right command pane.

...

  • You can add permissions to subfolders by entering a path in the Path field.

  • After receiving the email notification, your colleague can click on the link to log into Globus and access the guest collection. In the example below, user grohder1@globusid.org accesses the guest collection. Note that the collection name is Demo and the path is /, because this is what the user was given access to.

RIS Managed Endpoints

Click on any of these endpoint links to open them in the Globus Web App file manager.

Research Storage

Legacy

Personal Endpoints

Follow Globus’ instructions for running a “personal endpoint” on your own local machine in order to transfer between it and other endpoints.

...

Check out our documentation on Globus Connect Personal here.

Current Limitations

  • We do not yet include the Globus connectors for AWS S3, Google Drive, Box, or others. These are on the roadmap.

  • Our endpoints do not support Globus’ High Assurance Collections for Protected Data (i.e. HIPAA) features.

  • Globus will not log or report deleted filenames when the “delete files on destination that do not exist on source” transfer option is selected.

  • In the event of a failed transfer task, Globus will not log or report the names of any partially transferred files. Files should only be considered successfully transferred if the transfer task succeeds or the files appear in the list of successfully transferred files.

  • Names of files successfully transferred by a task are only retrievable with the CLI or SDK, not the web UI. See Viewing Transfer Task Status for more information.

  • Task information is only retained by Globus for 90 days. This is not controlled by RIS.