Globus
Globus is ARCC’s recommended method of transferring data onto our resources. Globus provides a secure, unified interface to your research data. Use Globus to 'fire and forget' high-performance data transfers between systems within and across organizations. With Globus, users can do more beyond just transfer data Globus users can move, share, transfer, sync, and find data via a single interface regardless of where the data actually “lives”. Whether your files are housed on a supercomputer, lab cluster, tape archive, public cloud or your laptop, you can manage this data from anywhere, using your existing identities, accessible through your web browser.
To get started using Globus read the Using Globus with ARCC Guide, Enabling Globus on Personal Computers, Sharing Data With Globus or read the Globus Key concepts below.
Globus Key Concepts
It’s important to be aware of a key concepts that only apply to Globus transfers. This will clear up much of the confusion for the content discussed below.
Collection
A collection is a named location containing data you can access with Globus. Collections can be hosted on many different kinds of systems, including on campus storage systems, HPC clusters, laptops, or even S3 buckets. When you use Globus, you don’t need to know a physical location or details about storage. You only need a collection name. A collection allows authorized Globus users to browse and transfer files. Collections can also be used for sharing data with others and for enabling discovery by other Globus users.
GridFTP
Globus GridFTP is a high-performance, secure data transfer protocol designed for large-scale scientific data movement across distributed systems. It acts as the underlying transfer engine for Globus services, enabling parallel streams, striped transfers for increased bandwidth, and reliable transfers that automatically recover from network interruptions.
Fire-And-Forget Data Transfer
After you request a file transfer, Globus takes over and does the work on your behalf. You can navigate away from the File Manager, close the browser window, and even logout. Globus will optimize the transfer for performance, monitor the transfer for completion and correctness, and recover from network errors and collection downtime. When a problem is encountered part-way through the transfer, Globus resumes from the point of failure and does not retransmit all of the data specified in the original request. Globus can handle extremely large data transfers, even those that don’t complete within the authentication expiration period of a collection (which is controlled by the collection administrator). If your credentials expire before the transfer completes, Globus will notify you to re-authenticate on the collection, after which Globus will continue the transfer from where it was paused.