File Transfers: Difference between revisions
Jump to navigation
Jump to search
No edit summary Tag: Manual revert |
(changes) |
||
| Line 3: | Line 3: | ||
:• '''Globus Online''': The preferred method for large files, with extra features for parallel data streams, auto-tuning and auto-fault recovery. Globus online is to transfer files between systems—between the CUNY HPC Center resources and XSEDE facilities, or even users’ desktops. A typical transfer rate ranges from 100 to 400 Mbps. | :• '''Globus Online''': The preferred method for large files, with extra features for parallel data streams, auto-tuning and auto-fault recovery. Globus online is to transfer files between systems—between the CUNY HPC Center resources and XSEDE facilities, or even users’ desktops. A typical transfer rate ranges from 100 to 400 Mbps. | ||
: [https://cunyhpc.csi.cuny.edu/zircon_d8/sites/default/files/Globus_Instructions.pdf Details on connecting via Globus can be found here.] | : [https://cunyhpc.csi.cuny.edu/zircon_d8/sites/default/files/Globus_Instructions.pdf Details on connecting via Globus can be found here.] | ||
: | |||
: • '''cea.csi.cuny.edu''': Only Secure FTP (SFTP) can be used to transfer files to cea.csi.cuny.edu. Each of the servers is mounted on cea under its own name. Thus the users could place their files on particular server simply by preceding /scratch with /<name of the server> e.g. '''sftp> put TEST.txt /penzias/scratch/john.doe/TEST2.txt''' | |||
:• '''cea.csi.cuny.edu''': Only Secure FTP (SFTP) can be used to transfer files to cea.csi.cuny.edu. Each of the servers is mounted on cea under its own name. Thus the users could place their files on particular server simply by preceding /scratch with /<name of the server> e.g. '''sftp> put TEST.txt /penzias/scratch/john.doe/TEST2.txt''' | |||
:• '''iRODS''': The data grid/data management tool provided by CUNY HPC Center for accessing the '''SR1''' resource. iRODS clients ('''<font face="courier">iput, iget, irsync</font>''') provide a data transfer mechanism featuring bulk upload and parallel streams. Not all methods are offered for all file systems. Here is a summary on the available methods per file space: | :• '''iRODS''': The data grid/data management tool provided by CUNY HPC Center for accessing the '''SR1''' resource. iRODS clients ('''<font face="courier">iput, iget, irsync</font>''') provide a data transfer mechanism featuring bulk upload and parallel streams. Not all methods are offered for all file systems. Here is a summary on the available methods per file space: | ||
[[File:Example File Transfer.png|image]] | [[File:Example File Transfer.png|image]] | ||
Revision as of 00:38, 3 March 2023
There are three methods of transferring data between the CUNY HPC systems and the rest of the world:
- • Globus Online: The preferred method for large files, with extra features for parallel data streams, auto-tuning and auto-fault recovery. Globus online is to transfer files between systems—between the CUNY HPC Center resources and XSEDE facilities, or even users’ desktops. A typical transfer rate ranges from 100 to 400 Mbps.
- Details on connecting via Globus can be found here.
- • cea.csi.cuny.edu: Only Secure FTP (SFTP) can be used to transfer files to cea.csi.cuny.edu. Each of the servers is mounted on cea under its own name. Thus the users could place their files on particular server simply by preceding /scratch with /<name of the server> e.g. sftp> put TEST.txt /penzias/scratch/john.doe/TEST2.txt
- • iRODS: The data grid/data management tool provided by CUNY HPC Center for accessing the SR1 resource. iRODS clients (iput, iget, irsync) provide a data transfer mechanism featuring bulk upload and parallel streams. Not all methods are offered for all file systems. Here is a summary on the available methods per file space:
