Difference between revisions of "Large File Transfer"

From Dryad wiki
Jump to: navigation, search
(Uploading files through WeTransfer)
m
Line 4: Line 4:
  
 
If you have files larger than 1GB and receive an error when using the the Dryad submission system, please contact the Dryad staff at [mailto:help@datadryad.org help@datadryad.org]. The staff will analyze your situation and make recommendations for how to proceed.
 
If you have files larger than 1GB and receive an error when using the the Dryad submission system, please contact the Dryad staff at [mailto:help@datadryad.org help@datadryad.org]. The staff will analyze your situation and make recommendations for how to proceed.
 +
 +
== Uploading files through FTP ==
 +
 +
If you have files that are too large to transfer via the normal upload process, the curator may ask you to upload your files through FTP. You will be given instructions for accessing the Dryad FTP server. However, you will still need to complete the regular Dryad submission process using "dummy" files as placeholders for your actual files.
  
 
== Splitting large files ==
 
== Splitting large files ==
  
To ensure easy downloads, Dryad recommends that you limit the size of each individual file to 1GB. If you have a file larger than 1GB, it can be broken into pieces that are 1GB or smaller.
+
To ensure accessible downloads, Dryad recommends that you limit the size of each individual file to 10GB. If you have a file larger than 10GB, it can be broken into smaller pieces.
  
 
On Unix, Linux, or Mac OSX, you may use the "split" command, like this:
 
On Unix, Linux, or Mac OSX, you may use the "split" command, like this:
Line 21: Line 25:
 
On Windows, you may recombine files with the "copy" command, like this:
 
On Windows, you may recombine files with the "copy" command, like this:
 
copy /B partaa+partab+partac dataFile.txt
 
copy /B partaa+partab+partac dataFile.txt
 
== Uploading files through FTP ==
 
 
If you have files that are too large to transfer via the normal upload process, the curator may ask you to upload your files through FTP. You will be given instructions for accessing the Dryad FTP server. However, you will still need to complete the regular Dryad submission process using "dummy" files as placeholders for your actual files.
 
  
 
== Technical Details ==
 
== Technical Details ==

Revision as of 09:25, 3 January 2018

This page contains instructions for Dryad users who want to deposit very large files. Files larger than 1GB may not work with the normal Dryad submission system. Uploading these files may fail due to unstable internet connections or web browser timeouts. In these situations, Dryad can accept content uploaded through alternate mechanisms.

What to do if your files are large

If you have files larger than 1GB and receive an error when using the the Dryad submission system, please contact the Dryad staff at help@datadryad.org. The staff will analyze your situation and make recommendations for how to proceed.

Uploading files through FTP

If you have files that are too large to transfer via the normal upload process, the curator may ask you to upload your files through FTP. You will be given instructions for accessing the Dryad FTP server. However, you will still need to complete the regular Dryad submission process using "dummy" files as placeholders for your actual files.

Splitting large files

To ensure accessible downloads, Dryad recommends that you limit the size of each individual file to 10GB. If you have a file larger than 10GB, it can be broken into smaller pieces.

On Unix, Linux, or Mac OSX, you may use the "split" command, like this: split -b 999m myVeryLargeFile.txt

On Windows machines, you may split files with HJsplit.

Recombining files that have been split

On Unix, Linux, or Mac OSX, you may recombine files with the "cat" command, like this: cat partaa partab partac > dataFile.txt

On Windows, you may recombine files with the "copy" command, like this: copy /B partaa+partab+partac dataFile.txt

Technical Details

See Large File Technology.