, printed on 11/21/2024
This page describes how authorized customers and partners can quickly and efficiently send large datasets, such as BCL, FASTQ, and BAM files, to 10x Genomics. This approach makes it easy to use multi-stream transfers to ensure optimal and secure utilization of your Internet connection. Redstone uploads to Amazon Web Services S3 which is secured using SSL during transport.
Illumina BCL Data
In the following example, replace the strings in
red with the specifics of your run.
/mnt/runs should be replaced with the parent folder that contains
your Illumina run folders.
$ tar -czvf H3NVFBBXX.tar.gz -C /mnt/runs --exclude=Thumbnail_Images \ 150929_ST-K00126_0004_AH3NVFBBXX
FASTQ or BAM Data
In the following example for a FASTQ directory, replace the strings in
red with the specifics of your run.
/fastq/directory should be replaced with actual directory that
contains your FASTQ files.
$ tar -czvf compressed-fastqs.tar.gz /fastq/directory
z option to compress.In the following example for a FASTQ directory, replace the strings in
red with the specifics of your run.
/fastq/directory should be replaced with actual directory that
contains your FASTQ files. The difference here is omitting a 'z' from the tar
command.
$ tar -cvf allfastqs.tar /fastq/directory
Imaging Data
<slide_serial_number>-<capture_area>.tifExample Slide: Lot# V19L22, ID# 001.
The four capture areas (from top to bottom) would be submitted as images named:
V19L22-001-A1.tif
V19L22-001-B1.tif
V19L22-001-C1.tif
V19L22-001-D1.tif
10x Genomics provides an easy-to-use multi-stream upload tool called Redstone. If you installed Cell Ranger v4 or higher, the Redstone executable is already located in the Cell Ranger install path cellranger-x.y.z/lib/bin/redstone and need not be installed again. You may add this path to your .bashrc using export PATH=/path/to/cellranger-x.y.z/lib/bin:$PATH or type in the full path on your command prompt to call Redstone.
The download link below is for the Linux version, compatible with RedHat/CentOS 5.2 or later, and Ubuntu 8.04 or later.
Redstone is a single 9MB executable that can be run directly and requires no
compilation or installation. Place the executable file in a directory that is on
your PATH. For this Linux version please make sure to chmod
700 to make it executable.
Make sure to perform this step from a computer with an unencumbered path to your Internet connection. That means hard-wired 1GbE or 10GbE connections, and not Wi-Fi.
Please also ensure the time/date are set correctly on your system, which is required for security reasons. If your clock is mis-set, you will encounter a RequestTimeTooSkewed error.
To run Redstone, just pass it your email address, the email address of the recipient at 10x Genomics, and the name of the file to be uploaded. Code in red must be customized before running.
$ redstone --from=[email protected] --to=[email protected] H3NVFBBXX.tar.gz Uploading file... Size: 10,773,244,733 bytes 50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
[email protected] as the recipient e-mail address.10x Genomics provides an easy-to-use multi-stream upload tool called Redstone. The link below is for the macOS version.
After you click the link to download redstone, the executable will download to your Downloads folder. Right click on the file and select Open. You will likely see this message on a pop-up window:
macOS cannot verify the developer of “redstone-mac”. Are you sure you want to open it?
Click Open. It is not a .dmg file so the installer will try to open the application in your default text editor. This file is not human-readable so you can close it. However, this step is necessary because it changes how the macOS gatekeeper views the file, from unsafe to safe.
Next, use the terminal program on your Mac to open a terminal window. A window will open with a prompt that looks something like this:
user.name@computer:~$
Use the chmod command to make the redstone-mac file executable.
user.name@computer:~$ chmod 700 Downloads/redstone-mac
Move the redstone program to the Applications folder on your computer using the mv command.
user.name@computer:~$ mv Downloads/redstone-mac /Applications
Next we need to add redstone to your
user.name@computer:~$ export PATH=/Applications:$PATH
Now if you type redstone-mac on the command line and push enter it will print the usage statement. The usage statement will contain a pattern for building the command you will use to transfer your data.
user.name@computer:~$ redstone-mac Usage: redstone --from=EMAIL --to=EMAIL[options] redstone -h | --help | --version
If you are sending a single file, such as a tissue optimization cDNA footrpint
tiff, you would replace
user.name@computer:~$ redstone-mac --from=[email protected] --to=[email protected] /Users/username/Downloads/V19L22-001-A1.tif Uploading file... Size: 10,773,244,733 bytes 50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
If you are sending a compressed directory of files, like you would have if you compressed multiple images using the tar command described in "Prepare Your Data" above, the final command will have the same structure as sending a single file and may look like this:
user.name@computer:~$ redstone-mac --from=[email protected] --to=[email protected] /Users/username/Downloads/H3NVFBBXX.tar.gz Uploading file... Size: 10,773,244,733 bytes 50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
[email protected] as the recipient e-mail address.10x Genomics provides an easy-to-use multi-stream upload tool called Redstone. The link below is for the windows version.
After you click the link to download redstone, the executable will download to your Downloads folder. Do NOT click on the downloaded file. Installation does not require the windows installer, it will give an error. Instead do the following:
Use the Command Prompt app to open a Command prompt. You can find Command Prompt by searching for it in the start menu search bar. A window will open with a prompt that looks something like this:
C:\Users\username>
Now if you type downloads\redstone.exe on the command line and push enter it will print the usage statement. The usage statement will contain a pattern for building the command you will use to transfer your data.
C:\Users\username> Downloads\redstone.exe Usage: redstone --from=EMAIL --to=EMAIL[options] redstone -h | --help | --version
To transfer a file the first step is to change directories to the directory that contains the file you want to transfer using the cd command. If the file is in the Downloads directory the cd command would look like this:
C:\Users\username> cd \Users\username\Downloads
The redstone.exe command would then start with a path to the Redstone executable followed by the --to= and --from= email addresses and ending with the name of the file that you want to transfer. The command would look like this:
C:\Users\username> \Users\username\Downloads\redstone.exe --from=[email protected] --to=[email protected] V19L22-001-A1.tif Uploading file... Size: 10,773,244,733 bytes 50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
[email protected] as the recipient e-mail address.Error Message: 'redstone.exe' is not recognized as an internal or external command, operable program or batch file.
This error indicates that redstone.exe program did not download properly, the full path to redstone.exe was not provided at the command prompt, or the name of the executable was misspelled. Please try one of these solutions: