10x Genomics Support/Cloud Analysis/Tutorials/

Upload via CLI for Linux

10x Genomics Cloud Analysis allows you to upload FASTQ files into your projects either using the 10x Genomics Cloud CLI (described below) or the Web Uploader.

For the easiest functionality, 10x Genomics recommends using the Web Uploader. For uploading large numbers of FASTQ files, it is recommended to use the Cloud CLI.

What is the 10x Genomics Cloud CLI?

The 10x Genomics Cloud CLI is a command line tool that allows you to upload custom references and FASTQ files to your 10x Genomics account, create projects from the command line, and manage other tasks related to your 10x Genomics account.

It is an executable that can be run directly and requires no compilation or installation.

The steps are operating system dependent. Please select the tab for your operating system below to continue.

Below is a step-by-step video demonstrating how to upload FASTQs to 10x Genomics Cloud on macOS operating systems. Linux users will find the installation and usage instructions nearly identical to macOS users.

Download the 10x Genomics Cloud CLI for your operating system and unpack it to a convenient location. Make sure you download the 10x Genomics Cloud CLI for the operating system where your data lives. Your home directory is a good default location.

Optionally, install the binary in a location where you can execute it globally (e.g., /usr/local/bin).

If you’re using a Linux system, use the command below to download:

curl -f -o txg-linux-v1.3.1.tar.gz https://cf.10xgenomics.com/cloud-cli/v1.3.1/txg-linux-v1.3.1.tar.gz

and then to unpack:

tar -zxvf txg-linux-v1.3.1.tar.gz

To run the 10x Genomics Cloud CLI, you need to enter the path to the executable.

Option 1: You can navigate into the executable's directory, for example txg-linux-v1.3.1, and run the command from within the directory every time you want to run a command. For example:

cd /home/user.name/apps/txg-linux-v1.3.1/ ./txg help

Option 2: Or you can specify the full path to the executable every time you want to run a command. For example:

/home/user.name/apps/txg-linux-v1.3.1/txg help

Option 3: Or you can add the txg executable to your $PATH. This means you can type txg [command] from anywhere to access the executable and run commands. If you use the echo command below, you should see a list of directories separated by colons similar to below.

echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin

If you put the 10x Genomics Cloud CLI executable in one of these directories, you've added it to your $PATH and can run the txg command from anywhere. There are several ways to add an executable to your $PATH. Here's one method:

First, navigate into the directory you just downloaded and unpacked.

cd txg-linux-v1.3.1/

Then print the path to current directory using this command:

pwd

The output should be the directory containing the txg executable. Copy the outputted path and use it in the following export command to add it into the $PATH variable.

export PATH=[directory containing the executable]:$PATH

For example:

export PATH=/mnt/home/user.name/yard/apps/txg-linux-v1.3.1:$PATH

To verify you correctly added the 10x Genomics Cloud CLI to your $PATH, enter:

which txg

You should see output similar to the following:

/mnt/home/user.name/yard/apps/txg-linux-v1.3.1/txg

Note: this only temporarily adds the executable to your $PATH. It's only valid in the current shell session. To permanently add txg to your $PATH, add the export command you just used to your .bashrc (Linux) or .zshrc (macOS v10.15+), which is a special script that runs everytime you login to your system. For more details, view this tutorial.

To upload files into a project, use the command provided in your project. You will also need to specify the project ID and the path to the FASTQ files.

You will need to execute the command from within the txg-linux-v1.3.1, txg-macos-v1.3.1, or txg-windows-v1.3.1 directory, specify the path to the directory, or add txg to your $PATH.

The project ID is a unique identifier that is automatically generated when you create a project. You can find the project ID in the upload command provided in your project.

./txg fastqs upload --project-id [insert unique project id] [insert path to fastqs]

Your FASTQ files must follow the Illumina naming convention, ex. SampleName_S1_L001_R1_001.fastq.gz.

Example upload commands

Below are three example commands for uploading FASTQ files to a project.

Note: the 10x Genomics Cloud CLI will only upload FASTQ files to your project, even if there are other file types present in the path you provide.

To upload all the FASTQ files in a given folder:

./txg fastqs upload --project-id fo91F3f3jQTipjfGKpdV6lg myfastqs/

To upload a specific file:

./txg fastqs upload --project-id fo91F3f3jQTipjfGKpdV6lg myfastqs/SampleName_S1_L001_R1_001.fastq.gz

To upload all the FASTQ files in a folder that begin with Sample01:

./txg fastqs upload --project-id fo91F3f3jQTipjfGKpdV6lg myfastqs/Sample01*

Enter your access token

The first time you use the uploader, you will be prompted to provide an access token, which verifies your identity. The access token can be found in your Account Settings.

After verifying your identity, you will see a message similar to the following:

You are about to upload 3 file(s) to the following project Project ID: 7e8fc23dd-fde3-4138-4138-f18aa5d57a96 Project Name: "My First Project" Number of files: 3 /Users/your.name/myfastqs/test_sample_S1_L001_I1_001.fastq.gz (1.5 kB) /Users/your.name/myfastqs/test_sample_S1_L001_R1_001.fastq.gz (2.9 kB) /Users/your.name/myfastqs/test_sample_S1_L001_R2_001.fastq.gz (7.2 kB) Total: 12 kB. You will receive an email when the upload is complete. To stop the upload, press Ctrl+C. Proceed? [y/N]

Example success message:

Starting upload... /Users/your.name/myfastqs/test_sample_S1_L001_I1_001.fastq.gz (1.5 kB) Done /Users/your.name/myfastqs/test_sample_S1_L001_R1_001.fastq.gz (2.9 kB) Done /Users/your.name/myfastqs/test_sample_S1_L001_R2_001.fastq.gz (7.2 kB) Done Your upload has successfully completed! You can view and analyze your data in the project "My First Project".

You will receive an email once the upload is complete. The email will be sent to the address associated with your 10x Genomics account. Once the upload is complete, go back to your project and start analyzing the uploaded files.

For a full reference of every command and flag available in the 10x Genomics Cloud CLI, visit the 10x Genomics Cloud CLI Documentation.