Quick answer: Public sequencing data lives in three connected places — the Sequence Read Archive (SRA, mirrored by ENA in Europe and DDBJ in Japan) holds raw reads, GEO holds curated studies with processed supplementary files, and BioProject/BioSample hold the metadata that ties them together. To get data into your pipeline: resolve the study to its runs, install SRA Toolkit, then prefetch a run and convert it with fasterq-dump --split-files. Faster for most of the world: skip the toolkit entirely and pull the same FASTQ files over HTTPS from ENA’s portal API, which lists exact URLs, byte sizes and MD5 checksums in one request.

The hands-on companion to bioinformatics databases on this blog — that guide explains what each archive is for; this one gets the bytes onto your disk, ready for the NGS QC step that always comes next.

Know your accession: which string means what

Every download problem starts as an identifier problem. The prefixes are not interchangeable:

PrefixExampleWhat it identifies
PRJNA / PRJEBPRJNA123456BioProject — the umbrella record for a study
SRP / ERPSRP012345SRA study — all runs under one project
SRR / ERR / DRRSRR1234567One sequencing run — what you actually download
SRS / ERSERS123456The biological sample a run came from
GSEGSE123456GEO Series — a curated study with processed files
GSMGSM1234567GEO Sample — one sample’s data and metadata

The rule: papers cite the study (PRJNA, GSE), tools download the run (SRR). The study-to-run mapping is public metadata — SRA Run Selector shows it as a table, and ENA’s API returns it as a file, which is the first thing to fetch before downloading anything. Taking two minutes to read the run table saves hours of downloading the wrong libraries: it tells you each run’s layout (single or paired), platform, library strategy (RNA-seq, WGS, amplicon), and size in gigabytes.

Route 1: SRA Toolkit

NCBI’s toolkit is the canonical path, installable from bioconda or the NCBI installer:

conda install -c bioconda sra-tools

# pull the raw SRA object, then convert to FASTQ
prefetch SRR1234567
fasterq-dump SRR1234567 --split-files --threads 4 --outdir fastq/

--split-files writes _1 and _2 for paired runs and a plain file for single-end. Two practical notes: fasterq-dump needs temporary space roughly equal to the output size, so point --temp at a disk that can take it; and for a whole project, loop the runs from the metadata table rather than guessing accessions one at a time. The older fastq-dump still appears in tutorials — it is slower and single-threaded, and new work should use fasterq-dump.

One caveat on scale: prefetching a hundred-gigabyte study one run at a time is slow. When the dataset is large, the next route is usually the better one.

Route 2: ENA’s API (usually faster)

ENA mirrors the same reads in Europe and publishes a plain HTTPS index for every accession — study, sample or run:

curl -s "https://www.ebi.ac.uk/ena/portal/api/filereport?accession=SRR1234567\
&result=read_run&fields=run_accession,fastq_ftp,fastq_bytes,fastq_md5" -o ena.tsv

The response contains direct ftp.sra.ebi.ac.uk URLs, per-file byte counts, and MD5 checksums — everything needed for a resumable, verifiable download:

# fastq_ftp holds ;-separated URLs; fetch them with curl and check sizes
awk -F'\t' 'NR>1 {n=split($2,u,";"); for(i=1;i<=n;i++) print u[i]}' ena.tsv |
while read -r url; do
  curl -L -C - -O "https://$url"
done

Because ENA returns the same runs as SRA (they are mirrors, not different data), this route is not a workaround — it is the standard method outside North America, and it sidesteps Toolkit’s temp-space requirement entirely. It also slots directly into workflows: the same API call inside a Nextflow or Snakemake process turns data acquisition into a reproducible pipeline step, as described in the workflows guide.

Route 3: GEO’s processed files

GEO is a curation layer, not a read archive. A GEO Series page carries three kinds of material:

  • Processed data — count matrices, differential-expression tables, peak files, in each sample’s Supplementary file section or the series FTP directory. Often this is all an reanalysis needs.
  • Raw reads — usually not stored in GEO itself; the record links out to its SRR runs in SRA. If a paper says “raw data available at GSE…”, follow the link table to the runs.
  • Series matrix files — expression values plus sample annotations, downloadable from the series FTP folder for quick reprocessing without touching FASTQ.
# GEO supplementary files live under a bucketed FTP path (nnn = last 3 digits)
wget -O sample_suppl.tar.gz \
  "https://ftp.ncbi.nlm.nih.gov/geo/samples/GSM12345nnn/GSM1234567/suppl/sample_suppl.tar.gz"

Download the processed matrix first and inspect it; most “I need the raw data” requests turn out to need only the matrix plus the metadata table.

Verify before you analyse

Three checks, all cheap, all worth doing before a week of compute:

  • Size. Compare downloaded bytes against fastq_bytes (ENA) or the Run Selector’s reported size. Truncated downloads are the most common silent failure — partial FASTQs parse surprisingly far before erroring.
  • Checksum. ENA publishes MD5 per file; md5sum the file and compare. A mismatch means a corrupted transfer, not a strange sample.
  • Record count. seqkit stats or a quick wc -l (divided by four for FASTQ) against the run’s spot count. Then run FastQC immediately — checking quality at download time tells you whether any later anomaly came from the data or from your analysis.

Storage discipline

Raw reads are the most expensive part of a project to keep. A single human RNA-seq lane is tens of gigabytes; a public project can be terabytes. Three habits that pay off: download only the runs you will use (the metadata table lets you filter by library strategy, layout and size); compress and archive only what passes QC, deleting intermediates that can be re-derived; and record the accession list with MD5s alongside your results — the moment your analysis stops being trivially re-runnable, that manifest becomes the difference between reproducible science and a folder of unexplained files.

Where the data comes from in a real pipeline

For most analyses the sequence is: resolve study → fetch run table → filter runs → download FASTQ with checksums → QC → process. The public archives make step one through four free and scriptable; the skills around them are the same terminal, file-format and pipeline skills the rest of this blog builds — the Linux for bioinformatics guide covers the shell mechanics these commands assume.

Key takeaways

  • Papers cite studies (PRJNA, GSE); downloads use runs (SRR). Resolve one to the other before fetching bytes.
  • SRA Toolkit (prefetch + fasterq-dump --split-files) is canonical; ENA’s portal API is usually faster and needs no temp space, returning URLs, sizes and MD5s in one call.
  • GEO holds processed matrices and links to raw runs — check the matrix before downloading terabytes.
  • Verify size, checksum and record count before analysis; a truncated FASTQ looks like a biology problem later.
  • Human genomic data is often controlled-access (dbGaP, EGA): application required, not just a link.
  • Always cite the accession in publications, and respect the original consent and any data-use conditions.

Challenge

Take a GEO accession from a paper in your field and, without clicking through any download buttons, produce a manifest: every run it contains, layout, platform, size, and an MD5 for each file — using nothing but ENA’s API and shell tools. Then download one small run, verify its checksum, and run FastQC on it. That manifest-and-verify loop is exactly what a reproducible reanalysis starts with, and it takes minutes once the API call is muscle memory.

Want guided practice turning public data into a defensible analysis? Ampersand Academy offers one-to-one mentoring in bioinformatics — part of Mahadhi, a development and digital marketing company.

FAQ

What is the difference between SRA, GEO and ENA?

SRA stores raw sequencing runs; ENA mirrors the same runs in Europe with its own API and download servers; GEO is a curated layer holding study descriptions, processed matrices and links out to the raw runs. Reads downloaded from ENA and SRA are the same data, so choose by download speed and tooling convenience.

Why does fasterq-dump need so much temporary space?

It converts runs in parallel for speed, writing intermediate files that can approach the size of the output. Point it at a large scratch disk with the –temp option, or use the ENA API route instead, which streams compressed FASTQ directly and needs no conversion workspace.

Can I download only part of a large run?

Yes, but carefully. You can limit the number of reads during conversion, which is useful for testing a pipeline, but never analyse a subset as if it were the full run — record the restriction explicitly so results are not later mistaken for whole-dataset conclusions.

How do I know which run in a project I actually need?

Download the run table first — from SRA Run Selector or the ENA filereport API — and filter on library strategy, layout, platform, sample title and size. That table is the project’s index, and reading it before downloading prevents the classic mistake of pulling an unrelated replicate or the wrong assay type.

The GEO page has count matrices. Do I still need the raw FASTQ files?

Not for many purposes. Differential-expression reanalysis, pathway enrichment and figure re-creation usually work from the matrix plus sample metadata. Return to raw reads only when you need a different pipeline, different QC, or to reprocess with newer methods.

Do I need permission to use public sequencing data?

Open data can be downloaded and re-analysed freely, with citation of the accession. Human genomic data is often controlled-access, requiring an approved application through dbGaP or EGA. Always check the study’s data-use conditions, especially for clinical or identifiable cohorts.

Should the downloaded files be converted from SRA to FASTQ?

Almost always. Most tools expect FASTQ, and conversion preserves per-base quality scores. Some packages can read SRA objects directly, but FASTQ is the portable interchange format — the same reasoning as the formats covered in the file-format guide — so convert once and keep the original SRA object for reproducibility.

Last updated on · Written by