Setting the file. One moment.
Skill 02 · Nextflow Development
Subchapter 2.1
references/geo-sra-acquisition.mdMarkdown14 KBView on GitHub
Download raw sequencing data from NCBI GEO/SRA and prepare it for nf-core pipelines.
Use this when: Reanalyzing published datasets, validating findings, or comparing results against public cohorts.
Example: “Find differentially expressed genes in GSE309891 (drug-treated vs control)”
┌─────────────────────────────────────────────────────────────────┐
│ GEO/SRA DATA ACQUISITION │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────┐
│ Fetch study info │
│ • Query NCBI/SRA │
│ • Get metadata │
│ • Detect organism │
│ • Identify data type │
└────────────────────────┘
│
▼
┌────────────────────────┐
│ Present summary │
│ • Organism: Human │
│ • Genome: GRCh38 │
│ • Type: RNA-Seq │
│ • Pipeline: rnaseq │
│ • Samples: 12 │
│ (6 treated, │
│ 6 control) │
│ • Size: ~24 GB │
└────────────────────────┘
│
▼
┌─────────────────┐
│ USER CONFIRMS │◄──── Decision point
│ genome/pipeline│
└─────────────────┘
│
▼
┌────────────────────────┐
│ Select samples │
│ • Group by condition │
│ • Show treated/ctrl │
└────────────────────────┘
│
▼
┌─────────────────┐
│ USER SELECTS │◄──── Decision point
│ sample subset │
└─────────────────┘
│
▼
┌────────────────────────┐
│ Download FASTQs │
│ • 24 files (R1+R2) │
│ • Parallel transfers │
│ • Auto-resume │
└────────────────────────┘
│
▼
┌────────────────────────┐
│ Generate samplesheet │
│ • Map SRR to files │
│ • Pair R1/R2 │
│ • Assign conditions │
└────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ NF-CORE PIPELINE EXECUTION │
│ (Continue with Step 1 of main workflow) │
└─────────────────────────────────────────────────────────────────┘When assisting users with GEO/SRA data acquisition:
Example confirmation question:
Question: "Which sample group would you like to download?"
Options:
- "RNA-Seq:PAIRED (42 samples, ~87 GB)"
- "RNA-Seq:SINGLE (7 samples, ~4.5 GB)"
- "All samples (49 samples, ~92 GB)"Get metadata about a GEO study before downloading.
python scripts/sra_geo_fetch.py info <GEO_ID>Example:
python scripts/sra_geo_fetch.py info GSE110004Output includes:
Save info to JSON:
python scripts/sra_geo_fetch.py info GSE110004 -o study_info.jsonView sample groups organized by data type and layout. This is useful for studies with mixed data types.
python scripts/sra_geo_fetch.py groups <GEO_ID>Example output:
Sample Group Count Layout GSM Range Est. Size
--------------------------------------------------------------------------------
RNA-Seq 42 PAIRED GSM2879618...(42 samples) 87.4 GB
RNA-Seq 7 SINGLE GSM2976181-GSM2976187 4.5 GB
--------------------------------------------------------------------------------
TOTAL 49 91.9 GB
Available groups for --subset option:
1. "RNA-Seq:PAIRED" - 42 samples (~87.4 GB)
2. "RNA-Seq:SINGLE" - 7 samples (~4.5 GB)List individual runs:
python scripts/sra_geo_fetch.py list <GEO_ID>
# Filter by data type
python scripts/sra_geo_fetch.py list GSE110004 --filter "RNA-Seq:PAIRED"DECISION POINT: Review the sample groups. Decide which subset to download if the study has multiple data types.
Download FASTQ files from ENA (faster than SRA).
python scripts/sra_geo_fetch.py download <GEO_ID> -o <OUTPUT_DIR>Options:
-o, --output: Output directory (required)-i, --interactive: Interactively select sample group to download-s, --subset: Filter by data type (e.g., “RNA-Seq:PAIRED”)-p, --parallel: Parallel downloads (default: 4)-t, --timeout: Download timeout in seconds (default: 600)Use -i flag for interactive sample selection when the study has multiple data types:
python scripts/sra_geo_fetch.py download GSE110004 -o ./fastq -iInteractive output:
============================================================
SELECT SAMPLE GROUP TO DOWNLOAD
============================================================
[1] RNA-Seq (paired)
Samples: 42
GSM: GSM2879618...(42 samples)
Size: ~87.4 GB
[2] RNA-Seq (single)
Samples: 7
GSM: GSM2976181-GSM2976187
Size: ~4.5 GB
[0] Download ALL (49 samples)
------------------------------------------------------------
Enter selection (0-2):Alternatively, specify the subset directly:
# Download only RNA-Seq paired-end data
python scripts/sra_geo_fetch.py download GSE110004 -o ./fastq \
--subset "RNA-Seq:PAIRED" --parallel 6Note: Downloads automatically skip existing files. Resume interrupted downloads by re-running the command.
Create a samplesheet compatible with nf-core pipelines.
python scripts/sra_geo_fetch.py samplesheet <GEO_ID> \
--fastq-dir <FASTQ_DIR> \
-o samplesheet.csvOptions:
-f, --fastq-dir: Directory containing downloaded FASTQ files (required)-o, --output: Output samplesheet path (default: samplesheet.csv)-p, --pipeline: Target pipeline (auto-detected if not specified)Example:
python scripts/sra_geo_fetch.py samplesheet GSE110004 \
--fastq-dir ./fastq \
-o samplesheet.csvOutput: The script will:
After generating the samplesheet, the script provides a suggested command.
Example output:
Suggested command:
nextflow run nf-core/rnaseq \
--input samplesheet.csv \
--outdir results \
--genome R64-1-1 \
-profile dockerDECISION POINT: Review and confirm:
Then return to the main SKILL.md workflow (Step 1: Environment Check) to proceed with pipeline execution.
The skill auto-detects appropriate pipelines based on library strategy. Pipelines marked with ★ are fully supported with configs, samplesheet generation, and documentation. Others are suggested but require manual setup following nf-core documentation.
| Library Strategy | Suggested Pipeline | Support |
|---|---|---|
| RNA-Seq | nf-core/rnaseq | ★ Full |
| ATAC-seq | nf-core/atacseq | ★ Full |
| WGS/WXS | nf-core/sarek | ★ Full |
| ChIP-seq | nf-core/chipseq | Manual |
| Bisulfite-Seq | nf-core/methylseq | Manual |
| miRNA-Seq | nf-core/smrnaseq | Manual |
| Amplicon | nf-core/ampliseq | Manual |
Common organisms with auto-suggested genomes:
| Organism | Genome | Notes |
|---|---|---|
| Homo sapiens | GRCh38 | Human reference |
| Mus musculus | GRCm39 | Mouse reference |
| Saccharomyces cerevisiae | R64-1-1 | Yeast S288C |
| Drosophila melanogaster | BDGP6 | Fruit fly |
| Caenorhabditis elegans | WBcel235 | C. elegans |
| Danio rerio | GRCz11 | Zebrafish |
| Arabidopsis thaliana | TAIR10 | Arabidopsis |
| Rattus norvegicus | Rnor_6.0 | Rat |
See scripts/config/genomes.yaml for the full list.
Reanalyze GSE110004 (yeast RNA-seq):
# 1. Get study info and sample groups
python scripts/sra_geo_fetch.py info GSE110004
# 2. Download with interactive selection
python scripts/sra_geo_fetch.py download GSE110004 -o ./fastq -i
# Select option [1] for RNA-Seq paired-end samples
# 3. Generate samplesheet
python scripts/sra_geo_fetch.py samplesheet GSE110004 \
--fastq-dir ./fastq \
-o samplesheet.csv
# 4. Run nf-core/rnaseq (continue with main SKILL.md workflow)
nextflow run nf-core/rnaseq \
--input samplesheet.csv \
--outdir results \
--genome R64-1-1 \
-profile docker# Review sample groups first
python scripts/sra_geo_fetch.py groups GSE110004
# Download specific subset directly
python scripts/sra_geo_fetch.py download GSE110004 \
--subset "RNA-Seq:PAIRED" \
-o ./fastq \
--parallel 4If ENA downloads fail, the data may need to be fetched directly from SRA:
# Create SRA tools environment
conda create -n sra_tools -c bioconda sra-tools
# Download with prefetch + fasterq-dump
conda run -n sra_tools prefetch SRR6357070
conda run -n sra_tools fasterq-dump SRR6357070 -O ./fastqSome GEO datasets only have processed data, not raw sequencing reads. Check:
python scripts/sra_geo_fetch.py info <GEO_ID>If “Runs: 0”, the dataset may not have raw data in SRA.
GEO SuperSeries (which contain multiple SubSeries) are automatically handled. The tool will:
Example: GSE110004 is a SuperSeries that links to BioProject PRJNA432544.
If the organism is not in the genome mapping, manually specify the genome:
# Check available iGenomes
python scripts/manage_genomes.py list
# Or provide custom reference files to nf-core
nextflow run nf-core/rnaseq --fasta /path/to/genome.fa --gtf /path/to/genes.gtfrequests library (optional but recommended)pyyaml library (optional, for genome config)Install optional dependencies:
pip install requests pyyaml