Profiles¶

A profile is a bundle of pre-set parameters and resource ceilings, selected with Nextflow's -profile flag. TaxTriage ships three kinds, and a real run normally chains one of each:

nextflow run https://github.com/jhuapl-bio/taxtriage \
  -r stable -latest \
  -profile desktop,docker \
  --input samplesheet.csv --outdir results

Workload profiles at a glance¶

The ceilings above are resourceLimits a cap, not a request. A process asking for 4 CPUs still gets 4 on the deep profile; the cap only stops a process from asking for more than the machine has.

Choosing one¶

What deep does differently¶

deep is the only profile that reduces read volume before classification, because at multi-GB scale the classifier and aligner are otherwise intractable:

1. bbnorm (--downsample, on) k-mer coverage normalisation, Illumina only. Runs after trimming, before host removal. Flattens the high-abundance background while leaving rare organisms intact. bbnorm_mindepth is set to 2 rather than bbnorm's own default of 6, which would delete exactly the low-abundance organisms triage is looking for.
2. seqtk (--subsample, 100 M reads) a blunt random cap, all platforms. This is the only reduction ONT/PacBio gets, since bbnorm's k-mer table is meaningless on noisy long reads.

deep also loads conf/deep_process.config, which sets per-process memory ladders that retry upward after an OOM (Kraken2 120 → 240 GB, bbnorm 96 → 192 GB, minimap2 64 → 160 GB). Those directives live in a separate file on purpose withName: blocks written inside a profile are silently overwritten by conf/modules.config.

Resource and database limits read before you launch¶

Database footprints. Sizes below are approximate and grow with every upstream release; check benlangmead.github.io/aws-indexes/k2 for the current figures, and see the downloadable database table.

The capped _8gb / _16gb builds are not smaller downloads of the same data - they are lossy, minimised indexes. They trade sensitivity and species-level resolution for fitting on the machine. A species missing from a standard8 run is not evidence of absence.

Disk, not just memory. Plan for roughly 3× the database tarball during setup (download + extraction + the extracted copy), plus the Nextflow work/ directory, which on a deep run routinely exceeds the size of the input FASTQ several times over. cleanup is deliberately left false so -resume works; run nextflow clean -f once you are done inspecting a completed run.

Downloads happen once, if you let them. --download_db caches into a per-name storeDir, so the multi-GB pull is skipped on later runs. Pointing --outdir somewhere new each time, or running on ephemeral cloud storage, re-downloads everything. On a shared or offline system, download once and pass --db /path/to/db instead.

Other things that will bite you:

• --low_memory is the escape hatch, and it is slow. It streams the index from disk instead of loading it, which can turn minutes into hours but on a fast NVMe it is usually better than not running at all. low sets it; viral, desktop, local and deep do not.
• Minimap2 has its own ceiling. The reference index is separate from the Kraken2 database, and a high --top_hits_count (the deep profile pulls 50) can push it past a single index budget. deep sets split_prefix = true with --mmap2_I 8G for this reason. The pipeline also backs memory off automatically across retries. See Troubleshooting.
• Ceilings are caps, not reservations. A profile does not check that your machine actually has what it claims. -profile deep on a 32 GB box will happily submit a job requesting 120 GB and get it killed. Match the profile to the hardware, or override with --max_cpus / --max_memory / --max_time.
• On a cluster, the profile is not the whole story. Your scheduler's per-job memory and wall-time limits apply on top. A 72-hour deep run on a partition with a 24-hour limit dies at hour 24.
• Test profiles are not starting points for real work. test, test_viral and (despite the name in its own header) low and viral set input to a bundled example samplesheet. Always pass your own --input, and drop test entirely once you are past verification.

Overriding a profile¶

Anything a profile sets can be overridden on the command line the CLI wins:

# desktop ceilings, but a bigger database and more headroom
nextflow run https://github.com/jhuapl-bio/taxtriage \
  -r stable -latest \
  -profile desktop,docker \
  --db standard --max_memory 96.GB \
  --input samplesheet.csv --outdir results

# deep, but preserving relative abundance
nextflow run https://github.com/jhuapl-bio/taxtriage \
  -r stable -latest \
  -profile deep,docker \
  --downsample false --subsample 50000000 \
  --input samplesheet.csv --outdir results

For anything you will reuse, put the overrides in your own -c custom.config rather than in the command. See Running the Pipeline and CLI Parameters.