Site icon Ampersand Tutorials

Docker and Apptainer for Bioinformatics: Run Any Tool Without Breaking Your System

Glass sphere containing a DNA helix, representing a container holding bioinformatics software

You install a variant caller, it needs an older version of a C library than your system ships, and suddenly half your other tools stop working. Two years later you try to reproduce a figure from that project and the exact version is gone from every package channel. Containers solve both problems at once: they freeze a tool and everything it depends on into a single image that behaves identically on your laptop, your colleague’s Mac and a SLURM cluster.

This tutorial is a working guide rather than a container theory lecture. You will run real Biocontainers with Docker, move the same image onto a cluster with Apptainer, write a Dockerfile for a tool you use weekly, and — most importantly — recognise the four failures that account for nearly every container error in bioinformatics.

Why bioinformatics tools break when you move machines

Compiled bioinformatics software is unusually fragile for three reasons. First, most tools link against system libraries (zlib, libcurl, HTSlib, OpenSSL) and expect particular versions. Second, the field moves fast, so package channels delete or overwrite old builds while papers cite them for years. Third, most tools are distributed as source archives you compile yourself, which captures your compiler version, your CPU flags and your local library paths — a combination that is difficult to reproduce deliberately and impossible to reproduce by accident.

Conda fixes the library problem inside a single environment, which is why our conda guide is the right first step. It does not fix the operating system underneath: glibc, system Python, kernel features and system compilers are still shared with every other environment on the machine. Containers cover exactly that gap.

Docker and Apptainer: same idea, different constraints

Both tools consume OCI images. The difference is who you have to be in order to run one.

DockerApptainer (Singularity)
PrivilegesNeeds a daemon running as root (or rootless setup)Runs entirely as your own user
Typical homeLaptop, workstation, cloud VMShared HPC clusters, national compute
Image formatLayered, stored in /var/lib/dockerSingle read-only .sif file you can copy
Who you are insideroot by default, mapped to a fake UIDYou — your real UID and GID
Data accessExplicit -v bind mountsHome and current directory auto-mounted, plus –bind
Building imagesdocker build, no special rightsneeds fakeroot, sudo, or a remote build

That fourth row explains the most confusing error in this whole area. In Docker you are root, so files written into a mounted directory come out owned by root on the host, and your own account can no longer edit them. In Apptainer you keep your identity, so outputs belong to you — but the container also inherits your ability to read only what you can read.

Running a container for the first time

The bioinformatics community publishes thousands of prebuilt images through the Biocontainers project on Quay.io. You almost never need to build the common tools yourself. Image tags follow a strict convention: tool version, double dash, then the build hash.

# Find the current tag for a tool
# https://quay.io/repository/biocontainers/fastqc?tab=tags

docker pull quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0

# Run it against a file in the current directory
docker run --rm \
  -v "$PWD":/data \
  -w /data \
  quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0 \
  fastqc sample_R1.fastq.gz -o qc/

Three flags do all the work. -v "$PWD":/data makes your working directory visible inside the container at /data. -w /data sets that as the working directory so relative paths behave the way you expect. --rm deletes the container afterwards instead of leaving stopped containers behind on every run.

Notice that you passed fastqc explicitly. Biocontainer images usually set an entrypoint, but naming the executable makes your command self-documenting and lets you run bash inside the same image when something goes wrong.

Get a shell when the command fails

docker run --rm -it \
  -v "$PWD":/data -w /data \
  --entrypoint bash \
  quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0

Inside that shell you can check which fastqc, look at the installed libraries, and test the exact command the pipeline was running. Ninety percent of mystery container errors resolve in this shell within a minute.

Mounting data and the read-only habit

A container cannot see your files unless you mount them. A good habit is to mount reference data read-only, so a typo in a tool invocation cannot overwrite a 3 GB genome index that took an hour to build.

docker run --rm \
  -v "$PWD":/work \
  -v /data/references:/refs:ro \
  -w /work \
  quay.io/biocontainers/bwa-mem2:2.2.1--he4d6f0e_5 \
  bwa-mem2 mem /refs/GRCh38.fa reads_R1.fq.gz reads_R2.fq.gz > aln.sam

Every mount is a hole in the container’s isolation. Mount the directory containing your inputs, not your home directory; on a cluster, mounting $HOME exposes every credential and dataset you own to a program you did not audit.

Writing a Dockerfile for a tool you use every week

You write your own image in two situations: the tool has no Biocontainer, or you need several tools plus a script in one environment because they share an intermediate file format. Keep the image small and pin every version.

FROM mambaorg/micromamba:1.5.10

# Pin versions; a floating tag makes the image a different tool next month.
RUN micromamba install -y -n base -c conda-forge -c bioconda \
      samtools=1.21 \
      bcftools=1.21 \
      htslib=1.21 \
 && micromamba clean --all --yes

USER $MAMBA_USER
WORKDIR /work
ENTRYPOINT ["/usr/local/bin/_entrypoint.sh"]
CMD ["bcftools", "--version"]

Two details matter more than they look. Pinning explicit versions means the same Dockerfile produces the same image a year from now. Using a conda base image instead of hand-compiling means you get security patches and correct library paths for free — the classic ubuntu:latest plus apt-get install pattern is the slowest way to build a bioinformatics image and the most likely to break.

docker build --platform linux/amd64 -t mylab/varcall:1.0 .
docker image inspect mylab/varcall:1.0 --format '{{index .RepoDigests 0}}'
# mylab/varcall@sha256:9f2c...  <- record this digest in your methods section

The digest is the only stable identifier of an image. Tags are mutable; a maintainer can repoint :1.0 at a rebuilt image tomorrow. Cite the digest.

Building an Apptainer image for a cluster

On a cluster you usually do not build at all — you convert. Apptainer can pull a Docker image directly and flatten it into a single .sif file, which is a normal file you can copy, archive, and hand to a collaborator.

# On a machine with network access (login node or your laptop)
apptainer pull fastqc.sif docker://quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0

# Run it; note the different flags
apptainer exec \
  --bind "$PWD":/data \
  --pwd /data \
  fastqc.sif fastqc sample_R1.fastq.gz -o qc/

The verbs map cleanly: docker run becomes apptainer exec, -v host:container becomes --bind host:container, and -w becomes --pwd. There is no daemon and no pull-on-run, so the image must already exist on disk — which is a feature, because it means a running job never depends on a registry staying up.

Make the container hermetic

By default Apptainer mounts your home directory and the current directory. That is convenient interactively and dangerous in a batch job, because it silently reintroduces the host’s ~/.config, user site-packages and locale files into what you thought was a fixed environment.

apptainer exec --containall --no-home \
  --bind /scratch/$USER/run:/data \
  --pwd /data \
  --env OMP_NUM_THREADS=8 \
  fastqc.sif fastqc sample_R1.fastq.gz

--containall gives the container its own private /tmp and empty home, and --no-home stops the home mount entirely. Add --env for anything the tool reads from the environment; do not rely on the ambient shell, because the batch scheduler’s environment is not yours.

If you genuinely need to build rather than convert, use apptainer build --fakeroot from a definition file, or build on a machine where you have root and copy the .sif. Most clusters now support fakeroot, which is the cleaner route.

The failures you will actually hit

1. Output files owned by root

You ran Docker, and now aln.sam is owned by root and rm refuses. This is not a bug: the container’s root is your user mapped through a namespace.

# Fix one file
docker run --rm -v "$PWD":/data alpine chown -R $(id -u):$(id -g) /data

# Avoid it: run as your own UID
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD":/data -w /data image tool

2. exec format error on Apple Silicon

Nearly every bioinformatics image is built for linux/amd64. On an M-series Mac it either refuses to start or runs under emulation at a fraction of the speed. Pin the platform explicitly and expect slow runs; do the heavy work on a Linux machine or in the cloud.

docker run --platform linux/amd64 --rm -v "$PWD":/d -w /d image tool

3. Permission denied reading your own data

Apptainer keeps your UID, so a file readable only by another user stays unreadable inside the container. If a collaborator’s dataset is group-readable but not world-readable, either ask for a copy with workable permissions or run the job as that user. Do not solve this with chmod 777.

4. The mount exists but the tool writes nothing

A relative mount path that resolved differently under the scheduler, or a tool that writes to its own install directory (read-only in both Docker and Apptainer) will print success and produce no output. Always --pwd into a real, writable bind, and check the exit code rather than the log tail.

What to hand over with the analysis

A container is only reproducible if the next person can rebuild your exact image. Hand over the Dockerfile or definition file and the resolved digest, plus the exact docker run or apptainer exec line from your pipeline log. That triple — recipe, digest, invocation — is enough to rerun the analysis years later without you.

This is the same principle behind every method we teach: an analysis is a claim that someone else can check. If you want the wider picture of how containers sit alongside conda, workflow managers and cluster schedulers in a real project, Ampersand Academy covers the full pipeline from raw reads to published figure.

Practice

  1. Pull the Biocontainers samtools image and print its version without installing samtools locally.
  2. Convert that same image to .sif and run flagstat on a small BAM with --containall.
  3. Write a Dockerfile that installs both fastp and multiqc at pinned versions and produces a working fastp --version.
  4. Deliberately break it: mount your reference directory read-only and try to write into it, then read the error carefully.

Is Docker or Apptainer the better choice for bioinformatics?

Use Docker on a machine you control and Apptainer on shared clusters. The image content is identical; the difference is privilege. Docker needs a root daemon, which most HPC centres will not grant, while Apptainer runs unprivileged and produces a single portable file.

Do containers replace conda environments?

No — they solve different layers. Conda pins package versions inside your user space; containers additionally pin the operating system libraries and toolchain. For exploratory analysis conda is faster to work with. For anything that must run identically on another machine or in three years, use a container.

How do I find the right Biocontainers tag?

Search the tool on quay.io under the biocontainers namespace and read the tag list. Tags are tool-version plus a build hash, for example 0.12.1–hdfd78af_0. Pick the newest tag matching the version you need and record the full tag plus the image digest in your notes.

Why are my output files owned by root?

Docker runs as root inside the container by default, and files written to a bind mount take that identity on the host. Run with –user "$(id -u):$(id -g)" to match your host account, or use Apptainer, which preserves your real UID automatically.

Can I run Docker on a SLURM cluster?

Usually not — Docker requires a daemon with root privileges, which most centres forbid. Check whether the cluster offers Apptainer or Singularity, which is the standard there. If neither exists, ask whether rootless Docker or Podman is available before assuming you cannot use containers.

How large are bioinformatics container images?

Between roughly 300 MB for a single small tool and several GB for anything bundling a reference or a Python scientific stack. Keep full reference genomes out of images: mount them read-only at run time so the image stays portable and versionable.

What is the difference between apptainer exec and apptainer run?

exec runs a command you specify inside the container, which is what you want in pipelines. run uses the image’s predefined runscript or entrypoint, which is convenient for interactive tools and ambiguous in automation. Prefer exec so the command you ran is visible in the log.

Exit mobile version