Category: Docker Compose

  • Docker Compose Logging: Complete Setup & Best Practices

    Docker Compose Logging: How to Configure, Rotate, and Centralize Container Logs

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever run docker compose up on a production host and come back three weeks later to find your disk full, you already know why docker compose logging deserves more attention than it usually gets. By default, Docker happily writes every line your containers print to stdout/stderr into JSON files on disk — forever, unless you tell it otherwise.

    This guide covers how docker compose logging actually works under the hood, how to configure logging drivers and rotation per-service, and how to ship logs somewhere useful instead of leaving them scattered across container filesystems.

    Why Default Docker Compose Logging Is a Problem

    Out of the box, Docker uses the json-file logging driver with no size limits and no rotation. Every console.log, every stack trace, every access log line gets appended to a file under /var/lib/docker/containers/<id>/<id>-json.log. On a busy API service or a chatty streaming transcoder, that file can grow into gigabytes within days.

    The problems compound in a typical docker-compose.yml setup because:

  • There’s no rotation by default, so logs grow unbounded.
  • Every service inherits the same behavior unless explicitly overridden.
  • docker compose logs reads the entire backing file, which gets slower as it grows.
  • Multiple services logging heavily can fill root partitions on small VPS instances.
  • If you’re running Compose stacks on a budget VPS — say, a DigitalOcean droplet with a 25GB disk — an unrotated log file can take your whole box down. That’s the failure mode we’re trying to prevent here.

    How Docker Compose Logging Drivers Work

    Docker Compose doesn’t have its own logging system — it passes configuration straight through to the Docker Engine’s logging drivers. The driver determines where log output goes and how it’s formatted. Common options include:

  • json-file — the default; writes structured JSON to disk, supports rotation
  • local — a more efficient binary format, rotates by default, recommended over json-file for most cases
  • syslog — forwards to a syslog daemon (local or remote)
  • journald — forwards to systemd’s journal on Linux hosts
  • fluentd — ships logs to a Fluentd collector for centralized aggregation
  • none — disables logging entirely for that container
  • You set the driver per-service under the logging key in your docker-compose.yml. Here’s a minimal example using the local driver, which is a good default for most self-hosted setups:

    services:
      web:
        image: nginx:latest
        logging:
          driver: local
          options:
            max-size: "10m"
            max-file: "3"

    This caps each log file at 10MB and keeps a maximum of 3 rotated files, so the total footprint for that service tops out around 30MB regardless of how noisy the container gets.

    Configuring Log Rotation for Every Service

    Repeating the same logging block in every service is tedious and error-prone. Compose supports YAML anchors to define the config once and reuse it:

    x-logging: &default-logging
      driver: local
      options:
        max-size: "10m"
        max-file: "5"
    
    services:
      api:
        image: myapp/api:latest
        logging: *default-logging
    
      worker:
        image: myapp/worker:latest
        logging: *default-logging
    
      redis:
        image: redis:7-alpine
        logging: *default-logging

    This pattern keeps your docker-compose.yml DRY and guarantees no service slips through without rotation. If you’re managing multiple stacks, pair this with the strategies in our Docker Compose networking guide so both your networking and logging conventions stay consistent across projects.

    If you’d rather set rotation globally instead of per-project, edit /etc/docker/daemon.json on the host:

    {
      "log-driver": "local",
      "log-opts": {
        "max-size": "10m",
        "max-file": "3"
      }
    }

    Restart Docker afterward with sudo systemctl restart docker. Note that this only applies to newly created containers — existing ones keep their original logging config until recreated.

    Reading and Filtering Logs with the Compose CLI

    Once logging is configured, docker compose logs is your primary tool for day-to-day debugging. A few flags make it dramatically more useful than the bare command:

    # Follow logs in real time for all services
    docker compose logs -f
    
    # Follow logs for a single service only
    docker compose logs -f api
    
    # Show only the last 100 lines
    docker compose logs --tail=100 api
    
    # Include timestamps
    docker compose logs -f -t api
    
    # Show logs since a specific time
    docker compose logs --since="2026-07-09T00:00:00" api

    For quick filtering, pipe through grep:

    docker compose logs api | grep -i error

    This works fine for a single host, but it doesn’t scale once you have logs spread across multiple containers, multiple hosts, or a Swarm/Kubernetes migration down the line. That’s where centralized logging comes in.

    Centralizing Logs with Fluentd, Loki, or a Managed Service

    For anything beyond a single small VPS, shipping logs off-host is worth the setup time. Two common self-hosted approaches:

    Option 1: Fluentd driver

    services:
      app:
        image: myapp:latest
        logging:
          driver: fluentd
          options:
            fluentd-address: localhost:24224
            tag: myapp.{{.Name}}

    This requires a Fluentd container or daemon listening on the given address, which then routes logs to Elasticsearch, S3, or another backend.

    Option 2: Grafana Loki

    Loki pairs naturally with Compose stacks that already expose Prometheus metrics. Install the Loki Docker driver plugin, then configure services to use it:

    services:
      app:
        image: myapp:latest
        logging:
          driver: loki
          options:
            loki-url: "http://localhost:3100/loki/api/v1/push"
            loki-retries: "3"
            loki-batch-size: "400"

    If self-hosting a full logging stack feels like overkill for your team size, a managed log management service removes the operational burden entirely. BetterStack offers log collection, alerting, and uptime monitoring in one dashboard, with a straightforward Docker/Compose integration — worth considering if you’d rather not run and patch your own Loki or Elasticsearch cluster. It plugs in well alongside the container health checks discussed in our guide to monitoring Docker containers.

    Debugging Common Docker Compose Logging Issues

    A few issues come up repeatedly when teams first configure logging:

  • “No such file or directory” on log driver plugins — install the driver plugin first with docker plugin install grafana/loki-docker-driver:latest --alias loki, then restart Docker.
  • Logs missing after switching drivers — drivers like syslog, fluentd, and loki don’t support docker compose logs for historical output; you must query the downstream system instead.
  • Container won’t start after adding logging config — validate your YAML indentation; a misplaced options key under logging is a common typo.
  • Rotation isn’t happening — confirm you’re using local or json-file with max-size set; other drivers handle retention differently or not at all.
  • Disk still filling up — check for orphaned volumes or bind-mounted log directories that bypass the Docker logging driver entirely, common with apps that write directly to /var/log inside the container.
  • For reference, the official Docker logging documentation lists every driver and its supported options in detail — worth bookmarking when you hit a driver-specific edge case.

    Choosing a Logging Strategy Based on Scale

  • Single small VPS, low traffic: local driver with max-size: 10m and max-file: 3 is sufficient.
  • Single busy server, multiple services: Use the YAML anchor pattern above, and consider journald if you already use journalctl for other host services.
  • Multi-host or team environment: Ship to Loki, Elasticsearch, or a managed provider like BetterStack so logs survive host failures and are searchable across services.
  • Compliance-sensitive workloads: Ensure retention policy and access controls exist at the aggregation layer, not just on individual containers.
  • Picking the right tier upfront saves you from a painful migration later — moving from unrotated json-file logs to a centralized system after a disk-full incident is a bad way to learn this lesson.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: Does docker compose logging affect performance?
    A: Minimally for local or json-file drivers under normal load. High-throughput logging (thousands of lines per second) can add measurable overhead, especially with network-based drivers like syslog or fluentd if the remote endpoint is slow to acknowledge writes.

    Q: Can I set different logging configs for different services in the same file?
    A: Yes. The logging key is defined per-service, so you can mix drivers — for example, local for a database and loki for a public-facing API that you want centrally searchable.

    Q: How do I clear existing log files without deleting containers?
    A: You generally can’t safely truncate the backing JSON file directly while a container is running. Instead, add rotation (max-size/max-file) and recreate the container with docker compose up -d --force-recreate, or use docker system prune cautiously for stopped containers.

    Q: Why doesn’t docker compose logs show anything after I switched to the syslog driver?
    A: docker compose logs reads from Docker’s internal log storage, which drivers like syslog, fluentd, and journald bypass. You need to query the destination system (syslog server, Fluentd backend, or journalctl) instead.

    Q: What’s the difference between json-file and local drivers?
    A: local uses a more compact binary format and enables rotation by default (with sane defaults even if you don’t set options), while json-file requires you to explicitly configure max-size and max-file or it will grow unbounded.

    Q: Should I log to stdout or write to a file inside the container?
    A: Log to stdout/stderr. Docker’s logging drivers are built around capturing standard streams; writing to files inside the container bypasses rotation, driver forwarding, and docker compose logs entirely, defeating the purpose of Compose’s logging config.

    Getting docker compose logging right isn’t complicated, but it’s easy to skip until it causes an outage. Set rotation limits from day one, pick a driver that matches your scale, and centralize logs before you actually need to search across five containers at 2 a.m.

  • Kubernetes vs Docker Compose: Which Should You Use?

    Kubernetes vs Docker Compose: Which One Actually Fits Your Workload?

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’re running containers past the “it works on my laptop” stage, you’ll eventually hit this fork in the road: stick with Docker Compose or move to Kubernetes. Both orchestrate containers, but they solve fundamentally different problems, and picking the wrong one wastes weeks of engineering time.

    This guide breaks down what each tool actually does, where they overlap, and how to decide without falling for hype. If you’re still setting up your first multi-container app, check out our Docker Compose tutorial for beginners before diving into orchestration decisions.

    What Is Docker Compose?

    Docker Compose is a tool for defining and running multi-container Docker applications on a single host. You describe your services, networks, and volumes in a docker-compose.yml file, and Compose handles building images, starting containers in the right order, and wiring up networking between them.

    A typical Compose file for a web app with a database looks like this:

    version: "3.9"
    services:
      web:
        build: .
        ports:
          - "8080:8080"
        environment:
          - DATABASE_URL=postgres://user:pass@db:5432/appdb
        depends_on:
          - db
      db:
        image: postgres:16-alpine
        volumes:
          - db_data:/var/lib/postgresql/data
        environment:
          - POSTGRES_PASSWORD=pass
    
    volumes:
      db_data:

    Run it with a single command:

    docker compose up -d

    That’s the entire appeal. No control plane, no cluster nodes, no separate learning curve for scheduling and networking abstractions. It’s docker run, just organized and repeatable.

    How Docker Compose Works Under the Hood

    Compose talks directly to the Docker Engine API on the host it’s running on. It creates a dedicated bridge network for your project, resolves service names as DNS hostnames inside that network, and manages container lifecycle based on the dependency graph in your YAML file. There’s no scheduler deciding where containers run — everything runs on the one machine you invoked docker compose from. That’s both its biggest strength (simplicity) and its hard ceiling (no horizontal scaling across hosts).

    What Is Kubernetes?

    Kubernetes (K8s) is a container orchestration platform designed to manage containerized workloads across a cluster of machines. Instead of one host, you have a control plane and worker nodes, and Kubernetes decides where your containers (grouped into Pods) actually run, restarts them on failure, scales them based on load, and handles rolling updates with zero downtime.

    The equivalent of the Compose file above, expressed as a Kubernetes Deployment and Service, looks like this:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: web
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: web
      template:
        metadata:
          labels:
            app: web
        spec:
          containers:
            - name: web
              image: myregistry/web:latest
              ports:
                - containerPort: 8080
              env:
                - name: DATABASE_URL
                  valueFrom:
                    secretKeyRef:
                      name: db-secret
                      key: url
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: web
    spec:
      selector:
        app: web
      ports:
        - port: 80
          targetPort: 8080
      type: LoadBalancer

    Apply it with:

    kubectl apply -f deployment.yaml

    Notice the immediate difference: replicas: 3 means Kubernetes will keep three copies of your app running across whatever nodes are available, automatically rescheduling them if a node dies. Compose has no concept of this — it’s single-host by design.

    Core Kubernetes Concepts You Need to Know

    Before comparing the two tools further, it helps to understand the building blocks Kubernetes introduces that Compose simply doesn’t have:

  • Pods — the smallest deployable unit, usually one container (sometimes a tightly coupled group)
  • Deployments — manage replica counts, rolling updates, and rollbacks
  • Services — stable networking endpoints that load-balance across Pods
  • ConfigMaps and Secrets — externalized configuration and sensitive data
  • Ingress controllers — route external HTTP/HTTPS traffic into the cluster
  • Namespaces — logical isolation for multi-tenant or multi-environment clusters
  • Each of these adds power, but also adds YAML, and adds concepts your team has to learn before shipping anything. The official Kubernetes documentation is the best source of truth once you start going deeper than this article.

    Kubernetes vs Docker Compose: Key Differences

    | Factor | Docker Compose | Kubernetes |
    |—|—|—|
    | Scope | Single host | Multi-node cluster |
    | Scaling | Manual, host-limited | Automatic, horizontal |
    | Self-healing | None (restart policies only) | Built-in, continuous |
    | Learning curve | Low | Steep |
    | Networking | Simple bridge network | CNI plugins, Services, Ingress |
    | Rolling updates | Manual | Native support |
    | Best for | Dev, small production apps | Production at scale, multi-service systems |

    The honest takeaway: Compose is a development and small-deployment tool. Kubernetes is an operations platform for running containers reliably at scale across many machines. They’re not really competitors — they solve different-sized problems.

    When to Use Docker Compose

    Stick with Compose when:

  • You’re running on a single VPS or dedicated server
  • Your traffic doesn’t require horizontal scaling across multiple machines
  • Your team is small and doesn’t have dedicated DevOps/SRE capacity
  • You want fast local development environments that mirror production
  • Downtime during deploys is acceptable or handled another way (e.g., a reverse proxy with health checks)
  • A huge number of production SaaS apps, internal tools, and even small streaming or media servers run perfectly well on a single well-provisioned VPS with Compose managing everything. If you’re in this bucket, don’t let Kubernetes hype talk you into unnecessary complexity — our guide on choosing the best VPS for Docker workloads covers sizing a single host properly.

    When to Use Kubernetes

    Reach for Kubernetes when:

  • You need to scale services independently across many nodes
  • Uptime requirements demand automatic failover and self-healing
  • You’re running dozens of microservices with complex networking needs
  • You need zero-downtime rolling deployments as a hard requirement
  • You already have (or are building) a dedicated platform/DevOps team
  • Kubernetes shines when the operational complexity it introduces is smaller than the complexity of the problem it solves. If your team is manually SSHing into three servers to restart a crashed container at 2 AM, that’s a signal you’ve outgrown Compose.

    Migrating from Docker Compose to Kubernetes

    You don’t have to rewrite everything by hand. The Kompose tool converts existing docker-compose.yml files into a first draft of Kubernetes manifests:

    kompose convert -f docker-compose.yml

    This generates Deployment, Service, and PersistentVolumeClaim YAML files as a starting point. You’ll still need to manually tune resource limits, health checks (livenessProbe/readinessProbe), and secrets management — Kompose gets you 60-70% of the way, not 100%.

    A common migration path looks like this:

    # 1. Convert existing compose file
    kompose convert -f docker-compose.yml -o k8s-manifests/
    
    # 2. Review and add resource requests/limits
    kubectl apply -f k8s-manifests/ --dry-run=client
    
    # 3. Apply to a staging namespace first
    kubectl apply -f k8s-manifests/ -n staging

    Test thoroughly in staging before touching production — networking assumptions that worked fine on a single Compose host (like hardcoded service names) often need adjustment for Kubernetes DNS and Service discovery.

    Common Migration Pitfalls

    Teams moving from Compose to Kubernetes tend to hit the same issues repeatedly:

  • No resource limits defined — Kubernetes needs explicit CPU/memory requests and limits, or the scheduler makes poor placement decisions
  • Missing health checks — Compose’s depends_on doesn’t translate to readiness; you need real livenessProbe and readinessProbe configs
  • Stateful services treated like stateless ones — databases need StatefulSets and PersistentVolumeClaims, not plain Deployments
  • Underestimating the ops burden — someone has to patch, upgrade, and monitor the cluster itself, not just the apps running on it
  • Hosting Considerations for Either Approach

    Whichever route you pick, the underlying infrastructure matters. For Docker Compose setups, a single well-sized droplet or VPS is usually enough — DigitalOcean offers straightforward Droplets that work well for Compose-based deployments, and their managed load balancers can front a single-host setup nicely as traffic grows.

    For Kubernetes, you generally want either a managed offering or predictable dedicated hardware to keep node costs sane at scale. Hetzner is a popular choice for cost-conscious teams running self-managed Kubernetes clusters, since their dedicated and cloud servers offer strong price-to-performance ratios compared to the major hyperscalers.

    Regardless of platform, once you’re running production workloads you’ll want real uptime monitoring rather than guessing — a service like BetterStack can alert you the moment a Pod or container starts failing health checks, which matters a lot more once you’ve got a multi-node cluster to keep an eye on.

    Performance and Operational Overhead

    Compose has near-zero orchestration overhead — it’s just talking to the local Docker daemon. Kubernetes runs a control plane (API server, scheduler, controller manager, etcd) that consumes real CPU and memory even before your workloads start. On a single small VPS, running full Kubernetes (even lightweight distributions like k3s) can eat a noticeable chunk of your available resources compared to Compose.

    That overhead buys you things Compose can’t offer: automatic node failover, horizontal pod autoscaling, and rolling deployments without downtime. Whether that trade is worth it depends entirely on your scale. For teams running fewer than a handful of services on one or two servers, that overhead usually isn’t justified yet.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Is Kubernetes always better than Docker Compose?
    No. Kubernetes is better for multi-node, high-availability, high-scale workloads. For single-server apps, Compose is simpler, faster to deploy, and easier to maintain without a dedicated ops team.

    Can I use Docker Compose in production?
    Yes, plenty of production applications run on Compose successfully, especially on a single well-provisioned server with proper backups, monitoring, and a reverse proxy handling TLS and health checks.

    Do I need Kubernetes experience to use Docker Compose effectively?
    No, they’re independent skill sets. Compose only requires familiarity with Docker itself. Kubernetes requires learning an entirely separate set of concepts around scheduling, networking, and cluster administration.

    What’s a lightweight alternative if I want some Kubernetes benefits without full complexity?
    k3s and MicroK8s are lightweight Kubernetes distributions designed for smaller clusters or edge deployments, offering a middle ground between Compose and full-scale managed Kubernetes.

    Can Docker Compose and Kubernetes be used together?
    Not directly at runtime, but Compose files are commonly used for local development while Kubernetes manifests (sometimes generated via Kompose) handle production deployment — giving you a fast local loop and a scalable production target.

    How do I know when it’s time to migrate from Compose to Kubernetes?
    When you consistently need more capacity than a single server can provide, require zero-downtime deployments as a hard requirement, or your on-call team is manually restarting failed containers across multiple machines, it’s time to evaluate Kubernetes.

    Final Verdict

    Docker Compose and Kubernetes aren’t rivals fighting for the same job — they’re tools sized for different problems. Start with Compose. Move to Kubernetes only when you have a concrete scaling or reliability requirement that Compose genuinely can’t meet, not because it’s the trendier option. For a deeper look at container fundamentals before making this call, see our complete guide to Docker networking.

  • Docker Compose Up Build: Full Guide & Best Practices

    Docker Compose Up Build: The Complete Guide to Rebuilding and Running Containers

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve spent any time working with multi-container applications, you’ve almost certainly typed docker compose up build into your terminal, only to get a syntax error or unexpected behavior. The correct command is docker compose up --build, and understanding exactly what it does — and when to use it — will save you hours of debugging “why isn’t my code change showing up” mysteries.

    This guide covers the full mechanics of docker compose up --build, how it differs from running docker compose build separately, common pitfalls with caching, and practical workflows for local development, CI pipelines, and production-adjacent staging environments.

    What Does docker compose up –build Actually Do

    The docker compose up --build command tells Compose to rebuild any service images before starting the containers. Without the --build flag, Compose only builds an image if it doesn’t already exist locally — if you’ve made changes to your Dockerfile or the files it copies in, those changes will be silently ignored and you’ll keep running the old image.

    Here’s the basic syntax:

    docker compose up --build

    This single command does three things in sequence:

  • Rebuilds every service in your docker-compose.yml that has a build: directive
  • Recreates containers that depend on those rebuilt images
  • Starts (or restarts) all services defined in the compose file
  • If you only want to rebuild and start a single service, you can scope it:

    docker compose up --build web

    This rebuilds and starts only the web service, along with any services it depends on via depends_on.

    Why the Plain docker compose up Command Isn’t Enough

    A lot of confusion comes from assuming Compose always rebuilds on up. It doesn’t. Compose checks whether an image already exists for the service; if it does, up reuses it, even if your source code or Dockerfile has changed since the image was built.

    This is a common source of “phantom bugs” where a developer fixes an issue, runs docker compose up, and the bug is still there — because the container is running stale code baked into an old image layer. Running docker compose up --build forces Compose to re-evaluate the build context and rebuild as needed.

    The Difference Between docker compose build and docker compose up –build

    These two approaches look similar but serve different purposes:

    # Build images without starting containers
    docker compose build
    
    # Build images and immediately start containers
    docker compose up --build

    docker compose build is useful in CI/CD pipelines where you want to build and push an image without running it locally. docker compose up --build is better suited for local development loops where you’re iterating quickly and want your changes reflected immediately.

    If you’re setting up a CI pipeline, it’s worth reading the official Docker Compose CLI reference for the full list of flags and how they interact with build caching.

    Common Flags and Options You’ll Actually Use

    Beyond the basic --build flag, there are several options that make the command far more useful in day-to-day work.

    Forcing a Full Rebuild with –no-cache

    Docker’s build cache is usually your friend — it speeds up rebuilds by reusing unchanged layers. But sometimes the cache causes problems, especially when a base image has been updated upstream or a RUN apt-get install step needs fresh package lists.

    docker compose build --no-cache
    docker compose up --build --no-cache

    Note that --no-cache isn’t actually a flag on up directly in older Compose versions — if you hit an error, run the no-cache build separately first, then bring the stack up:

    docker compose build --no-cache && docker compose up

    Running in Detached Mode

    For background services, especially on a VPS or remote server, combine --build with -d:

    docker compose up --build -d

    This rebuilds images and runs containers in detached mode so your terminal session stays free. This is the pattern most people use when deploying to a DigitalOcean droplet or similar cloud VM — SSH in, pull the latest code, and run this one command to redeploy.

    Forcing Container Recreation

    Sometimes Compose decides a container doesn’t need to be recreated even after a rebuild, particularly if only environment variables changed. Force it with:

    docker compose up --build --force-recreate

    This guarantees old containers are torn down and replaced, not just restarted.

    Practical Workflow Examples

    Let’s walk through a realistic scenario. Say you have a simple Node.js API with the following docker-compose.yml:

    version: "3.9"
    services:
      api:
        build:
          context: .
          dockerfile: Dockerfile
        ports:
          - "3000:3000"
        environment:
          - NODE_ENV=development
        volumes:
          - ./src:/app/src
      db:
        image: postgres:16
        environment:
          - POSTGRES_PASSWORD=devpassword
        volumes:
          - pgdata:/var/lib/postgresql/data
    
    volumes:
      pgdata:

    And a Dockerfile like this:

    FROM node:20-alpine
    WORKDIR /app
    COPY package*.json ./
    RUN npm install
    COPY . .
    EXPOSE 3000
    CMD ["node", "src/index.js"]

    If you add a new npm dependency to package.json, running plain docker compose up won’t install it — the existing image already has node_modules baked in from the last build. You need:

    docker compose up --build

    This re-runs npm install inside the image build step, picking up the new dependency, then starts both the api and db services.

    Handling Multi-Service Rebuilds Efficiently

    In larger projects with multiple services (API, worker, frontend, reverse proxy), rebuilding everything every time is wasteful. Scope your builds:

    docker compose up --build api worker

    This rebuilds only api and worker, leaving db and any unchanged services untouched, which speeds up your iteration loop considerably.

    If you’re managing a growing compose file, it’s worth reviewing our guide to organizing multi-container Docker projects for patterns on splitting services across multiple compose files with docker compose -f.

    Troubleshooting Common docker compose up –build Issues

    A few issues come up repeatedly with this command, and most have simple fixes.

  • Changes not appearing after rebuild: Check whether you have a bind-mounted volume overwriting the container’s code directory with an outdated local copy. Volumes mounted in docker-compose.yml take precedence over what’s baked into the image.
  • Build succeeds but container immediately exits: Check your CMD or ENTRYPOINT — the container will exit if the foreground process crashes or completes. Run docker compose logs <service> to see the actual error.
  • “no space left on device” during build: Docker’s build cache and dangling images accumulate over time. Run docker system prune -a (carefully — this removes unused images) to reclaim space.
  • Environment variable changes not taking effect: Environment variables set in .env files or environment: blocks don’t always force a rebuild. Combine with --force-recreate to be safe.
  • Build context is too large and slow: Add a .dockerignore file to exclude node_modules, .git, and other large directories from being sent to the Docker daemon during build.
  • Speeding Up Builds with BuildKit

    Modern Docker installations use BuildKit by default, which parallelizes build steps and caches more intelligently. If you’re on an older setup, enable it explicitly:

    DOCKER_BUILDKIT=1 docker compose up --build

    BuildKit also supports cache mounts for package managers, which can dramatically speed up repeated npm install or pip install steps. See Docker’s own documentation on BuildKit for configuration details.

    Deploying with docker compose up –build on a Remote Server

    When deploying to a VPS, the workflow typically looks like this: SSH into the server, pull the latest code, and rebuild.

    ssh user@your-server-ip
    cd /opt/myapp
    git pull origin main
    docker compose up --build -d

    For production deployments, consider adding a health check to your compose file so Compose (and any orchestration layer) knows when a service is actually ready:

    services:
      api:
        build: .
        healthcheck:
          test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
          interval: 30s
          timeout: 5s
          retries: 3

    If you’re running this on a budget VPS provider, Hetzner offers solid price-to-performance ratios for small Docker Compose deployments, and their cloud consoles make it easy to spin up a fresh instance to test a deployment before pushing to your main server.

    For teams that need uptime monitoring on these deployed services, pairing your compose stack with BetterStack gives you alerting when a container-based service goes down, which is especially useful right after a --build deploy where a bad image can silently break a health check.

    If your compose-based app sits behind a public domain, routing DNS and basic DDoS protection through Cloudflare is a near-zero-cost way to harden a VPS-hosted deployment before you invest in a full load balancer setup.

    Automating Rebuilds in CI/CD

    Many teams wire docker compose up --build into a deployment script triggered by a git push. A minimal GitHub Actions step might look like:

    - name: Deploy via SSH
      run: |
        ssh -o StrictHostKeyChecking=no user@${{ secrets.SERVER_IP }} 
          "cd /opt/myapp && git pull && docker compose up --build -d"

    This pattern works well for small to medium projects. For anything with higher uptime requirements, you’ll want a proper CI pipeline that builds and tests images before they ever touch your production compose file — check out our beginner’s guide to Docker fundamentals if you’re still solidifying the basics before automating deployments.

    Best Practices Checklist

    Before you wire docker compose up --build into your regular workflow, keep these practices in mind:

  • Always add a .dockerignore file to keep build contexts small and fast
  • Use --build only when source files or Dockerfiles have changed — otherwise plain up is faster
  • Combine with -d for background services on remote servers
  • Use --force-recreate when environment variables or configs change without a corresponding image rebuild
  • Periodically run docker system prune to clean up unused build cache and dangling images
  • Pin base image versions (e.g., node:20-alpine instead of node:latest) to avoid unexpected behavior from upstream updates
  • Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Is “docker compose up build” a valid command?
    No. The correct syntax requires two dashes: docker compose up --build. Without the dashes, Docker will interpret build as a service name and likely return an error saying no such service exists.

    Does docker compose up –build always rebuild every service?
    Yes, by default it rebuilds all services in the compose file that have a build: directive. To limit the rebuild to specific services, list them after the command, like docker compose up --build api.

    What’s the difference between docker compose up –build and docker-compose up –build?
    Functionally they’re the same. docker compose (no hyphen) is the newer Compose V2 syntax integrated into the Docker CLI, while docker-compose is the older standalone Python-based tool. Compose V2 is faster and actively maintained, so it’s the recommended syntax going forward.

    Why isn’t my code change showing up even after running docker compose up –build?
    The most common cause is a bind-mounted volume overriding the container’s working directory with a stale local copy, or the build cache reusing a layer that should have been invalidated. Try docker compose build --no-cache followed by docker compose up --force-recreate.

    Does docker compose up –build slow down my workflow?
    It adds build time on every run, so it’s slower than plain up when nothing has changed. Use it selectively — only when you’ve modified your Dockerfile, dependencies, or files copied during the build step.

    Can I use docker compose up –build in production?
    It’s more common in development and simple deployment scripts. For production, most teams prefer building images in CI, pushing them to a registry, and having the production compose file reference a specific image tag rather than rebuilding on the server itself.

    Wrapping Up

    docker compose up --build is one of the most-used commands in a containerized development workflow, but its behavior around caching and image reuse trips up even experienced developers. The core rule to remember: if you’ve changed anything that affects the image (Dockerfile, dependencies, copied files), you need --build to see those changes reflected. Combine it with -d for background services, --force-recreate when configs change, and --no-cache when you suspect stale cache layers are the culprit, and you’ll avoid most of the common headaches associated with this command.

  • Docker Compose Log Command: A Complete Debugging Guide

    Docker Compose Log Command: A Complete Debugging Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If your containers are misbehaving, the first place to look is the log output — and the docker compose logs command is how you get it. This guide covers everything from basic syntax to advanced filtering, log driver configuration, and shipping logs to a centralized platform for production monitoring.

    Whether you’re running a single Compose stack on your laptop or a multi-service application in production, mastering the docker compose log workflow will save you hours of guesswork when something breaks.

    This article assumes you already have Docker and Docker Compose installed and a working docker-compose.yml — if you’re setting up a stack from scratch, get the containers running first with docker compose up -d, then come back here once you need to actually read what they’re doing. Everything below works identically whether your compose file defines two services or twenty, and the same flags apply whether you’re debugging on a local dev machine or SSH’d into a remote host.

    What Is the docker compose logs Command?

    docker compose logs is a subcommand of the Docker Compose CLI that aggregates and displays the stdout/stderr output from every container defined in your docker-compose.yml file (or a specific service, if you name one). Unlike docker logs, which only works against a single container, docker compose logs understands your entire stack — it knows which containers belong to which project and can interleave their output with service-name prefixes so you can tell at a glance which container printed what.

    This matters because most real applications aren’t a single container. A typical stack might include a web server, a database, a cache, and a background worker — four separate log streams that you’d otherwise have to tail individually with four terminal windows.

    Basic Syntax and Options

    The command follows this pattern:

    docker compose logs [OPTIONS] [SERVICE...]

    Run it with no arguments from the directory containing your docker-compose.yml and you’ll get the full log history for every service in the project:

    docker compose logs

    To scope it to one or more services, just name them:

    docker compose logs web db

    Following Logs in Real Time

    Static output is useful for a quick check, but during active debugging you want a live stream. Use -f or --follow:

    docker compose logs -f web

    This behaves like tail -f — new log lines appear as they’re written, and the command keeps running until you press Ctrl+C. This is the single most-used flag in the docker compose logs toolkit, and it’s usually the first thing you run after docker compose up -d.

    Filtering Logs by Service

    When you’re running several services and only care about one, always scope by service name rather than grepping through the combined output:

    docker compose logs -f --tail=100 api

    You can also pass multiple service names at once to watch related containers together, which is handy when debugging an interaction between, say, a frontend proxy and its backend API.

    Limiting Log Output with –tail

    By default, docker compose logs prints the entire buffered history, which can be enormous for long-running containers. The --tail flag limits output to the last N lines:

    docker compose logs --tail=50 web

    Combine it with -f to see recent context and then continue following:

    docker compose logs -f --tail=50 web

    Advanced Log Filtering Techniques

    Timestamps and Time Ranges

    Add -t (or --timestamps) to prefix every line with an ISO 8601 timestamp — essential when correlating log events with an incident timeline:

    docker compose logs -t web

    To narrow output to a specific window, use --since and --until:

    docker compose logs --since 2026-07-08T10:00:00 --until 2026-07-08T10:15:00 web

    You can also use relative durations like --since 30m or --since 2h, which is often faster than typing out a full timestamp when you’re chasing something that just happened.

    Combining Logs with grep

    docker compose logs doesn’t have a built-in search flag, but because it writes to stdout, you can pipe it straight into standard Unix tools:

    docker compose logs --no-color web | grep -i "error"

    The --no-color flag strips ANSI color codes so your grep matches aren’t broken up by escape sequences. This pattern — Compose logs piped into grep, awk, or jq (for JSON-formatted app logs) — covers the vast majority of manual debugging you’ll ever need to do.

    Handling Logs for Scaled Services

    If you’re running multiple replicas of a service with docker compose up --scale worker=3, docker compose logs worker interleaves output from all three containers, each prefixed with its own container name so you can tell them apart. This is useful for spotting whether a bug affects every replica or just one — a single misbehaving replica often points to a stuck connection or a stale cache entry rather than a code bug. If you need to isolate a single replica’s output, use docker ps to find its container ID and switch to plain docker logs <container_id> instead.

    Configuring Log Drivers in Docker Compose

    By default, Docker uses the json-file logging driver, which writes each container’s output to a JSON file on disk. That’s fine for development, but it has two problems in production: unbounded log files can fill your disk, and json-file logs disappear if the host is replaced.

    You can configure log rotation and an alternate driver per service directly in your docker-compose.yml:

    services:
      web:
        image: myapp:latest
        logging:
          driver: json-file
          options:
            max-size: "10m"
            max-file: "3"

    This caps each container’s log file at 10 MB and keeps a maximum of 3 rotated files, which prevents runaway disk usage — a surprisingly common cause of production outages that has nothing to do with your application code.

    For teams that need durable, searchable logs, switching to a shipping-friendly driver like journald or syslog, or running a sidecar log forwarder, is the next step. That’s a bigger topic than this article can fully cover, but if you’re already tuning Docker networking for multi-container apps, log driver configuration belongs in the same pass.

    Centralizing Logs for Production

    docker compose logs is a local debugging tool — it reads whatever the log driver has retained on that specific host. That’s a real limitation once you’re running more than one server, since logs vanish when a container is removed or a host is rebuilt. For production stacks, you want logs shipped somewhere durable and searchable.

    A few practical options:

  • Log shipping agents like Filebeat, Fluentd, or Vector can tail the Docker log directory and forward entries to a central store.
  • Managed log platforms such as BetterStack give you searchable, alertable log aggregation without running your own Elasticsearch cluster — worth evaluating if you’re outgrowing docker compose logs -f as your primary debugging tool.
  • Cloud-native logging — if you’re hosting on a VPS, pairing your Compose stack with a provider like DigitalOcean makes it straightforward to attach block storage for log retention or spin up a dedicated logging droplet.
  • Structured logging — configure your application to emit JSON logs so downstream tools can parse fields instead of regex-matching plain text.
  • If you’re just getting started with Compose in general, our Docker Compose restart policies guide is a good companion piece — restart behavior and logging are usually the first two things you tune once a stack graduates from “works on my machine” to “runs in production.”

    For the official reference on every driver and option, the Docker documentation on logging drivers covers configuration syntax that’s easy to get subtly wrong, particularly around driver-specific options that Compose passes through without validation.

    Rotating Logs Automatically

    Log rotation deserves its own callout because it’s the most commonly skipped step. Without max-size and max-file set, json-file logs grow forever. On a busy service, that can mean gigabytes of logs eating disk space within days. Add rotation to every service in your Compose file, not just the noisy ones — the quiet services are the ones nobody remembers to check until the disk is full.

    Using docker compose logs in CI/CD Pipelines

    CI runners are another place docker compose logs earns its keep. When an integration test fails inside a containerized service, the test runner has usually already exited before you can attach a terminal. The fix is to dump logs as a post-failure step in your pipeline configuration, for example in a GitHub Actions job:

    - name: Dump logs on failure
      if: failure()
      run: docker compose logs --no-color > compose-output.log

    Running this on failure means you get the full container output attached to the CI job log, instead of a bare “test failed” message with no context. This one habit saves more debugging time than almost any other logging practice on this list.

    Common Issues and Troubleshooting

    A few problems come up often enough to call out specifically:

  • “No such service” error — you named a service that doesn’t exist in docker-compose.yml, or you’re running the command from the wrong directory. docker compose logs only sees services defined in the compose file in your current working directory (or the one passed via -f).
  • Logs appear empty or truncated — check the container’s log driver. If it’s set to none or a non-default driver like syslog without local buffering, docker compose logs won’t have anything to show, since it reads from the driver’s local output, not the application directly.
  • Color codes cluttering piped output — always add --no-color when piping into grep, awk, or a file.
  • Logs missing after container restart — by default, Docker keeps logs from the current container instance. If a container was recreated (not just restarted), previous logs may be gone unless you’re shipping them elsewhere.
  • Interleaved output is hard to read — when following multiple services at once, each line is prefixed with the service name and color-coded in an interactive terminal, but this can still get noisy with high-throughput services. Narrow to one service when you need to focus.
  • Comparing docker logs vs docker compose logs

    It’s worth being explicit about the difference, since the two commands look similar but aren’t interchangeable:

    docker logs my_container_1
    docker compose logs my_service

    docker logs operates on a single container by name or ID and knows nothing about Compose projects. docker compose logs operates on services defined in your compose file and can span multiple containers per service if you’re running replicas. If you only remember one rule, remember this: use docker compose logs when you’re thinking in terms of your application’s services, and docker logs when you’re debugging one specific container instance.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: How do I see logs for only one service in a multi-service Compose stack?
    A: Pass the service name as an argument: docker compose logs web. You can list multiple service names separated by spaces to watch several at once.

    Q: How do I follow logs in real time?
    A: Add the -f or --follow flag: docker compose logs -f. Press Ctrl+C to stop following.

    Q: Why does docker compose logs show no output at all?
    A: Usually because the container isn’t running, the log driver is set to something that doesn’t retain local output (like none), or you’re running the command outside the directory containing your docker-compose.yml.

    Q: Can I export docker compose logs to a file?
    A: Yes — redirect stdout: docker compose logs --no-color > app.log. Use --no-color first so the file doesn’t contain ANSI escape codes.

    Q: How do I limit how much log history is shown?
    A: Use --tail=N to show only the last N lines per service, or --since to restrict output to a time window.

    Q: Does docker compose logs work after a container has stopped?
    A: Yes, as long as the container hasn’t been removed. Docker retains the log file for stopped containers until they’re deleted with docker compose down or docker rm.

    Wrapping Up

    The docker compose logs command is the fastest path from “something’s wrong” to “here’s why.” Start with -f and --tail for everyday debugging, add --since and -t when you need to correlate events with a timeline, and don’t skip log rotation once you’re running anything in production. When local log files stop being enough, that’s your signal to invest in centralized, searchable logging rather than scrolling through terminal history.

  • Docker Compose Volumes: The Complete Setup Guide

    Docker Compose Volumes: A Practical Guide to Persistent Data

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever restarted a container and watched your database vanish into thin air, you already understand why docker compose volumes matter. Containers are ephemeral by design — anything written inside a container’s writable layer disappears the moment that container is removed. Volumes are Docker’s answer to this problem, and getting them right is one of the first things you need to master before running anything stateful in production.

    This guide covers everything you need to know about docker compose volumes: named volumes, bind mounts, tmpfs mounts, permissions, backups, and the common mistakes that cause data loss. We’ll use real, runnable docker-compose.yml examples throughout.

    Why Volumes Exist

    Docker containers are meant to be disposable. You should be able to destroy a container and recreate it from its image without worrying about what happens to your application’s state. But that raises an obvious problem: where does your database, your uploaded files, or your configuration live if the container itself is throwaway?

    The answer is volumes — storage that exists independently of the container lifecycle. Docker supports three main mechanisms for persisting data:

  • Named volumes — managed by Docker, stored under /var/lib/docker/volumes/ on the host
  • Bind mounts — a direct mapping between a host path and a container path
  • tmpfs mounts — in-memory storage that never touches disk, useful for secrets or caches
  • Each has different tradeoffs, and Compose makes it easy to declare any of them declaratively in a single YAML file.

    Named Volumes: The Default Choice

    Named volumes are the recommended approach for most production workloads. Docker manages the storage location for you, and volumes persist independently of any container — even if you run docker compose down, the volume survives unless you explicitly pass -v or --volumes.

    Here’s a basic example using PostgreSQL:

    services:
      db:
        image: postgres:16
        environment:
          POSTGRES_PASSWORD: examplepass
        volumes:
          - pgdata:/var/lib/postgresql/data
        ports:
          - "5432:5432"
    
    volumes:
      pgdata:

    Notice the two-part structure: the volumes: key under the service maps a named volume (pgdata) to a path inside the container. The top-level volumes: block declares that pgdata exists and lets Compose create it if it doesn’t already.

    You can inspect the volume directly:

    docker volume inspect pgdata
    docker volume ls

    Named volumes are portable across host filesystems, get automatic driver support (local, NFS, cloud-backed drivers), and are the correct choice when you don’t need to browse the files from the host directly.

    Bind Mounts: Direct Host Access

    Bind mounts map a specific directory or file on the host machine into the container. They’re most useful during development, when you want code changes on your host to reflect immediately inside the container without rebuilding the image.

    services:
      app:
        build: .
        volumes:
          - ./src:/usr/src/app
          - ./config/nginx.conf:/etc/nginx/nginx.conf:ro
        ports:
          - "3000:3000"

    The :ro suffix mounts the file read-only inside the container, which is a good habit for configuration files you don’t want the app accidentally overwriting. Bind mounts are also the standard way to expose the Docker socket for tools that need to manage containers, though that comes with security implications worth understanding before you do it in a shared environment — see Docker’s own guidance on socket security if you’re considering it.

    The downside of bind mounts is portability: the host path has to exist and be correct on every machine that runs the Compose file, which makes them a poor fit for production deployments across different servers.

    tmpfs Mounts: Ephemeral In-Memory Storage

    tmpfs mounts never persist to disk. They’re wiped when the container stops, which makes them ideal for temporary secrets, session data, or scratch space that shouldn’t leak onto the host filesystem.

    services:
      cache:
        image: redis:7
        tmpfs:
          - /data

    This is a niche use case compared to named volumes and bind mounts, but it’s worth knowing it exists — especially for compliance-sensitive workloads where you need a guarantee that certain data never touches persistent storage.

    A Real-World Multi-Service Example

    Here’s a more complete example combining a web app, a database, and a reverse proxy — a common pattern for anything you’d deploy on a VPS:

    services:
      web:
        build: .
        volumes:
          - ./app:/usr/src/app
          - static_files:/usr/src/app/static
        depends_on:
          - db
    
      db:
        image: postgres:16
        volumes:
          - pgdata:/var/lib/postgresql/data
        environment:
          POSTGRES_PASSWORD: examplepass
    
      nginx:
        image: nginx:alpine
        volumes:
          - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
          - static_files:/usr/src/app/static:ro
        ports:
          - "80:80"
    
    volumes:
      pgdata:
      static_files:

    This pattern shares the static_files volume between the web and nginx services, so nginx can serve static assets directly without proxying every request through the app server. If you’re setting up a similar stack on your own infrastructure, our guide on deploying Docker Compose to a production VPS walks through the full process including firewall and TLS setup.

    Permissions and Ownership Gotchas

    One of the most common issues developers hit with docker compose volumes is file permission mismatches. If your container runs as a non-root user (which it should, for security reasons), files written by that user inside a bind-mounted directory will be owned by the corresponding UID on the host — which may not match any real user on your system.

    A few practical fixes:

  • Match the container’s UID to your host user with the user: directive in Compose
  • Use a startup script that runs chown on the mounted directory before dropping privileges
  • For named volumes, let Docker manage ownership — it’s usually less painful than bind mounts for this exact reason
  • If you’re chasing a permission denied error, check docker compose exec <service> id and compare it against the ownership of the files with ls -la on the host. Mismatched UIDs are the cause more often than any actual bug in your application.

    Backing Up and Restoring Volumes

    Since named volumes live outside your source-controlled Compose file, you need a separate backup strategy. The simplest approach uses a throwaway container to tar up the volume contents:

    docker run --rm 
      -v pgdata:/data 
      -v $(pwd):/backup 
      alpine 
      tar czf /backup/pgdata-backup.tar.gz -C /data .

    To restore, reverse the process:

    docker run --rm 
      -v pgdata:/data 
      -v $(pwd):/backup 
      alpine 
      tar xzf /backup/pgdata-backup.tar.gz -C /data

    For anything running in production, don’t rely on manual backups alone. Automated monitoring that alerts you before a disk fills up or a backup job silently fails is worth the setup time — BetterStack offers uptime and log monitoring that integrates cleanly with Dockerized stacks if you want alerts wired up without building your own tooling from scratch.

    Choosing the Right Storage Backend for Your Host

    Where your volumes physically live matters more than most people realize until they hit an I/O bottleneck. If you’re running database-heavy workloads in Compose, the underlying disk speed of your VPS provider directly affects query latency. Providers like Hetzner and DigitalOcean both offer NVMe-backed instances that handle volume-heavy workloads noticeably better than budget shared hosting — worth considering if you’re scaling past a hobby project. If your Compose stack sits behind a domain and you want DDoS protection and caching in front of it, Cloudflare is a standard, low-friction addition to the stack.

    Common Mistakes to Avoid

  • Forgetting the top-level volumes: declaration — Compose will still create an anonymous volume if you skip it, but you lose the ability to reference it by name elsewhere
  • Using bind mounts for production databases — path mismatches between environments cause subtle, hard-to-debug failures
  • Running docker compose down -v without thinking — this deletes named volumes along with the containers, which has destroyed more than one person’s local database
  • Not setting :ro on config files — an app with a bug that writes to its own config file can silently corrupt your setup
  • Ignoring volume drivers — if you need multi-host storage (NFS, cloud block storage), you need a volume driver configured; the default local driver only works on a single host
  • If you’re moving from a single-container setup to a full multi-service stack, it’s also worth reading through our breakdown of Docker networking modes, since volumes and networking decisions often need to be made together when you’re designing how services talk to each other and to the outside world.

    Wrapping Up

    Docker Compose volumes aren’t complicated once you understand the distinction between named volumes, bind mounts, and tmpfs — the hard part is picking the right one for each use case and avoiding the permission and lifecycle mistakes that catch people off guard. For production database storage, default to named volumes. For local development where you need live code reloading, bind mounts make sense. Reserve tmpfs for genuinely ephemeral data that should never touch disk.

    Get comfortable inspecting volumes with docker volume ls and docker volume inspect, build a backup habit early, and pay attention to the underlying disk performance of whatever host you’re running on — it matters more than most Compose tutorials mention.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: What’s the difference between a named volume and a bind mount?
    A: A named volume is managed entirely by Docker and stored in Docker’s own directory structure on the host, while a bind mount maps a specific host path directly into the container. Named volumes are more portable; bind mounts give you direct host-side access to the files.

    Q: Does docker compose down delete my volumes?
    A: No, by default docker compose down stops and removes containers but leaves named volumes intact. You have to explicitly add the -v flag (docker compose down -v) to remove volumes as well.

    Q: Can I share one volume between multiple services in the same Compose file?
    A: Yes. Declare the volume once in the top-level volumes: block and reference it in the volumes: section of each service that needs it, as shown in the nginx/web example above.

    Q: Why do I get permission denied errors when writing to a mounted volume?
    A: This usually happens because the UID the container process runs as doesn’t match the ownership of the files on the host. Check with docker compose exec <service> id and adjust ownership or the container’s user: directive accordingly.

    Q: How do I back up a named volume?
    A: Run a temporary container that mounts the volume alongside a host directory, then use tar to archive the volume’s contents to that host directory, as shown in the backup example above.

    Q: Should I use volumes for stateless services?
    A: Generally no. If a service doesn’t need to retain data across restarts, skip the volume entirely and let the container’s writable layer handle it — adding a volume you don’t need just adds cleanup overhead later.

  • Dockerfile vs Docker Compose: Key Differences Explained

    Dockerfile vs Docker Compose: What’s the Real Difference?

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’re new to containers, the confusion between a Dockerfile and Docker Compose is almost a rite of passage. Both files live in your project, both start with the word “Docker,” and both seem to do something with building and running containers. But they solve different problems, and understanding that difference is the key to writing clean, maintainable Docker setups.

    This guide breaks down exactly what a Dockerfile does, what Docker Compose does, when you need one versus both, and how to avoid the mistakes most beginners make when moving from a single container to a full multi-service stack.

    What a Dockerfile Actually Does

    A Dockerfile is a plain-text script of instructions that tells Docker how to build a single image. Think of it as a recipe: start with a base layer, install dependencies, copy in your code, and define what happens when a container starts. Every line becomes a cached layer, and Docker builds those layers in order.

    Here’s a minimal Dockerfile for a Node.js API:

    FROM node:20-alpine
    
    WORKDIR /app
    
    COPY package*.json ./
    RUN npm ci --omit=dev
    
    COPY . .
    
    EXPOSE 3000
    CMD ["node", "server.js"]

    You build it with:

    docker build -t my-node-api:1.0 .

    And run it with:

    docker run -d -p 3000:3000 --name node-api my-node-api:1.0

    That’s the entire scope of a Dockerfile: define one image. It doesn’t know or care about other containers, networks, volumes, or environment-specific configuration beyond what’s baked in at build time or passed via docker run flags. If your application is just one container talking to nothing else, a Dockerfile alone is enough.

    Why Layer Caching Matters

    Docker caches each instruction as a layer. If you change a line further down the file, only that layer and everything after it gets rebuilt — everything above is reused from cache. This is why the COPY package*.json ./ and RUN npm ci steps happen before COPY . . in the example above. Your dependencies rarely change, but your source code changes constantly. Ordering instructions from least-to-most volatile can cut build times from minutes to seconds during active development.

    What Docker Compose Actually Does

    Docker Compose operates one layer up. Instead of describing how to build a single image, it describes how to run and connect multiple containers — services, networks, volumes, and environment variables — using a single declarative YAML file: docker-compose.yml (or compose.yaml in newer syntax).

    Here’s a realistic example: a Node.js API, a PostgreSQL database, and Redis for caching.

    version: "3.9"
    
    services:
      api:
        build: .
        ports:
          - "3000:3000"
        environment:
          - DATABASE_URL=postgres://appuser:apppass@db:5432/appdb
          - REDIS_URL=redis://cache:6379
        depends_on:
          - db
          - cache
    
      db:
        image: postgres:16-alpine
        environment:
          - POSTGRES_USER=appuser
          - POSTGRES_PASSWORD=apppass
          - POSTGRES_DB=appdb
        volumes:
          - db_data:/var/lib/postgresql/data
    
      cache:
        image: redis:7-alpine
    
    volumes:
      db_data:

    One command spins up the entire stack:

    docker compose up -d

    Notice the build: . line under the api service. That’s the connection point — Compose doesn’t replace the Dockerfile, it calls it. Compose builds the api image using the Dockerfile in the current directory, then runs it alongside the db and cache containers, automatically wiring them together on a shared internal network where each service can reach the others by name (db, cache).

    The Networking Piece Nobody Mentions

    One of the biggest reasons teams adopt Compose isn’t just convenience — it’s networking. Every docker compose up creates a dedicated bridge network for that project, and container names become resolvable hostnames inside it. That’s why the example above uses db:5432 instead of an IP address. If you tried to replicate this with plain docker run commands, you’d need to manually create a network with docker network create and attach every container to it with --network, then track IPs or aliases yourself. Compose handles all of that from a single file, which is a big part of why it fits so well into local development and small production deployments alike.

    Dockerfile vs Docker Compose: Side-by-Side

    | | Dockerfile | Docker Compose |
    |—|—|—|
    | Purpose | Build one image | Orchestrate multiple containers |
    | Format | Instruction script | Declarative YAML |
    | Scope | Single service | Full application stack |
    | Networking | None built-in | Automatic shared network |
    | Command | docker build | docker compose up |
    | Typical use | Defining an app’s runtime environment | Local dev environments, small multi-service deployments |

    The short version: a Dockerfile answers “how do I package this one application?” Docker Compose answers “how do multiple packaged applications run together?” You’ll almost never choose one instead of the other in a real project — you’ll write a Dockerfile for each custom service, then reference those Dockerfiles from a Compose file that ties everything together with prebuilt images like Postgres and Redis.

    Common Beginner Mistakes

  • Putting build logic inside docker-compose.yml. Compose can technically inline some build args, but application build steps (installing dependencies, compiling code) belong in the Dockerfile, not YAML.
  • Hardcoding secrets in either file. Use .env files with Compose’s env_file: directive instead of committing credentials to version control.
  • Rebuilding unnecessarily. Running docker compose up alone won’t rebuild your image after a Dockerfile change — you need docker compose up --build or docker compose build first.
  • Skipping .dockerignore. Without it, COPY . . in your Dockerfile will pull in node_modules, .git, and other bloat, slowing builds and inflating image size.
  • Using latest tags in production. Pin explicit versions (postgres:16-alpine, not postgres:latest) so a base image update doesn’t silently break your stack.
  • If you’re still fuzzy on how images and containers relate at a lower level, our beginner’s guide to Docker containers covers the fundamentals before you dive into multi-service setups. And once your Compose stack grows past a handful of services, it’s worth reading up on Docker networking modes to understand bridge, host, and overlay networks in more depth.

    When You Only Need a Dockerfile

    If you’re shipping a single stateless service — a static site, a small API with no database, a CLI tool packaged as a container — a Dockerfile is all you need. Add Compose and you’re adding a layer of abstraction (and a file to maintain) for zero actual benefit. Keep it simple:

    docker build -t my-tool .
    docker run --rm my-tool

    When You Need Both

    Any app with more than one moving part — a web server plus a database, a worker queue plus a cache, a frontend plus a backend plus a reverse proxy — benefits from Compose. It replaces long, error-prone docker run commands with a single reproducible file that any teammate (or your future self) can spin up with one command. This is also where Compose shines for local development: your entire stack — including database seed data via volumes — starts and stops in seconds, matching production topology closely enough to catch integration bugs early.

    For production, Compose still works well for small-to-medium deployments on a single VPS. If you’re scaling past one host or need rolling updates and self-healing, that’s when tools like Kubernetes or Docker Swarm enter the picture — but for most side projects, SaaS MVPs, and small business apps, Compose on a well-sized VPS is genuinely enough. Both DigitalOcean and Hetzner publish solid reference docs on running Compose stacks on their droplets/servers, and either is a reasonable starting point if you’re picking infrastructure.

    Deploying a Compose Stack to a VPS

    A typical low-friction production setup looks like this:

    # On your VPS
    git clone https://github.com/yourorg/yourapp.git
    cd yourapp
    cp .env.example .env   # fill in real secrets
    docker compose up -d --build

    Pair that with a process to pull the latest image on deploy, and you have a repeatable, low-maintenance pipeline without needing a full orchestration platform. If uptime matters for that stack, it’s worth wiring in external monitoring — a service like BetterStack can alert you the moment a container health check starts failing, which is far better than finding out from an angry user. For securing the public-facing side of that VPS, Cloudflare in front of your reverse proxy adds DDoS protection and free TLS with minimal setup.

    Once your Compose-based deployment is live, our guide to managing containers with Portainer is a good next read if you want a visual dashboard on top of the CLI.

    Quick Reference Commands

  • docker build -t name . — build an image from a Dockerfile
  • docker run -p 3000:3000 name — run a single container from an image
  • docker compose up -d — start all services defined in docker-compose.yml, detached
  • docker compose down — stop and remove containers, networks (add -v to also remove volumes)
  • docker compose logs -f api — tail logs for a specific service
  • docker compose exec api sh — open a shell inside a running service container
  • Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Do I need a Dockerfile if I’m using Docker Compose?
    Only for services you’re building yourself. Prebuilt images like postgres or redis don’t need a Dockerfile — Compose pulls them directly from a registry. You only write a Dockerfile for custom application code that Compose then references via build: ..

    Can Docker Compose replace a Dockerfile entirely?
    No. Compose can reference an existing image, but it can’t build a custom image from source code without a Dockerfile (or an equivalent build: context). They’re complementary, not interchangeable.

    Is docker-compose different from docker compose?
    Yes, slightly. docker-compose (with a hyphen) is the older standalone Python tool. docker compose (space, no hyphen) is the newer Go-based plugin built into the Docker CLI. Functionality is nearly identical, but the plugin version is actively maintained and recommended for new projects.

    Should I use Docker Compose in production?
    For small-to-medium single-host deployments, yes — it’s simple, reliable, and well-documented. For multi-host, auto-scaling, or high-availability needs, look at Kubernetes or Docker Swarm instead.

    What’s the difference between docker-compose.yml and compose.yaml?
    They’re functionally the same; compose.yaml is the newer preferred filename per the Compose Specification, but Docker still auto-detects either name.

    Can I use environment variables across both files?
    Yes. Compose reads a .env file in the same directory automatically and can pass those values into both the environment: section and build args for the Dockerfile via args: under build:.

    Final Takeaway

    A Dockerfile packages one application. Docker Compose orchestrates many. Nearly every real-world project uses both: Dockerfiles for each custom service, and a Compose file to wire those services together with databases, caches, and networking — all from one command. Once you internalize that division of labor, the rest of the Docker ecosystem — volumes, networks, multi-stage builds — starts making a lot more sense.

  • Docker Compose Rebuild: Complete Guide & Best Tips

    Docker Compose Rebuild: How to Rebuild Containers the Right Way

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever edited a Dockerfile or changed a dependency in requirements.txt or package.json and then run docker compose up, only to find your container is still running the old code, you’ve hit the single most common Docker Compose gotcha: Compose doesn’t rebuild images automatically just because the underlying files changed. You have to tell it to rebuild, and there’s more than one way to do that — each with different tradeoffs around speed, cache usage, and which services get touched.

    This guide covers every practical way to do a docker compose rebuild, when to use --build versus docker compose build, how to force a truly clean rebuild with --no-cache, how to rebuild a single service without restarting your entire stack, and the pitfalls that trip up even experienced engineers. We’ll also touch on wiring rebuilds into CI/CD so you’re not doing this by hand forever.

    Why You Need to Rebuild Docker Compose Services

    Docker images are built once from a Dockerfile and then cached as layers. docker compose up will happily start a container from an existing image even if the source code, dependencies, or base image have changed since that image was built. Compose only rebuilds automatically in a couple of narrow cases (like when the image doesn’t exist yet). Everything else — a code change, a new apt-get install, an updated FROM line — requires an explicit rebuild step.

    This is different from bind-mounted development setups where your code changes are reflected live because the host directory is mounted into the container. If you’re relying on volumes for hot-reload, you may not need a rebuild at all for code changes — just Compose configuration changes. But for anything baked into the image itself, a rebuild is mandatory.

    What “Rebuild” Actually Means in Docker Compose

    When people say “rebuild,” they usually mean one of two distinct operations:

    1. Rebuilding the image — re-running the instructions in your Dockerfile (or a subset of them, depending on the Docker layer cache) to produce a new image.
    2. Recreating the container — tearing down the running container and starting a new one, which is necessary even after a successful image rebuild, because a running container is tied to the image ID it started from.

    docker compose up --build does both in sequence: it rebuilds any image whose build context has an associated build: key in your compose.yaml, then recreates and starts the containers using the fresh images. Understanding this two-step nature is key to debugging rebuild issues later.

    docker compose build vs. up –build

    The cleanest way to just rebuild images without touching running containers is docker compose build:

    # Rebuild all services defined with a `build:` key
    docker compose build
    
    # Rebuild only one service
    docker compose build web
    
    # Rebuild and show full build output (no truncated logs)
    docker compose build --progress=plain

    This command only builds images — it does not start or restart anything. That makes it ideal for CI pipelines where you want to build and push an image before deployment, separate from the runtime step.

    If you want to rebuild and immediately bring the stack up with the new images, use:

    docker compose up --build

    This is the command most developers reach for during local development. It rebuilds any stale images, recreates containers that depend on them, and leaves unaffected services alone. Add -d to run detached:

    docker compose up --build -d

    One subtlety: --build still respects the Docker layer cache. If your Dockerfile hasn’t changed and neither has anything copied into the image (like COPY . .), Compose may reuse cached layers even with --build passed. That’s usually what you want — it’s fast — but it can bite you if you’ve changed a file that Docker’s cache invalidation didn’t catch, which brings us to the next point.

    Forcing a Clean Rebuild with –no-cache

    Sometimes the layer cache lies to you — a base image was updated remotely, a package version got pinned but not bumped, or you’re debugging a build that behaves inconsistently across machines. In those cases, force Docker to ignore the cache entirely:

    docker compose build --no-cache

    Combine it with --pull to also fetch the latest version of any base images referenced in FROM lines, rather than reusing a stale local copy:

    docker compose build --no-cache --pull

    Then bring the stack up with the freshly built images:

    docker compose up -d --force-recreate

    --force-recreate is worth calling out separately — it recreates containers even if Compose thinks nothing changed, which is useful when you’ve modified environment variables or mounted config files but the image itself is identical. For the full command reference, see the official Docker Compose CLI documentation.

    A full clean-slate rebuild, useful when things are genuinely broken and you want zero ambiguity, looks like this:

    docker compose down
    docker compose build --no-cache --pull
    docker compose up -d

    Be aware --no-cache rebuilds every layer from scratch, which can take significantly longer for images with large dependency installs. Don’t reach for it as your default — reserve it for when you suspect cache poisoning or inconsistent build behavior.

    Rebuilding a Single Service Without Touching the Rest

    In a multi-service stack — say, a web app, a worker, a Postgres database, and Redis — you rarely want to rebuild everything just because you changed one Dockerfile. Target the specific service by name:

    docker compose build worker
    docker compose up -d --no-deps worker

    --no-deps is important here. Without it, Compose will also recreate any services listed under depends_on for that service, which can cause unnecessary downtime for your database or cache layer. If you only want the one container recreated with its fresh image, --no-deps keeps the blast radius contained.

    For a related walkthrough on when to use up versus run for one-off commands against a service, see our guide on docker compose up vs run.

    Common Docker Compose Rebuild Pitfalls

    A handful of mistakes account for most of the “I rebuilt it but nothing changed” reports:

  • Editing a Dockerfile but running plain docker compose up — without --build, Compose reuses the existing image, no matter how recently the Dockerfile changed.
  • Assuming --build bypasses the cache — it doesn’t. --build triggers a build, but the Docker layer cache still applies unless you add --no-cache.
  • COPY ordering that invalidates cache too aggressively — if COPY . . happens before RUN npm install or RUN pip install -r requirements.txt, every source change forces a full dependency reinstall. Copy dependency manifests first, install, then copy the rest of the source.
  • Stale volumes masking a rebuild — if a named volume is mounted over /app or /node_modules, a rebuilt image’s files can be shadowed by old volume data. Run docker compose down -v (carefully — this deletes volume data) if you suspect this.
  • Forgetting --no-deps — rebuilding one service accidentally restarts dependent services, causing unnecessary connection drops for your database or message queue.
  • Confusing image tags across environments — if your compose.yaml references image: myapp:latest and you build locally but deploy a pulled image remotely, the rebuild only affects your local tag, not what’s running elsewhere.
  • If your stack also touches disk space issues from repeated rebuilds, our Docker cleanup and prune guide covers reclaiming space from dangling images and build cache without nuking things you still need.

    Automating Rebuilds in CI/CD

    Manual rebuilds are fine for local development, but production deployments should treat image builds as a discrete, versioned step — not something that happens implicitly on the server. A typical pattern:

    # CI step: build and tag with the commit SHA
    docker compose build
    docker tag myapp_web:latest registry.example.com/myapp:$(git rev-parse --short HEAD)
    docker push registry.example.com/myapp:$(git rev-parse --short HEAD)

    On the deployment server, you then pull the pre-built image rather than rebuilding from source, which avoids surprises from a build environment drifting between CI and production. The official Docker build command reference documents the underlying flags Compose wraps, which is useful when you need finer control than Compose exposes directly, like --build-arg or multi-platform builds with buildx.

    If you’re hosting the Compose stack itself on a VPS, a droplet from DigitalOcean gives you predictable resources for running your build and deploy pipeline without fighting shared-tenancy noisy neighbors. And once your rebuild-and-deploy pipeline is live, pairing it with uptime monitoring from BetterStack means you’ll get paged the moment a bad rebuild takes a service down, rather than finding out from a user complaint.

    Verifying a Rebuild Actually Took Effect

    After any rebuild, confirm the running container is actually using the new image rather than trusting that the command succeeded silently:

    docker compose images
    docker compose ps
    docker inspect --format='{{.Created}}' $(docker compose images -q web)

    The Created timestamp should match your rebuild time. If it doesn’t, you’re still running a stale image, and it’s worth checking whether the service actually has a build: key defined in compose.yaml at all — Compose can’t rebuild an image it was never told how to build; it will just keep pulling the same tagged image from a registry instead.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: Does docker compose up rebuild images automatically?
    A: No. docker compose up only builds an image if it doesn’t already exist locally. If the image exists — even if it’s outdated — Compose will use it as-is. You need --build to force a rebuild check.

    Q: What’s the difference between docker compose build and docker compose up --build?
    A: docker compose build only builds images and does not start containers. docker compose up --build builds images first, then creates and starts containers from them. Use build alone in CI pipelines and up --build for local development loops.

    Q: Why doesn’t --no-cache seem to make a difference?
    A: Check that the service actually has a build: block in your compose.yaml. If it only has an image: key pointing at a registry image, Compose has nothing to build — it will just pull, and --no-cache has no effect on pulls.

    Q: How do I rebuild without restarting my database container?
    A: Rebuild and recreate only the target service with docker compose build <service> followed by docker compose up -d --no-deps <service>. This avoids cascading restarts to dependencies listed under depends_on.

    Q: Will rebuilding delete my database data?
    A: No, as long as your data lives in a named volume or bind mount rather than the container’s writable layer. Rebuilding the image and recreating the container doesn’t touch volumes unless you explicitly run docker compose down -v.

    Q: How can I speed up slow rebuilds?
    A: Order your Dockerfile so dependency installation happens before copying source code, use .dockerignore to exclude unnecessary files from the build context, and avoid --no-cache unless you specifically need a clean build — the layer cache exists precisely to make repeat rebuilds fast.

  • Docker Compose Env: Manage Variables the Right Way

    Docker Compose Env: A Complete Guide to Managing Environment Variables

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve spent any time building multi-container applications, you already know that hardcoding configuration values into your docker-compose.yml is a fast track to pain. Passwords end up in git history, staging and production drift apart, and every new team member has to guess which values actually matter. Getting docker compose env handling right — cleanly, securely, and predictably — is one of the highest-leverage things you can do for a Compose-based stack.

    This guide covers every practical way Compose lets you inject environment variables into containers: the environment key, .env files, the env_file directive, shell interpolation, and the precedence rules that decide which value wins when they collide. We’ll also cover common mistakes, security considerations, and how this fits into a broader Docker Compose networking guide if you’re building out a full stack.

    Why Environment Variables Matter in Compose

    Environment variables are the standard way to configure containerized applications without baking configuration into the image itself. This follows the twelve-factor app methodology, which recommends storing config in the environment rather than in code. Docker Compose gives you several mechanisms to do this, and understanding when to use each one saves you from a lot of “why isn’t this variable set” debugging sessions.

    The environment Key

    The most direct way to set variables is the environment key inside a service definition:

    services:
      web:
        image: myapp:latest
        environment:
          - NODE_ENV=production
          - API_PORT=3000
          - DEBUG=false

    You can also write it as a map instead of a list:

    services:
      web:
        image: myapp:latest
        environment:
          NODE_ENV: production
          API_PORT: 3000
          DEBUG: "false"

    Both forms are equivalent. The map form is often easier to read in larger files, and it avoids quoting ambiguity around booleans and numbers since YAML will coerce unquoted values.

    You can also pass through variables from your shell without hardcoding a value:

    services:
      web:
        environment:
          - API_KEY

    If API_KEY is exported in your shell, Compose passes that value straight into the container. This is useful for secrets you don’t want sitting in the compose file at all.

    Using .env Files for Variable Substitution

    Compose automatically reads a file named .env in the same directory as your docker-compose.yml and uses it for variable interpolation inside the compose file itself — not for injecting variables directly into containers. That distinction trips up a lot of people.

    A typical .env file looks like this:

    POSTGRES_USER=appuser
    POSTGRES_PASSWORD=supersecret
    POSTGRES_DB=appdb
    APP_PORT=8080

    You reference these values inside docker-compose.yml using ${VARIABLE} syntax:

    services:
      db:
        image: postgres:16
        environment:
          POSTGRES_USER: ${POSTGRES_USER}
          POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
          POSTGRES_DB: ${POSTGRES_DB}
        ports:
          - "${APP_PORT}:5432"

    This lets you keep one compose file and swap out .env contents per environment (dev, staging, prod) without touching YAML. You can verify how Compose resolves these values before actually starting containers:

    docker compose config

    This command prints the fully resolved configuration, with every ${VAR} substitution applied — extremely useful for debugging why a container isn’t getting the value you expect.

    The env_file Directive

    Where .env handles substitution in the compose file, env_file loads variables directly into a specific service’s container environment:

    services:
      api:
        image: myapp:latest
        env_file:
          - .env.api
          - .env.secrets

    This is the cleanest option when you have many variables, since you avoid listing them one by one under environment. It also lets different services load entirely different variable sets, which environment alone can’t do as elegantly. A typical .env.api might contain:

    LOG_LEVEL=info
    CACHE_TTL=300
    FEATURE_FLAG_NEW_UI=true

    Keep in mind that values loaded via env_file are not available for ${} interpolation elsewhere in the compose file — only the root .env file gets that treatment.

    Precedence: Which Value Actually Wins?

    When the same variable is defined in multiple places, Compose applies a strict precedence order, from highest to lowest priority:

  • Values set with docker compose run -e VAR=value on the command line
  • Variables defined directly under the environment key in the service
  • Variables loaded through env_file
  • Variables already set in the container image (via ENV in the Dockerfile)
  • Variables exported in the shell that runs docker compose, when referenced via ${VAR}
  • In practice this means environment always overrides env_file, so if you’re debugging a mysterious value that won’t change, check whether it’s hardcoded under environment somewhere overriding your .env.api file.

    Overriding Variables per Environment with Multiple Compose Files

    A common pattern for managing dev vs. production differences is layering compose files:

    docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

    Your base file defines shared services, and docker-compose.prod.yml overrides just the pieces that differ:

    services:
      web:
        environment:
          NODE_ENV: production
        env_file:
          - .env.production

    This avoids maintaining two nearly identical full compose files and keeps environment-specific values isolated. For a deeper look at structuring multi-file Compose projects, see our guide to Docker Compose networking, which covers how these override patterns interact with custom networks.

    Common Mistakes to Avoid

    A handful of mistakes account for most of the “my env vars aren’t working” support requests:

  • Committing .env to version control. If it contains secrets, it belongs in .gitignore, with a .env.example template committed instead.
  • Assuming .env variables reach the container automatically. They only drive interpolation in the compose file — use environment or env_file to actually inject them.
  • Quoting issues in .env files. Compose does not process shell-style quoting the way bash does; VALUE="hello world" will include the literal quote characters. Leave values unquoted unless you specifically need them.
  • Forgetting to restart containers after changing .env. Compose doesn’t hot-reload environment values — you need docker compose up -d --force-recreate or at least a restart of the affected service.
  • Mixing up build-time and run-time variables. ARG in a Dockerfile is only available during the image build; it has nothing to do with the runtime environment block unless you explicitly pass it through with ENV.
  • Debugging Environment Variables Inside a Running Container

    Sometimes the fastest way to confirm what a container actually received is to just ask it. docker compose exec lets you run commands inside a running service:

    docker compose exec web printenv

    This prints every environment variable Compose actually passed to that container’s process — after all .env interpolation, environment overrides, and env_file merging have already happened. If a specific variable is missing, grep for it directly:

    docker compose exec web printenv | grep API_KEY

    For a container that crashes before you can exec into it, add a temporary sleep or check the logs with docker compose logs web to see if the application logged a missing-variable error on startup. Combining this with docker compose config — which shows what Compose resolved before the container even starts — usually narrows the problem down to either a compose-level substitution issue or an application-level parsing issue.

    Security Considerations

    Environment variables are convenient, but they’re not a secure secrets store. Anything set via environment or env_file is visible to anyone who can run docker inspect or exec into the container and read /proc/1/environ. For genuinely sensitive values — API keys, database passwords, TLS private keys — consider Docker secrets in Swarm mode, or a dedicated secrets manager, rather than relying purely on plaintext .env files.

    If you’re self-hosting your Compose stack, the underlying infrastructure matters just as much as the compose configuration. Running your containers on a provider like DigitalOcean gives you predictable, isolated droplets where you control firewall rules around which ports and services are actually exposed — critical when a misconfigured environment block accidentally binds something to 0.0.0.0. And once your stack is live, pairing it with uptime and log monitoring through a service like BetterStack means you’ll catch a broken deployment (say, a service that silently fails to start because a required env var is missing) within minutes instead of discovering it from a support ticket.

    If you’re new to Compose generally, our beginner’s guide to environment variables in Linux is a good primer on how shell-level exports interact with anything Compose eventually passes to a container.

    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Q: Does Compose automatically load .env from any directory?
    A: No. Compose only reads a .env file located in the same directory as the docker-compose.yml file you’re running, or the directory you specify with --project-directory. It does not search parent directories.

    Q: Can I use a different filename instead of .env?
    A: Yes, with docker compose --env-file .env.staging up -d. This only changes which file Compose uses for interpolation — it doesn’t affect env_file directives inside services, which you set explicitly per service.

    Q: Why does my variable show up as an empty string instead of failing?
    A: Compose treats an undefined variable referenced via ${VAR} as an empty string by default rather than erroring. Use ${VAR:?error message} syntax to make Compose fail loudly if a required variable is missing.

    Q: How do I check what values Compose is actually resolving?
    A: Run docker compose config to print the fully rendered configuration after all substitutions are applied. This is the fastest way to debug a misbehaving variable.

    Q: Should secrets go in environment or env_file?
    A: Neither is truly secure — both end up as plaintext inside the container’s environment. For production secrets, prefer Docker secrets, a vault service, or your cloud provider’s secret manager, and reserve environment/env_file for non-sensitive configuration.

    Q: Can I set default values if a variable isn’t defined?
    A: Yes. Use ${VAR:-default} syntax in your compose file, e.g. ${APP_PORT:-8080}, which falls back to 8080 if APP_PORT is unset or empty.

    Wrapping Up

    Getting comfortable with the interplay between environment, .env, and env_file removes an entire category of “works on my machine” bugs from your Compose workflow. Start with docker compose config any time behavior looks wrong, keep secrets out of files that touch version control, and layer compose files rather than duplicating them for environment-specific overrides. Once your configuration strategy is solid, the rest of your stack — networking, volumes, scaling — gets a lot easier to reason about.

  • Postgres Docker Compose: Full Setup Guide for 2026

    Postgres Docker Compose: The Complete Setup Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Running PostgreSQL locally used to mean installing packages, fighting with version conflicts, and cleaning up leftover config files. With postgres docker compose, you get a reproducible, disposable database environment that starts with one command and leaves no trace on your host system when you’re done.

    This guide covers everything from a minimal single-container setup to a production-ready configuration with persistent volumes, health checks, environment-based secrets, and automated backups. If you’re deploying containerized apps regularly, pair this with our Docker networking guide for a deeper understanding of how services talk to each other inside Compose.

    Why Use Docker Compose for Postgres

    Docker Compose turns a multi-step manual setup into a declarative YAML file. Instead of remembering flags for docker run, you define the database service, its volumes, networks, and environment variables once, then bring the whole stack up or down with a single command.

    This matters for a few practical reasons:

  • Reproducibility — the same docker-compose.yml produces an identical environment on your laptop, your CI runner, and your staging server.
  • Isolation — Postgres runs in its own container with its own filesystem, so it won’t collide with a system-installed version or another project’s database.
  • Easy teardowndocker compose down -v wipes the database cleanly, which is ideal for testing migrations or resetting a dev environment.
  • Multi-service orchestration — you can add your app server, Redis, pgAdmin, or a reverse proxy to the same file and manage them together.
  • If you’re new to containers generally, our Docker for beginners walkthrough covers the fundamentals before you dive into Compose specifics.

    Prerequisites

    Before starting, make sure you have:

  • Docker Engine 20.10+ installed
  • Docker Compose v2 (bundled with Docker Desktop and modern Docker Engine installs, invoked as docker compose, not the old standalone docker-compose)
  • Basic familiarity with YAML syntax
  • A terminal and a text editor
  • Check your versions with:

    docker --version
    docker compose version

    If docker compose version fails, consult the official Docker Compose installation docs for your platform.

    Basic Postgres Docker Compose Setup

    Start with a minimal working example. Create a project directory and a docker-compose.yml file:

    mkdir postgres-compose-demo && cd postgres-compose-demo
    touch docker-compose.yml

    Add this configuration:

    services:
      db:
        image: postgres:16
        restart: unless-stopped
        environment:
          POSTGRES_USER: appuser
          POSTGRES_PASSWORD: changeme
          POSTGRES_DB: appdb
        ports:
          - "5432:5432"
        volumes:
          - pgdata:/var/lib/postgresql/data
    
    volumes:
      pgdata:

    Bring it up:

    docker compose up -d

    Verify the container is running and healthy:

    docker compose ps
    docker compose logs db

    Connect using psql from your host (if installed) or directly inside the container:

    docker compose exec db psql -U appuser -d appdb

    That’s a fully working Postgres instance with a named volume for persistence. The pgdata volume survives container restarts and recreation, so your data isn’t lost when you run docker compose down (without the -v flag).

    Understanding the Volume Mapping

    The line pgdata:/var/lib/postgresql/data maps a Docker-managed named volume to Postgres’s internal data directory. This is the single most important line in the file — without it, every docker compose down followed by up wipes your database clean.

    There are two common approaches:

  • Named volumes (recommended): Docker manages the storage location, and you don’t need to worry about host filesystem permissions. Use pgdata:/var/lib/postgresql/data as shown above.
  • Bind mounts: You map a specific host directory, e.g. ./data:/var/lib/postgresql/data. This gives you direct filesystem access but can run into UID/GID permission mismatches between your host user and the container’s postgres user (UID 999 in the official image).
  • For most setups, named volumes are simpler and less error-prone. Reserve bind mounts for cases where you need to inspect or back up raw data files directly from the host.

    Environment Variables and Secrets

    Hardcoding passwords in docker-compose.yml is a common mistake that gets committed to git accidentally. Instead, use a .env file:

    # .env
    POSTGRES_USER=appuser
    POSTGRES_PASSWORD=a-much-stronger-password-here
    POSTGRES_DB=appdb

    Reference it in your Compose file:

    services:
      db:
        image: postgres:16
        restart: unless-stopped
        env_file:
          - .env
        ports:
          - "5432:5432"
        volumes:
          - pgdata:/var/lib/postgresql/data
    
    volumes:
      pgdata:

    Add .env to your .gitignore immediately:

    echo ".env" >> .gitignore

    For production deployments, consider Docker secrets or a dedicated secrets manager instead of plain .env files — environment variables are visible to anything that can inspect the running container (docker inspect), which is a real exposure risk on shared hosts.

    Adding Health Checks and a Custom Network

    A production-grade Compose file should confirm Postgres is actually accepting connections before dependent services (like your app) try to connect. Add a health check:

    services:
      db:
        image: postgres:16
        restart: unless-stopped
        env_file:
          - .env
        ports:
          - "5432:5432"
        volumes:
          - pgdata:/var/lib/postgresql/data
        healthcheck:
          test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
          interval: 10s
          timeout: 5s
          retries: 5
        networks:
          - backend
    
      app:
        build: .
        depends_on:
          db:
            condition: service_healthy
        environment:
          DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
        networks:
          - backend
    
    volumes:
      pgdata:
    
    networks:
      backend:

    The depends_on.condition: service_healthy clause ensures your app container waits for Postgres to actually be ready, not just started. This eliminates the classic “connection refused” error on the first boot of a multi-container stack.

    Notice the app connects to db, not localhost or 127.0.0.1. Compose automatically creates a DNS entry for each service name on the shared network, so containers reach each other by service name.

    Backups and Data Recovery

    Even with persistent volumes, you need actual backups — a corrupted volume or accidental docker compose down -v can still destroy your data. Use pg_dump inside the running container:

    docker compose exec db pg_dump -U appuser appdb > backup.sql

    Restore into a fresh container:

    cat backup.sql | docker compose exec -T db psql -U appuser -d appdb

    For automated, scheduled backups, add a lightweight sidecar service using a cron-based image, or run pg_dump from a host cron job that shells into the container. If you’re managing this on a remote VPS, our Linux server hardening checklist covers securing SSH and cron jobs so scheduled backup scripts aren’t a security liability.

    For larger datasets, consider pg_basebackup for physical backups, which are faster to restore than logical pg_dump exports. The official PostgreSQL backup documentation covers the tradeoffs between logical and physical backup strategies in detail.

    Running Multiple Postgres Versions Side by Side

    One underrated benefit of Docker Compose is testing against multiple Postgres major versions without touching your host system. Just change the image tag per project:

    services:
      db-15:
        image: postgres:15
        ports:
          - "5433:5432"
        volumes:
          - pgdata15:/var/lib/postgresql/data
    
      db-16:
        image: postgres:16
        ports:
          - "5434:5432"
        volumes:
          - pgdata16:/var/lib/postgresql/data
    
    volumes:
      pgdata15:
      pgdata16:

    This is invaluable when validating an upgrade path before touching production. Spin up both versions, dump data from the old one, restore into the new one, and confirm your app behaves identically before committing to the upgrade.

    Choosing Where to Host Your Postgres + Compose Stack

    Once your Compose setup works locally, you’ll want to deploy it to a real server. A small VPS with a few gigabytes of RAM handles most small-to-medium Postgres workloads comfortably. Providers like DigitalOcean and Hetzner both offer affordable droplets/instances that run Docker Compose stacks well out of the box — DigitalOcean’s managed load balancers and snapshots are convenient if you want less manual ops work, while Hetzner tends to offer more raw compute per dollar if you’re comfortable managing more yourself.

    Whichever you choose, make sure to configure a firewall so port 5432 isn’t exposed to the public internet — bind it to 127.0.0.1:5432:5432 instead of 5432:5432 if only local services need access, and put Postgres behind a private network or SSH tunnel for remote administration.

    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Does Docker Compose persist Postgres data by default?
    No. Without an explicit volume mapping, data is stored inside the container’s writable layer and is lost when the container is removed. Always define a named volume or bind mount for /var/lib/postgresql/data.

    What’s the difference between docker compose down and docker compose down -v?
    docker compose down stops and removes containers and networks but leaves named volumes intact. Adding -v also deletes the volumes, which permanently erases your database data — use it carefully.

    Can I run pgAdmin alongside Postgres in the same Compose file?
    Yes. Add a pgadmin service using the dpage/pgadmin4 image on the same network as your db service, then connect using the service name db as the host in pgAdmin’s connection settings.

    Why does my app container fail to connect to Postgres on startup?
    This usually means the app tried to connect before Postgres finished initializing. Add a healthcheck to the db service and use depends_on.condition: service_healthy on the app service to fix the race condition.

    How do I change the Postgres password after the container has already initialized?
    Environment variables like POSTGRES_PASSWORD only apply on first initialization of an empty data directory. To change the password afterward, connect with psql and run ALTER USER appuser WITH PASSWORD 'newpassword'; directly.

    Is it safe to expose port 5432 publicly?
    Generally no. Bind the port to localhost only, or better, don’t publish it at all and let other containers reach Postgres over the internal Compose network by service name.

    Networking Between Services

    By default, Compose creates a private network for all services defined in the same file. The app service above reaches PostgreSQL simply by using db as the hostname — Docker’s internal DNS resolves it automatically. You don’t need to expose port 5432 to the host at all unless you want external tools (like a local GUI client) to connect directly.

    If you do need host access for debugging with a tool like pgAdmin or DBeaver, keep the port mapping but restrict it:

    ports:
      - "127.0.0.1:5432:5432"

    Binding to 127.0.0.1 instead of 0.0.0.0 prevents the database port from being exposed on your machine’s public network interface — a small change that matters a lot if you’re running this on a cloud VPS rather than a laptop.

    Multi-Container Example with pgAdmin

    For local development, it’s often convenient to run a web-based admin UI alongside PostgreSQL:

    services:
      db:
        image: postgres:16
        restart: unless-stopped
        env_file:
          - .env
        volumes:
          - pgdata:/var/lib/postgresql/data
        healthcheck:
          test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
          interval: 10s
          timeout: 5s
          retries: 5
    
      pgadmin:
        image: dpage/pgadmin4:latest
        restart: unless-stopped
        environment:
          PGADMIN_DEFAULT_EMAIL: [email protected]
          PGADMIN_DEFAULT_PASSWORD: changeme
        ports:
          - "127.0.0.1:8080:80"
        depends_on:
          - db
    
    volumes:
      pgdata:

    Access pgAdmin at http://localhost:8080, then add a new server connection pointing to host db, port 5432, using the credentials from your .env file.

    Production Hardening Checklist

    Before you point real traffic at a containerized PostgreSQL instance, review these items:

  • Set strong, unique passwords and rotate them periodically.
  • Bind the host port to 127.0.0.1 or remove the port mapping entirely if only internal services need access.
  • Enable regular automated backups with off-host storage (S3, Backblaze, or similar).
  • Use restart: unless-stopped so the database recovers automatically after a host reboot.
  • Monitor disk usage on the volume — PostgreSQL doesn’t cap growth automatically.
  • Pin the image tag (postgres:16.3, not just postgres:latest) to avoid surprise upgrades.
  • Consider running PostgreSQL on a dedicated, appropriately sized VPS rather than co-locating it with CPU-heavy application containers.
  • If you’re hosting this yourself rather than using a managed database, providers like Hetzner offer solid price-to-performance ratios for dedicated database VPS instances, which matters since PostgreSQL performance is heavily tied to disk I/O and available RAM for caching.

    Resource Limits

    Compose lets you cap CPU and memory so a runaway query doesn’t starve other containers on the same host:

    services:
      db:
        image: postgres:16
        deploy:
          resources:
            limits:
              cpus: "2"
              memory: 2G

    Note that deploy.resources is fully honored under Docker Swarm; under plain docker compose up, Compose v2 does apply these limits on recent Docker Engine versions, but it’s worth verifying with docker stats after deployment rather than assuming it’s enforced.

    Common Errors and Fixes

    A few issues come up repeatedly when people first containerize PostgreSQL:

  • “role does not exist” — usually means the POSTGRES_USER env var wasn’t set before the volume was initialized for the first time. Environment variables only take effect on first container creation; changing them later has no effect on an existing volume.
  • “connection refused” — the health check isn’t wired up, and your app is connecting before PostgreSQL finishes booting.
  • Permission denied on data directory — usually a bind-mount UID mismatch between host and container. Named volumes avoid this entirely.
  • Data vanished after docker compose down — someone ran docker compose down -v and deleted the volume along with the containers.
  • Wrapping Up

    A well-structured docker-compose.yml turns Postgres from a fragile local install into a portable, version-controlled piece of infrastructure. Start with the minimal setup, add named volumes early, move secrets into a .env file, and layer on health checks once you’re connecting other services. From there, backups and a hardened network configuration are the last steps between a dev setup and something you’d trust in production.

    If you’re scaling this stack beyond a single server, revisit our Docker networking guide for multi-host and reverse proxy patterns that build directly on the concepts covered here.

  • Docker Compose Secrets: Secure Config Management Guide

    Docker Compose Secrets: How to Stop Leaking Credentials in Your Containers

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever pasted a database password directly into a docker-compose.yml file under environment:, you’ve already leaked it. It’s sitting in plaintext in your git history, your shell history, and probably a Slack thread from six months ago. Docker Compose secrets exist specifically to fix this problem, and most teams still aren’t using them correctly — or at all.

    This guide covers what Docker Compose secrets actually are, how they differ from environment variables, and how to wire them into real services like Postgres, Redis, and custom apps. We’ll also cover the common mistakes that make people think secrets “don’t work” in Compose (they usually do — you’re just using the wrong driver mode).

    Why Environment Variables Aren’t Enough

    The default instinct for passing a database password to a container is:

    services:
      db:
        image: postgres:16
        environment:
          POSTGRES_PASSWORD: hunter2

    This works, but it has real problems:

  • The value is visible in docker inspect <container> output to anyone with Docker socket access.
  • It’s committed to version control unless you’re careful with .gitignore, and even then it often ends up in CI logs.
  • It shows up in process listings and container logs if any tooling echoes environment variables during startup.
  • Every service that needs the secret gets it injected as a full environment variable, widening the blast radius if one container is compromised.
  • Docker Compose secrets solve this by mounting sensitive values as files inside the container’s filesystem, typically under /run/secrets/, rather than injecting them into the environment. Only the services that explicitly declare a dependency on a secret can read it, and the value never appears in docker inspect or docker compose config output.

    How Docker Compose Secrets Actually Work

    A secret in Compose is defined at the top level of your docker-compose.yml, then attached to individual services. There are two ways to source a secret:

    1. File-based — the secret’s contents come from a local file on the host.
    2. External — the secret already exists (commonly in Docker Swarm) and Compose just references it by name.

    Here’s the file-based approach, which is what most single-host and local dev setups use:

    services:
      db:
        image: postgres:16
        environment:
          POSTGRES_PASSWORD_FILE: /run/secrets/db_password
        secrets:
          - db_password
    
    secrets:
      db_password:
        file: ./secrets/db_password.txt

    Notice the POSTGRES_PASSWORD_FILE variable instead of POSTGRES_PASSWORD. The official Postgres image (and many others) supports a _FILE suffix convention: instead of reading the value directly from an environment variable, the entrypoint script reads it from the file path you provide. Inside the container, Compose mounts the secret at /run/secrets/db_password, readable only by root by default with 0444 permissions.

    Create the secret file locally:

    mkdir -p secrets
    echo "hunter2" > secrets/db_password.txt
    chmod 600 secrets/db_password.txt

    Then add secrets/ to your .gitignore immediately — this is the single most common way secrets leak back into git despite people setting up the whole system correctly.

    echo "secrets/" >> .gitignore

    Using Secrets in Your Own Application Code

    Not every image supports the _FILE convention, so for custom apps you’ll read the secret file directly. Here’s a minimal Node.js example:

    const fs = require('fs');
    
    function readSecret(name) {
      const path = `/run/secrets/${name}`;
      return fs.readFileSync(path, 'utf8').trim();
    }
    
    const dbPassword = readSecret('db_password');
    const apiKey = readSecret('third_party_api_key');

    And the equivalent Python pattern:

    from pathlib import Path
    
    def read_secret(name: str) -> str:
        return Path(f"/run/secrets/{name}").read_text().strip()
    
    db_password = read_secret("db_password")

    This pattern works identically whether you’re running plain docker compose up locally or deploying to a Swarm cluster, which makes it worth adopting even if you think you’ll never touch Swarm.

    Multiple Secrets Across Multiple Services

    A realistic stack usually needs several secrets shared across services with different access levels. Here’s a fuller example with Postgres, Redis with a password, and an app container that needs both plus a third-party API key:

    services:
      app:
        build: .
        depends_on:
          - db
          - cache
        secrets:
          - db_password
          - api_key
    
      db:
        image: postgres:16
        environment:
          POSTGRES_PASSWORD_FILE: /run/secrets/db_password
        secrets:
          - db_password
    
      cache:
        image: redis:7
        command: ["redis-server", "--requirepass-file", "/run/secrets/redis_password"]
        secrets:
          - redis_password
    
    secrets:
      db_password:
        file: ./secrets/db_password.txt
      redis_password:
        file: ./secrets/redis_password.txt
      api_key:
        file: ./secrets/api_key.txt

    Note that redis-server doesn’t natively support a --requirepass-file flag out of the box in older versions — check your image’s documentation, and if it’s unsupported, wrap the container’s entrypoint with a small shell script that reads the file and passes it as --requirepass "$(cat /run/secrets/redis_password)" instead.

    Each service only lists the secrets it actually needs under its own secrets: key. The app service can read db_password and api_key, but has no access to redis_password unless you explicitly add it — this is the principle of least privilege applied to container secrets.

    Secrets vs. .env Files: When to Use Which

    A lot of confusion comes from Compose also supporting .env files and env_file: for non-sensitive configuration. These are different tools for different jobs:

  • .env / env_file — good for non-sensitive configuration like LOG_LEVEL=debug or APP_PORT=3000. Values still land in the environment and are visible via docker inspect.
  • secrets: — good for anything sensitive: passwords, API keys, TLS private keys, tokens. Values are mounted as files, not injected into the process environment, and are excluded from docker compose config output when using external secret backends.
  • A reasonable rule: if leaking the value in a support ticket or screen share would be embarrassing or dangerous, it belongs in secrets:, not .env.

    For teams running this on a VPS rather than a laptop, pairing this setup with a hardened host matters just as much as the Compose file itself. If you’re provisioning that host yourself, a provider like DigitalOcean or Hetzner gives you a clean Ubuntu box to configure a proper firewall and non-root Docker setup on — we cover the baseline hardening steps in our Docker security checklist.

    Secrets in Docker Swarm vs. Plain Compose

    It’s worth being explicit about a limitation: in plain docker compose up (no Swarm), secrets are still just bind-mounted files under the hood, not encrypted at rest by Docker itself. The security benefit is about avoiding environment variable exposure and docker inspect leakage, not full encryption.

    In Docker Swarm mode, secrets are genuinely encrypted at rest in the Raft log and only decrypted in memory on the nodes running containers that need them. If you need that stronger guarantee, you’d deploy with docker stack deploy instead of docker compose up, using the same secrets: syntax:

    docker swarm init
    docker stack deploy -c docker-compose.yml mystack

    For most single-host deployments — a small VPS running a handful of services — plain Compose secrets are sufficient as long as you also lock down file permissions on the host and restrict who has SSH and Docker group access. For anything handling real customer data across multiple nodes, Swarm secrets or a dedicated secrets manager like HashiCorp Vault is the more defensible choice.

    Rotating Secrets Without Downtime

    One underrated benefit of file-based secrets is that rotation doesn’t require rebuilding your image. To rotate the db_password:

    echo "new-strong-password" > secrets/db_password.txt
    docker compose up -d --force-recreate db app

    This recreates only the affected containers with the new secret mounted, without touching unrelated services. Compare that to environment-variable-based secrets baked into a .env file referenced by many services — rotating those often means restarting everything and hoping you didn’t miss a reference somewhere.

    If you’re monitoring uptime during a rotation like this, a service like BetterStack will catch it immediately if a recreated container fails health checks, which is worth having in place before you start touching production credentials.

    Common Mistakes to Avoid

  • Committing the secrets file anyway. .gitignore only prevents future commits — if secrets/db_password.txt was ever committed, it’s in your git history forever unless you rewrite it with git filter-repo or BFG Repo-Cleaner.
  • Using POSTGRES_PASSWORD and POSTGRES_PASSWORD_FILE together. Some images will silently prefer one over the other, or error out. Pick one convention per variable.
  • Forgetting file permissions on the host. A secret file with 644 permissions readable by any local user defeats the purpose. Use chmod 600.
  • Assuming plain Compose secrets are encrypted at rest. They aren’t, only Swarm secrets are — treat host-level access control as the real security boundary for single-host setups.
  • Hardcoding fallback defaults in application code. If your app falls back to a hardcoded password when the secret file is missing, you’ve built a silent security hole that only shows up when something goes wrong with the mount.
  • We go into more depth on locking down the host itself, including firewall rules and non-root container users, in our companion piece on hardening a Docker host on a VPS.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Do Docker Compose secrets work without Docker Swarm?
    Yes. File-based secrets work fine with plain docker compose up. They’re mounted as read-only files under /run/secrets/ inside the container. The main difference with Swarm is that Swarm additionally encrypts secrets at rest in its cluster state; plain Compose does not.

    Why isn’t my POSTGRES_PASSWORD_FILE being picked up?
    Check that you’re not also setting POSTGRES_PASSWORD in the same service — many official images check for the plain variable first and will ignore the _FILE variant, or throw an error if both are set. Also confirm the secret is actually listed under that service’s secrets: key, not just in the top-level secrets: block.

    Can I use environment variables to reference secret values directly?
    Not securely, no. If you do MY_SECRET: ${SOME_SECRET_VALUE} in your Compose file, the value still gets injected as a plain environment variable and shows up in docker inspect. The whole point of the secrets: mechanism is to avoid that path by mounting a file instead.

    How do I pass secrets in a CI/CD pipeline without writing them to disk?
    Most CI systems (GitHub Actions, GitLab CI) let you inject secrets as masked environment variables, then have your pipeline write them to the expected secret file paths just before running docker compose up, and clean them up afterward. Avoid echoing the values in any pipeline logs.

    Is it safe to store secrets files in a Docker volume instead of a bind mount?
    You can, but it adds complexity for little benefit in most single-host setups. A bind-mounted file on the host with restrictive permissions is simpler to audit and rotate than a value living inside a named volume.

    What happens to secrets when a container is removed?
    The mounted secret file inside the container is removed along with the container’s filesystem. The source file on the host (referenced in the file: path) is untouched, so recreating the container remounts the same secret unless you’ve changed the source file.

    Wrapping Up

    Docker Compose secrets aren’t complicated once you get past the initial unfamiliarity — the pattern is always: define the secret at the top level, attach it to the services that need it, and read it from /run/secrets/<name> inside the container instead of an environment variable. Combined with proper .gitignore hygiene, restrictive file permissions, and a hardened host, this closes off one of the most common and preventable ways credentials leak out of containerized applications.

    If you’re setting this up on a fresh VPS, take the time to also review basic firewall and SSH hardening before you deploy anything with real credentials attached — a secrets-aware Compose file doesn’t help much if the host itself is wide open.