Category: Docker Compose

  • Mongodb Docker Compose

    MongoDB Docker Compose: A Complete Setup and Operations Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Running MongoDB in a container is one of the fastest ways to get a working document database for local development or a small production service. This guide walks through a real mongodb docker compose configuration, from a minimal single-node setup to authentication, persistent storage, replica sets, and backups, so you have a working reference rather than a toy example.

    MongoDB pairs naturally with Compose because the database, its configuration, and any supporting services (an admin UI, an app server, a replica-set init container) can all be described in one file and brought up with a single command. That’s the core appeal of a mongodb docker compose workflow: reproducibility. Anyone on the team runs docker compose up and gets the same database, same version, same configuration, every time.

    Why Use MongoDB with Docker Compose

    Before diving into YAML, it’s worth being clear about what problem this actually solves. Installing MongoDB natively on a laptop or server means managing a system service, dealing with OS-specific package quirks, and hoping the version matches what’s running in staging or production. A mongodb docker compose file sidesteps all of that:

  • The exact MongoDB version is pinned in the image tag, not “whatever the package manager installed.”
  • Configuration lives in version control alongside the application code, not scattered across /etc/mongod.conf on a machine nobody remembers setting up.
  • Tearing down and rebuilding the environment is a single command, which makes it trivial to test upgrades or reproduce a bug from a clean state.
  • Multiple services (MongoDB, a caching layer, the application itself) can be orchestrated together with defined startup order and shared networking.
  • This isn’t unique to MongoDB — the same reasoning applies to running Postgres in Docker Compose or Redis in Docker Compose — but MongoDB has a few of its own operational wrinkles (replica sets, its own authentication model, WiredTiger storage behavior) that are worth covering specifically.

    When Compose Is the Right Tool

    Docker Compose is well suited to local development, single-node staging environments, and small production deployments running on a single host. If you need multi-host orchestration, automated failover across machines, or horizontal scaling driven by a scheduler, you’re in Kubernetes territory instead — see our comparison of Kubernetes vs Docker Compose for where that line sits. For most teams, a single well-configured MongoDB container (or a small replica set of three containers) on one VPS is more than sufficient, and a lot simpler to operate.

    A Minimal MongoDB Docker Compose Setup

    Here’s a minimal, working mongodb docker compose file. It uses the official image, exposes the default port, and persists data to a named volume so container restarts don’t wipe your database.

    services:
      mongodb:
        image: mongo:7
        container_name: mongodb
        restart: unless-stopped
        ports:
          - "27017:27017"
        volumes:
          - mongo_data:/data/db
    
    volumes:
      mongo_data:

    Run it with:

    docker compose up -d

    Check that it’s healthy:

    docker compose ps
    docker compose logs mongodb

    Connect from the host using mongosh (or any MongoDB client) at mongodb://localhost:27017. That’s the whole minimal case — a single service, one volume, one port. Everything past this point is adding the pieces you actually need for anything beyond a throwaway sandbox.

    Adding Authentication to Your MongoDB Docker Compose File

    The minimal example above has no authentication at all, which is fine for a completely isolated local sandbox but not acceptable for anything reachable from a network. The official MongoDB image supports root-user bootstrapping through environment variables, which is the standard way to enable auth in a mongodb docker compose setup.

    services:
      mongodb:
        image: mongo:7
        container_name: mongodb
        restart: unless-stopped
        environment:
          MONGO_INITDB_ROOT_USERNAME: admin
          MONGO_INITDB_ROOT_PASSWORD: ${MONGO_ROOT_PASSWORD}
        ports:
          - "27017:27017"
        volumes:
          - mongo_data:/data/db
    
    volumes:
      mongo_data:

    Note the ${MONGO_ROOT_PASSWORD} reference instead of a hardcoded password. That value should live in a .env file next to your compose file, which Compose reads automatically — never commit real credentials to version control. Our guide on managing Docker Compose environment variables covers this pattern in more depth, and if you need to store the credential itself more securely (rather than a plaintext .env value), Docker Compose secrets is the next step up.

    Restricting Network Exposure

    Once auth is in place, also reconsider whether MongoDB needs to be reachable from outside the Docker host at all. If only your application container talks to it, drop the ports mapping entirely and rely on Compose’s internal network — services on the same Compose network can reach each other by service name (mongodb:27017) without any port being published to the host. This is a meaningful security improvement with zero functional cost when nothing external needs a direct connection.

    services:
      mongodb:
        image: mongo:7
        restart: unless-stopped
        environment:
          MONGO_INITDB_ROOT_USERNAME: admin
          MONGO_INITDB_ROOT_PASSWORD: ${MONGO_ROOT_PASSWORD}
        volumes:
          - mongo_data:/data/db
        networks:
          - backend
    
      app:
        build: .
        environment:
          MONGO_URI: mongodb://admin:${MONGO_ROOT_PASSWORD}@mongodb:27017
        depends_on:
          - mongodb
        networks:
          - backend
    
    networks:
      backend:
    
    volumes:
      mongo_data:

    Creating Application-Specific Users

    Using the root user for your application isn’t good practice. MongoDB’s image supports an initialization script directory (/docker-entrypoint-initdb.d) that runs once, on first container startup with an empty data directory, letting you create a scoped user for your actual application database:

    // init-mongo.js
    db = db.getSiblingDB('appdb');
    db.createUser({
      user: 'appuser',
      pwd: process.env.MONGO_APP_PASSWORD,
      roles: [{ role: 'readWrite', db: 'appdb' }],
    });

    Mount it as a volume:

        volumes:
          - mongo_data:/data/db
          - ./init-mongo.js:/docker-entrypoint-initdb.d/init-mongo.js:ro

    This script only runs when the data volume is empty — if you’re modifying user setup after the fact, you’ll need to remove the volume or apply the change manually via mongosh.

    Persisting Data Correctly

    Data persistence is where a lot of first-time mongodb docker compose setups go wrong. MongoDB stores its data files under /data/db inside the container by default, and that directory must be backed by a Docker volume — otherwise every docker compose down (or container recreation) wipes the database. The examples above already do this correctly with a named volume, but it’s worth understanding the alternatives:

  • Named volumes (mongo_data:/data/db) — Docker manages the storage location. This is the recommended default; it works consistently across host operating systems and is easy to back up with docker run --volumes-from.
  • Bind mounts (./data:/data/db) — you control the exact host path. Useful if you need direct filesystem access to the data files, but MongoDB’s WiredTiger storage engine can behave inconsistently with certain host filesystems (notably some network-mounted or non-POSIX-compliant filesystems), so test this carefully before relying on it in production.
  • tmpfs mounts — data lives in memory only, useful for ephemeral test databases that should never persist, never for anything you care about keeping.
  • Whichever you choose, confirm persistence actually works before trusting it: bring the stack up, insert a document, run docker compose down (not down -v, which deletes volumes), bring it back up, and confirm the document is still there.

    Running a MongoDB Replica Set with Docker Compose

    A single MongoDB instance is a single point of failure and, notably, transactions require a replica set even with just one member. Many applications that use MongoDB transactions need at least a single-node replica set even in development. Here’s a three-node replica set defined in one mongodb docker compose file:

    services:
      mongo1:
        image: mongo:7
        command: ["--replSet", "rs0", "--bind_ip_all"]
        volumes:
          - mongo1_data:/data/db
        networks:
          - mongo_cluster
    
      mongo2:
        image: mongo:7
        command: ["--replSet", "rs0", "--bind_ip_all"]
        volumes:
          - mongo2_data:/data/db
        networks:
          - mongo_cluster
    
      mongo3:
        image: mongo:7
        command: ["--replSet", "rs0", "--bind_ip_all"]
        volumes:
          - mongo3_data:/data/db
        networks:
          - mongo_cluster
    
    networks:
      mongo_cluster:
    
    volumes:
      mongo1_data:
      mongo2_data:
      mongo3_data:

    After bringing this up, initialize the replica set once from inside any of the containers:

    docker compose exec mongo1 mongosh --eval '
    rs.initiate({
      _id: "rs0",
      members: [
        { _id: 0, host: "mongo1:27017" },
        { _id: 1, host: "mongo2:27017" },
        { _id: 2, host: "mongo3:27017" }
      ]
    })'

    Health Checks and Startup Ordering

    Compose’s depends_on only waits for a container to start, not for MongoDB inside it to actually be ready to accept connections. For any service that depends on MongoDB being available (your app, or an init script), add a proper health check:

        healthcheck:
          test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
          interval: 10s
          timeout: 5s
          retries: 5

    Then reference it with depends_on: { mongodb: { condition: service_healthy } } on the dependent service so it actually waits for a real ready state rather than just container start.

    Backups and Debugging

    A running mongodb docker compose stack still needs a backup strategy — persistence against docker compose down doesn’t protect against disk failure, accidental docker compose down -v, or a bad migration. mongodump/mongorestore work the same way inside a container as anywhere else:

    docker compose exec mongodb mongodump --out /data/backup
    docker cp mongodb:/data/backup ./backup-$(date +%F)

    When something isn’t working, docker compose logs is the first place to look — the same debugging approach covered in our Docker Compose logs guide applies directly to MongoDB containers, including following logs live with -f and filtering by service name in a multi-container stack.

    If you need to rebuild the image after changing a Dockerfile that wraps the MongoDB base image (for example, to bake in custom config), see our Docker Compose rebuild guide for the difference between up --build and a full build --no-cache.


    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Does MongoDB in Docker Compose persist data by default?
    No. Without an explicit volume mounted at /data/db, all data is lost when the container is removed. Always define a named volume or bind mount for that path in any mongodb docker compose configuration you intend to keep data in.

    Can I run MongoDB and my application in the same Compose file?
    Yes, and it’s the standard pattern. Define both services in the same docker-compose.yml, put them on the same Compose network, and reference MongoDB from your app using the service name as the hostname (e.g., mongodb://mongodb:27017) rather than localhost.

    Why does my application fail to connect immediately after docker compose up?
    MongoDB takes a few seconds to initialize before it accepts connections, and Compose’s default depends_on doesn’t wait for that. Add a healthcheck to the MongoDB service and a condition: service_healthy dependency on the app service to fix race conditions at startup.

    Do I need a replica set for local development?
    Only if your application code uses MongoDB transactions or change streams, both of which require one. Otherwise, a single-node instance without --replSet is simpler and sufficient for most local development.

    Conclusion

    A solid mongodb docker compose setup starts minimal — one service, one volume, one port — and grows to include authentication, network isolation, and eventually a replica set as your requirements demand it. The key operational habits are the same regardless of scale: never skip the data volume, never hardcode credentials into the compose file, and verify persistence and health checks actually work before relying on them. For further reference on MongoDB’s own configuration options and replica set behavior, the official MongoDB documentation and Docker’s Compose file reference are both worth keeping bookmarked while you build this out. If you’re deploying this stack on your own server rather than a managed platform, a VPS provider like DigitalOcean is a reasonable place to host a single-node or three-node MongoDB Compose deployment.

  • Mongo Docker Compose

    Mongo Docker Compose: The Complete Setup Guide for 2026

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Running MongoDB in a container is one of the fastest ways to get a working development or staging database without touching a package manager or wrestling with system service files. A mongo docker compose setup lets you define the database, its volumes, its network, and its environment variables in a single declarative file, then bring the whole thing up or down with one command. This guide walks through a real, working configuration, explains the options that actually matter, and covers the mistakes that trip up most people the first time they try it.

    Whether you’re spinning up a local instance for a Node.js app, provisioning a staging replica set, or just want a disposable database for testing, the patterns below apply. We’ll build the compose file piece by piece, then look at authentication, persistence, networking, backups, and troubleshooting.

    Why Use Mongo Docker Compose Instead of a Manual Install

    Installing MongoDB directly on a host means managing its systemd unit, its config file location, its log rotation, and its upgrade path independently of everything else on the machine. A mongo docker compose file replaces all of that with a single YAML document that lives next to your application code, gets checked into version control, and behaves identically on your laptop, your CI runner, and your production VPS.

    The core benefits:

  • Reproducibility — anyone on the team runs docker compose up and gets the exact same MongoDB version and configuration.
  • Isolation — the database process and its dependencies don’t pollute the host system.
  • Easy teardowndocker compose down removes the container cleanly, and you can choose whether volumes survive.
  • Multi-service coordination — MongoDB can sit alongside your app server, a cache, or a message queue in the same file, all sharing a private network.
  • This isn’t unique to MongoDB — the same reasoning applies to Postgres Docker Compose and Redis Docker Compose setups, and if you’re choosing between a full install and a container in the first place, it’s worth reading up on Dockerfile vs Docker Compose to understand where each tool fits.

    Building a Basic Mongo Docker Compose File

    Start with the minimal version that gets a single MongoDB instance running with a named volume for persistence.

    services:
      mongo:
        image: mongo:7
        container_name: mongo_db
        restart: unless-stopped
        ports:
          - "27017:27017"
        environment:
          MONGO_INITDB_ROOT_USERNAME: admin
          MONGO_INITDB_ROOT_PASSWORD: changeme
        volumes:
          - mongo_data:/data/db
    
    volumes:
      mongo_data:

    Bring it up with:

    docker compose up -d

    That’s a complete, working mongo docker compose stack. The mongo_data named volume ensures your data survives container restarts and even docker compose down (as long as you don’t pass -v). The official image comes from Docker Hub and is maintained upstream, so pinning a major version like mongo:7 rather than mongo:latest avoids surprise upgrades when you rebuild months later.

    Choosing the Right Image Tag

    Always pin to at least a major version tag. mongo:latest will silently pull a newer major release the next time you run docker compose pull, which can break your application if a driver-incompatible change ships. mongo:7 or mongo:7.0 gives you predictable behavior while still receiving patch updates within that line. Check the MongoDB documentation for the current supported version matrix before deciding which line to track long-term.

    Exposing or Hiding the Port

    The ports mapping in the example above exposes MongoDB on the host’s 27017, which is convenient for local development tools like MongoDB Compass or mongosh connecting from outside the container. In a production mongo docker compose deployment where only your application container needs access, omit the ports block entirely and rely on Docker’s internal network — the app container can still reach mongo:27017 by service name, but nothing outside the Docker host can.

    Authentication and Security in Mongo Docker Compose

    Never run MongoDB in production without authentication enabled. The MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD environment variables shown above are only read the very first time the container initializes an empty data directory — changing them later has no effect until you wipe the volume, which is a common source of confusion.

    A more robust pattern separates secrets from the compose file itself:

    services:
      mongo:
        image: mongo:7
        restart: unless-stopped
        env_file:
          - .env
        volumes:
          - mongo_data:/data/db
    
    volumes:
      mongo_data:

    # .env
    MONGO_INITDB_ROOT_USERNAME=admin
    MONGO_INITDB_ROOT_PASSWORD=a-strong-generated-password

    This keeps credentials out of version control if .env is gitignored. For a deeper look at handling variables this way, see this site’s guide on Docker Compose Env and the related guide on Docker Compose Environment Variables. If you need to manage credentials more formally — separate secret files mounted read-only rather than plain environment variables — the Docker Compose Secrets guide covers that pattern in detail, and it applies just as well to a mongo docker compose stack as it does to any other service.

    Creating Application-Specific Users

    Running everything as the root MongoDB user is bad practice once you move past local experimentation. Use an init script mounted into the container’s entrypoint directory to create a scoped user for your application on first boot:

    services:
      mongo:
        image: mongo:7
        restart: unless-stopped
        environment:
          MONGO_INITDB_ROOT_USERNAME: admin
          MONGO_INITDB_ROOT_PASSWORD: changeme
        volumes:
          - mongo_data:/data/db
          - ./init-mongo.js:/docker-entrypoint-initdb.d/init-mongo.js:ro
    
    volumes:
      mongo_data:

    // init-mongo.js
    db = db.getSiblingDB('app_db');
    db.createUser({
      user: 'app_user',
      pwd: 'app_user_password',
      roles: [{ role: 'readWrite', db: 'app_db' }]
    });

    Any .js or .sh file dropped into /docker-entrypoint-initdb.d/ runs automatically the first time the container initializes an empty /data/db directory, exactly like the equivalent mechanism in the official Postgres image.

    Connecting an Application to Mongo Docker Compose

    The real value of a mongo docker compose setup shows up when your application container joins the same Docker network and connects by service name instead of an IP address or localhost.

    services:
      app:
        build: .
        restart: unless-stopped
        depends_on:
          - mongo
        environment:
          MONGO_URI: mongodb://app_user:app_user_password@mongo:27017/app_db
        ports:
          - "3000:3000"
    
      mongo:
        image: mongo:7
        restart: unless-stopped
        environment:
          MONGO_INITDB_ROOT_USERNAME: admin
          MONGO_INITDB_ROOT_PASSWORD: changeme
        volumes:
          - mongo_data:/data/db
    
    volumes:
      mongo_data:

    Here, mongo in the connection string resolves via Docker’s internal DNS to the database container — no port mapping to the host is even necessary for the app to reach it. depends_on controls startup order but does not wait for MongoDB to actually finish initializing, so your application code should still implement connection retry logic on boot, particularly for the very first container start when the database is creating its data files.

    Health Checks for Reliable Startup

    Because depends_on alone doesn’t guarantee MongoDB is ready to accept connections, add a health check so dependent services can wait for a real ready signal:

    services:
      mongo:
        image: mongo:7
        restart: unless-stopped
        environment:
          MONGO_INITDB_ROOT_USERNAME: admin
          MONGO_INITDB_ROOT_PASSWORD: changeme
        volumes:
          - mongo_data:/data/db
        healthcheck:
          test: echo 'db.runCommand("ping").ok' | mongosh --quiet
          interval: 10s
          timeout: 5s
          retries: 5
    
      app:
        build: .
        depends_on:
          mongo:
            condition: service_healthy
    
    volumes:
      mongo_data:

    With condition: service_healthy, Compose won’t start the app container until MongoDB’s health check passes, which eliminates a whole class of “connection refused on first boot” errors.

    Debugging and Managing a Mongo Docker Compose Stack

    Once the stack is running, a handful of commands cover almost everything you’ll need day to day.

    # View live logs from the mongo service
    docker compose logs -f mongo
    
    # Open a mongosh shell inside the running container
    docker compose exec mongo mongosh -u admin -p
    
    # Check container status
    docker compose ps
    
    # Rebuild and restart after a compose file change
    docker compose up -d --build

    If logs aren’t giving you enough context, this site’s Docker Compose Logs guide and the related Docker Compose Logging article go deeper into log drivers and formatting options. And if you change the image version or add init scripts and need the container to pick up the changes cleanly, the Docker Compose Rebuild guide walks through the difference between restart, up --build, and a full down/up cycle.

    Backing Up and Restoring Data

    The named volume protects against container removal, but it doesn’t protect against disk failure or accidental docker volume rm. Use mongodump and mongorestore from inside the running container for portable backups:

    # Backup
    docker compose exec mongo mongodump --username admin --password changeme 
      --authenticationDatabase admin --archive=/data/db/backup.archive
    
    # Copy the archive out to the host
    docker cp mongo_db:/data/db/backup.archive ./backup.archive
    
    # Restore into a fresh container
    docker cp ./backup.archive mongo_db:/data/db/backup.archive
    docker compose exec mongo mongorestore --username admin --password changeme 
      --authenticationDatabase admin --archive=/data/db/backup.archive

    Schedule this as a cron job or an n8n workflow if you’re already running automation infrastructure — for teams doing broader workflow automation around their Docker stack, see n8n Self Hosted for a comparable containerized setup pattern.

    Shutting Down Cleanly

    When you’re done with a stack, understand the difference between stopping and removing it:

    docker compose stop        # stops containers, keeps volumes and networks
    docker compose down        # removes containers and networks, volumes persist
    docker compose down -v     # removes containers, networks, AND volumes (data loss)

    The full breakdown of these options, including what happens to networks and orphaned containers, is covered in Docker Compose Down — worth reading before you run any of these in a shared environment, since -v is irreversible without a backup.

    Where to Host Your Mongo Docker Compose Stack

    For anything beyond local development, you’ll want a VPS with enough memory and disk I/O to handle MongoDB’s working set comfortably — MongoDB is memory-hungry by design since it memory-maps its data files for performance. A provider like DigitalOcean or Hetzner offers straightforward block-storage-backed instances that work well for a single-node or small replica-set mongo docker compose deployment. Whichever provider you choose, mount your data volume on a disk with predictable I/O rather than ephemeral storage, and monitor memory usage closely as your working set grows.


    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Does a mongo docker compose setup support replica sets?
    Yes, but it requires additional configuration beyond a single service block — each replica set member needs its own service, a shared Docker network, and an rs.initiate() call once all members are reachable. For local development, a single-node deployment is usually sufficient; save the replica-set complexity for staging or production environments that need failover.

    How do I change the MongoDB root password after the container has already initialized?
    Environment variables like MONGO_INITDB_ROOT_PASSWORD only apply on first initialization of an empty data directory. To change credentials afterward, connect with mongosh and run db.changeUserPassword() against the admin database, or wipe the volume and reinitialize if you don’t need to preserve existing data.

    Can I run MongoDB and my application in the same compose file as other databases?
    Yes — Compose supports any number of services in one file. It’s common to see MongoDB alongside Redis for caching or Postgres for a different subsystem, all defined in the same docker-compose.yml and connected via the same internal network, each reachable by its own service name.

    Why does my app container fail to connect to Mongo on the very first docker compose up?
    MongoDB needs a few seconds to initialize its data files on a truly fresh volume, and depends_on alone doesn’t wait for that. Add a healthcheck block to the mongo service and use condition: service_healthy on the dependent service, or implement retry logic with backoff in your application’s database connection code.

    Conclusion

    A mongo docker compose file gives you a reproducible, version-controlled way to run MongoDB alongside your application, whether that’s a single local container or a multi-service stack with health checks and a dedicated application user. Start with the minimal image-plus-volume configuration, add authentication and an init script once you need scoped users, and layer in health checks and backup routines as the deployment moves from local development toward production. The same core patterns — named volumes for persistence, service-name networking, and environment-based secrets — carry over directly if you later add other containerized services like Postgres Docker Compose to the same stack. For the full range of official configuration options and image variants, the Docker Compose documentation is the authoritative reference to keep bookmarked.

  • Docker Compose Command

    Docker Compose Command: A Complete Reference Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Every Docker Compose workflow revolves around a handful of core commands, but understanding exactly what each docker compose command does — and when to use it — separates a smooth deployment from a debugging session at 2 AM. This guide walks through the essential docker compose command reference, from the basics of starting a stack to advanced troubleshooting patterns you’ll actually use in production.

    Whether you’re running a single-container app or a multi-service stack with a database, cache, and reverse proxy, the same core set of commands applies. This article breaks down what each docker compose command does, common flags worth knowing, and practical patterns for day-to-day operations.

    Understanding the Docker Compose Command Structure

    Every docker compose command follows a consistent pattern: docker compose [OPTIONS] COMMAND [ARGS]. Modern Docker installations use the integrated docker compose (no hyphen) syntax rather than the older standalone docker-compose binary, though both accept largely the same subcommands and flags.

    Compose reads its configuration from a compose.yaml (or docker-compose.yml) file in the current directory by default. You can point at a different file with the -f flag, which is useful when managing multiple environments (development, staging, production) from separate compose files.

    The Anatomy of a Compose File

    Before diving into commands, it helps to understand what Compose is actually acting on. A minimal file looks like this:

    services:
      web:
        image: nginx:latest
        ports:
          - "8080:80"
        depends_on:
          - api
      api:
        build: ./api
        environment:
          - NODE_ENV=production

    Each top-level entry under services becomes a target for docker compose commands — you can run docker compose up web to start just that service, or docker compose logs api to view logs for a single container without touching the rest of the stack.

    Global Flags Available on Every Docker Compose Command

    A few flags apply across almost every docker compose command and are worth memorizing:

  • -f, --file — specify an alternate compose file
  • -p, --project-name — override the default project name (normally the directory name)
  • --env-file — load environment variables from a specific file
  • -d, --detach — run containers in the background (used with up)
  • --profile — activate a named service profile
  • The Core Docker Compose Command for Starting Services

    The docker compose up command is the one most people learn first, and for good reason — it builds images if needed, creates networks and volumes, and starts every service defined in the file.

    docker compose up -d

    Running this docker compose command with -d detaches the process so your terminal isn’t tied to container output. Without -d, Compose streams combined logs from every service directly to your terminal, which is genuinely useful the first time you bring up a new stack, since you can watch for startup errors in real time.

    Rebuilding Images Before Starting

    If you’ve changed a Dockerfile or application code that gets baked into an image, up alone won’t rebuild it — Compose reuses existing images unless told otherwise. Add the --build flag:

    docker compose up -d --build

    This is one of the most common docker compose command variations used in local development loops, since it guarantees you’re running the latest code rather than a stale cached layer. For a deeper look at rebuild behavior and caching pitfalls, see our guide on Docker Compose Rebuild.

    Starting Only Specific Services

    You rarely need to bring up an entire stack when debugging a single component:

    docker compose up -d api

    Compose will still start any services listed in depends_on for api, but it won’t touch unrelated services like a frontend container that isn’t in the dependency chain.

    Stopping and Removing Containers with the Docker Compose Command Set

    Two commands are frequently confused: docker compose stop and docker compose down. Knowing the difference matters, because one is reversible and the other is destructive to more than just containers.

    docker compose stop halts running containers but leaves them, along with their networks and volumes, intact on disk. You can resume exactly where you left off with docker compose start.

    docker compose down, by contrast, removes containers and networks entirely. By default it preserves named volumes, but adding -v deletes those too — meaning any database data stored in an unnamed or anonymous volume is gone permanently. We cover this distinction, along with safe shutdown patterns, in detail in Docker Compose Down: Full Guide to Stopping Stacks.

    docker compose down --remove-orphans

    The --remove-orphans flag is worth adding as a habit — it cleans up containers for services that used to exist in your compose file but have since been removed, preventing orphaned containers from lingering silently.

    Inspecting Running Services

    Viewing Logs with docker compose logs

    Once services are running, docker compose logs is the fastest way to see what’s happening inside them without shelling into a container:

    docker compose logs -f --tail=100 api

    The -f flag follows the log stream live, and --tail limits initial output so you’re not scrolling through thousands of historical lines. For patterns around filtering, timestamping, and combining logs across services, see Docker Compose Logs: The Complete Debugging Guide.

    Checking Service Status

    docker compose ps lists containers managed by the current project, along with their state, exposed ports, and health status if a healthcheck is defined:

    docker compose ps

    This differs from plain docker ps in that it’s scoped to the current project directory’s compose file, so you only see containers relevant to the stack you’re working in — helpful when running multiple unrelated projects on the same host.

    Executing Commands Inside a Running Container

    Sometimes you need a shell or a one-off command inside a running service without stopping it. docker compose exec handles this:

    docker compose exec api sh

    This is different from docker compose run, which starts a brand-new container (useful for one-off tasks like running database migrations) rather than attaching to an already-running one.

    Managing Configuration and Environment Variables

    A large share of real-world Compose problems come from environment variable misconfiguration rather than the docker compose command itself failing. Compose merges variables from .env files, shell environment, and the environment: block in your service definition, and the precedence order isn’t always intuitive.

  • .env file values are substituted into the compose file at parse time
  • environment: block values set inside the running container
  • env_file: directive loads a file’s contents directly into the container
  • Shell-exported variables override .env file values during substitution
  • If a docker compose command isn’t picking up a variable you expect, running docker compose config prints the fully resolved configuration after all substitutions — an easy way to confirm what Compose actually sees. Our dedicated guide on Docker Compose Env: Manage Variables the Right Way walks through common pitfalls in more depth, and Docker Compose Environment Variables: Complete Guide covers additional substitution edge cases.

    Validating Your Compose File

    Before deploying, it’s worth running the validation docker compose command to catch syntax errors early:

    docker compose config --quiet

    This exits silently on success and prints a detailed error if the YAML is malformed or references an undefined variable, which is far faster than discovering the problem after up fails partway through starting your stack.

    Practical Multi-Service Example

    To tie these commands together, here’s a realistic compose file combining an application, a database, and a reverse proxy — the kind of stack you’d actually run in a homelab or small production deployment:

    services:
      app:
        build: .
        depends_on:
          - db
        environment:
          - DATABASE_URL=postgres://user:pass@db:5432/appdb
        restart: unless-stopped
    
      db:
        image: postgres:16
        volumes:
          - db_data:/var/lib/postgresql/data
        restart: unless-stopped
    
    volumes:
      db_data:

    With this file in place, the typical docker compose command sequence for a deployment looks like:

    docker compose config --quiet
    docker compose up -d --build
    docker compose ps
    docker compose logs -f app

    If you’re running Postgres specifically, our guide on Postgres Docker Compose: Full Setup Guide for 2026 covers volume persistence and backup strategies in more detail, and if you need a fast in-memory layer alongside it, Redis Docker Compose: The Complete Setup Guide is a useful companion reference.

    Choosing Where to Run Your Docker Compose Stack

    The docker compose command set works identically whether you’re running on a laptop or a cloud VPS, but production stacks benefit from predictable, dedicated resources rather than a shared or oversubscribed host. If you’re setting up a new server specifically to run Compose-managed services, a provider with straightforward block storage and predictable networking makes volume management and backups much less error-prone. DigitalOcean is a common choice for exactly this kind of small-to-medium Compose deployment, since droplets come with consistent CPU and disk performance that scales cleanly as you add services.


    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    What’s the difference between docker compose up and docker compose start?
    docker compose up creates containers, networks, and volumes if they don’t exist and then starts them — it’s the command you use for a fresh deployment or after changing the compose file. docker compose start only restarts containers that already exist and were previously stopped; it won’t pick up configuration changes.

    Does the docker compose command work the same as docker-compose (with a hyphen)?
    Largely yes. The docker compose command (integrated into the Docker CLI as a plugin) is the actively maintained version, while the standalone docker-compose Python-based binary is the legacy tool. Syntax and most flags are compatible, but new features land in the integrated version first, per the official Docker documentation.

    How do I restart a single service without affecting the rest of the stack?
    Use docker compose restart <service-name>. This stops and starts just that container, leaving other services running uninterrupted — useful after changing an environment variable that doesn’t require a rebuild.

    Why does docker compose down sometimes remove data I wanted to keep?
    This happens when the -v flag is included, which deletes named and anonymous volumes along with containers. If your database uses an anonymous volume rather than a named one declared under the top-level volumes: key, that data has no separate lifecycle from the container and is removed unless you explicitly avoid -v.

    Conclusion

    The docker compose command set is small enough to memorize but nuanced enough that misusing a flag — particularly around down -v or forgetting --build — can cause real problems. Sticking to a consistent workflow (config --quiet to validate, up -d --build to deploy, logs -f to verify, ps to check status) covers the vast majority of day-to-day operations. For orchestration needs beyond a single host, it’s also worth understanding how Compose concepts map onto larger systems like Kubernetes, since the service/network/volume model in Compose mirrors — at a smaller scale — the same primitives you’ll encounter there.

  • Docker Compose Stop

    Docker Compose Stop: A Practical Guide to Halting Containers Safely

    Anyone running multi-container applications eventually needs a reliable way to pause services without destroying them, and that’s exactly what docker compose stop is for. This guide covers how docker compose stop works, when to use it instead of other commands, and how to apply it correctly across development and production workflows.

    Why Docker Compose Stop Matters for Container Lifecycle Management

    When you’re managing a stack of interdependent services — a web app, a database, a cache, maybe a message queue — you rarely want an all-or-nothing approach to shutting things down. docker compose stop sends a termination signal to running containers defined in your docker-compose.yml file, halting their processes while leaving the containers, networks, and volumes intact on disk. This distinction matters enormously in practice.

    Unlike docker compose down, which removes containers and networks entirely, docker compose stop is non-destructive. Your container filesystem state, environment configuration, and any data written inside the container (outside of mapped volumes) remain exactly as they were. This makes docker compose stop the correct tool for temporary pauses: maintenance windows, resource reallocation, debugging, or simply freeing up CPU and memory on a shared host without losing your setup.

    The Signal Flow Behind Docker Compose Stop

    Under the hood, docker compose stop first sends a SIGTERM to the main process (PID 1) inside each container, then waits a grace period — 10 seconds by default — before escalating to SIGKILL if the process hasn’t exited. This graceful-then-forceful pattern gives applications a chance to close database connections, flush buffers, and finish in-flight requests cleanly.

    docker compose stop

    Running this with no arguments stops every service defined in the compose file. You can also target specific services:

    docker compose stop web worker

    This stops only the web and worker services, leaving everything else (like your database) running uninterrupted.

    Docker Compose Stop vs Other Shutdown Commands

    A common point of confusion is understanding how docker compose stop differs from its siblings. Getting this wrong can lead to accidentally destroying containers you meant to preserve, or leaving resources running when you thought you’d shut them down.

  • docker compose stop — halts running containers, keeps them and their networks/volumes on disk
  • docker compose down — stops containers and removes containers, default networks, and (with -v) volumes
  • docker compose pause — freezes container processes in place using cgroups freezer, without sending any signal
  • docker compose kill — sends SIGKILL immediately, skipping the graceful shutdown window
  • docker compose restart — stops and then starts containers again in one step
  • If you’re unsure which command fits your situation, the deciding question is: do I need this container gone, or just quiet? If it’s the latter, docker compose stop is almost always the right call. For a deeper comparison of the destructive alternative, see this guide on stopping full stacks with docker compose down.

    Comparing Stop and Pause

    While docker compose stop terminates the container’s main process, docker compose pause freezes it entirely using the kernel’s cgroup freezer subsystem — no signal is sent, and the process simply stops receiving CPU cycles until you run docker compose unpause. This is useful for very short freezes (like snapshotting a filesystem) but isn’t suitable for longer pauses, since the container’s network connections and timers remain open in a frozen state rather than being cleanly released.

    How to Use Docker Compose Stop in Practice

    The most common invocation of docker compose stop is straightforward, but there are several flags and patterns worth knowing to use it effectively.

    # Stop all services with the default 10-second grace period
    docker compose stop
    
    # Stop with a custom timeout (in seconds) before SIGKILL
    docker compose stop -t 30
    
    # Stop a single service
    docker compose stop redis

    The -t (or --timeout) flag is particularly important for services that need more time to shut down gracefully — a database flushing writes to disk, for example, may need longer than the default 10 seconds to avoid corruption.

    Handling Services with Custom Stop Signals

    Some applications don’t respond well to SIGTERM and expect a different signal to trigger graceful shutdown (Nginx, for instance, historically preferred SIGQUIT for a graceful stop). You can override the default in your compose file:

    services:
      web:
        image: nginx:latest
        stop_signal: SIGQUIT
        stop_grace_period: 20s

    Setting stop_grace_period in the compose file has the same effect as passing -t on the command line, but it’s persisted and version-controlled alongside your service definition, which is generally the better practice for anything beyond a one-off manual stop.

    Verifying Container State After Stopping

    After running docker compose stop, containers move into an “Exited” state rather than disappearing. You can confirm this directly:

    docker compose ps -a

    This will show your services with a status like Exited (0), confirming the container still exists and can be restarted with docker compose start at any time — no rebuild, no reconfiguration, no lost state.

    Common Scenarios Where Docker Compose Stop Is the Right Choice

    There are several recurring situations where reaching for docker compose stop, rather than a more destructive command, saves time and avoids unnecessary risk.

    Freeing host resources temporarily. If you’re running several projects on a single VPS and need to reclaim memory or CPU for another task, stopping the containers you’re not actively using is far faster than tearing them down and rebuilding later.

    Debugging without losing configuration. If a container is misbehaving, docker compose stop lets you halt it, inspect logs or configuration files on disk, and restart it without regenerating anything. Pairing this with a log review is often the fastest path to root cause — see this guide to debugging with docker compose logs for the companion workflow.

    Scheduled maintenance windows. For a service that needs periodic downtime (batch data imports, backups, schema migrations), docker compose stop before the operation and docker compose start after is cleaner than a full recreation cycle, since it avoids re-pulling images or re-mounting volumes.

    CI/CD pipeline cleanup between test runs. In ephemeral test environments, docker compose stop between test suites (as opposed to down) can speed up repeated pipeline runs when you don’t need a fully clean slate every time.

    Best Practices When Using Docker Compose Stop

    Getting the most out of docker compose stop means being deliberate about timeouts, signal handling, and what you actually want to preserve.

  • Always set an explicit stop_grace_period for stateful services like databases — the default 10 seconds is often too short for a clean write flush
  • Use docker compose stop <service> rather than stopping the whole stack when only one component needs a pause
  • Check docker compose ps -a afterward to confirm containers actually exited rather than assuming success
  • Combine docker compose stop with docker compose logs --tail=50 before stopping if you need a final snapshot of recent activity
  • Document custom stop_signal requirements directly in your docker-compose.yml so the behavior isn’t tribal knowledge
  • Environment-Specific Considerations

    In production, be cautious about stopping services that other containers depend on via depends_on — Compose does not automatically stop dependents first, so a database going down mid-request can cause connection errors in an application container that’s still running. If you’re managing environment-specific behavior across dev, staging, and production, it’s worth reviewing how your environment variables are structured across compose files, since stop/start cycles are a common place where environment drift becomes visible.

    If your project relies on secrets mounted at container start (API keys, database credentials), remember that docker compose stop does not clear those — they persist with the container. For guidance on managing this correctly, see this guide on secure configuration with Docker Compose secrets.

    Troubleshooting Docker Compose Stop Issues

    Occasionally docker compose stop doesn’t behave as expected. A few patterns account for most of the friction people run into.

    Containers taking the full grace period every time. If every stop takes exactly 10 seconds (or your configured timeout), it usually means the container’s main process isn’t handling SIGTERM at all, and Compose is falling through to SIGKILL after the wait. Check whether your application or entrypoint script traps and responds to SIGTERM explicitly.

    “Service is already stopped” messages. This is expected if the container exited on its own already (e.g., due to a crash) before you issued the command — it’s not an error, just Compose reporting current state accurately.

    Stop appears to hang indefinitely. This usually points to a zombie process issue inside the container, often caused by not using an init process to reap child processes. Adding init: true to the service definition in your compose file resolves this in most cases, since it runs a minimal init system (like tini) as PID 1.

    For broader reference on process signal handling and container lifecycle behavior, the official Docker documentation and Kubernetes documentation on pod termination (useful for comparison, since the SIGTERM/grace-period model is conceptually similar) are both authoritative references worth bookmarking.

    Conclusion

    Docker compose stop is a deliberately conservative command: it halts running processes while preserving everything else about your stack, making it the right default whenever you need a pause rather than a teardown. Understanding the SIGTERM-then-SIGKILL grace period, knowing how to target individual services, and configuring stop_grace_period appropriately for stateful services will cover the overwhelming majority of real-world use cases. When you do eventually need to remove containers and networks entirely, that’s the point where you’d reach for docker compose down instead — but for everyday pausing, restarting, and resource management, docker compose stop remains the safer, faster tool.

    FAQ

    Does docker compose stop delete my data?
    No. Docker compose stop only halts the running processes inside containers. Containers, their filesystems, associated networks, and volumes all remain on disk. Data is only removed if you separately run docker compose down -v, which explicitly deletes volumes.

    What’s the difference between docker compose stop and docker compose down?
    Docker compose stop halts containers but keeps them, along with their networks, on the host. Docker compose down goes further, removing the containers and default networks entirely (and volumes too, if you add the -v flag). Use stop for temporary pauses and down when you want a clean removal.

    How do I restart containers after using docker compose stop?
    Run docker compose start to bring the same containers back up using their existing configuration, or docker compose up if you also want Compose to check for any changes in your compose file. Neither requires rebuilding images.

    Can I stop just one service instead of the whole stack?
    Yes. Pass the service name as an argument, for example docker compose stop web. This stops only that service while leaving the rest of the stack running, which is useful when only one component needs maintenance or debugging.

  • Prometheus Docker Compose

    Prometheus Docker Compose: A Complete Setup Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Running Prometheus via Docker Compose is one of the fastest ways to get real, working metrics collection on a server without wrestling with systemd units or manual binary installs. This guide walks through a full prometheus docker compose setup – from a minimal config to scraping targets, adding Grafana, persisting data, and avoiding the mistakes that trip up most first-time setups.

    Whether you’re monitoring a single VPS or a small fleet of containers, a docker-compose.yml-driven Prometheus stack gives you a reproducible, version-controllable monitoring layer that you can tear down and rebuild in seconds. That reproducibility is the real reason most teams choose a prometheus docker compose approach over a manually configured install: the whole stack lives in one file, checked into git, instead of scattered across server state nobody remembers.

    Why Use Docker Compose for Prometheus

    Prometheus itself is a single static binary with a YAML config file, so in theory you don’t need Docker at all. In practice, Docker Compose solves several problems at once for a prometheus docker compose deployment:

  • Consistent versioning – you pin an exact Prometheus image tag instead of whatever your package manager happens to ship.
  • Isolated networking – Prometheus, Grafana, and any exporters share a private Docker network without exposing internal ports to the host.
  • Easy teardown/rebuild – docker compose down && docker compose up -d gives you a clean slate in seconds.
  • Portable configuration – the same docker-compose.yml and prometheus.yml work identically on a laptop, a staging VPS, or production.
  • If you’re already running other services with Compose, adding Prometheus to the same host is a natural extension rather than a separate operational concern.

    Prometheus vs. a Manual Install

    A manual install (downloading the binary, writing a systemd unit, managing the config path by hand) still works fine and is arguably lighter-weight for a single always-on server. The tradeoff is that upgrades become manual binary swaps, and there’s no built-in isolation between Prometheus’s dependencies and the rest of the host. A prometheus docker compose setup trades a small amount of Docker overhead for consistency and easier rollback – you can revert to a previous image tag instantly if an upgrade misbehaves.

    Prerequisites

    Before setting up prometheus docker compose, you need:

  • Docker Engine and the Compose plugin installed (docker compose version should return a version string, not an error).
  • A server or VPS with enough free memory – Prometheus’s memory footprint grows with the number of scraped series, so a small always-on instance (1-2GB RAM) is fine for a handful of targets, but larger deployments need more headroom.
  • Basic familiarity with YAML, since both docker-compose.yml and prometheus.yml are plain YAML files.
  • If you’re setting this up on a VPS for the first time, see the general guide to self-hosting n8n on a VPS for a comparable pattern of running a Docker Compose stack on a small server – the same host sizing and networking considerations apply.

    Basic Prometheus Docker Compose Setup

    The simplest possible prometheus docker compose file runs just the Prometheus server, mounts a config file, and exposes port 9090.

    Start with a project directory containing two files: docker-compose.yml and prometheus.yml.

    # docker-compose.yml
    services:
      prometheus:
        image: prom/prometheus:latest
        container_name: prometheus
        restart: unless-stopped
        volumes:
          - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
          - prometheus_data:/prometheus
        ports:
          - "9090:9090"
        command:
          - "--config.file=/etc/prometheus/prometheus.yml"
          - "--storage.tsdb.retention.time=15d"
    
    volumes:
      prometheus_data:

    And a minimal prometheus.yml that scrapes Prometheus’s own metrics endpoint:

    # prometheus.yml
    global:
      scrape_interval: 15s
    
    scrape_configs:
      - job_name: "prometheus"
        static_configs:
          - targets: ["localhost:9090"]

    Bring it up with:

    docker compose up -d

    Visit http://your-server-ip:9090 and you should see the Prometheus web UI. Under Status → Targets, the prometheus job should show as UP. This confirms the base prometheus docker compose stack is working before you add anything else.

    Understanding the Volume Mounts

    The prometheus.yml:ro mount is read-only by design – Prometheus never writes back to its own config file, so mounting it read-only prevents accidental modification from inside the container. The prometheus_data named volume is where Prometheus stores its time-series database (TSDB); without it, every docker compose down (without -v) would still preserve data, but a full volume removal would wipe your metric history. Keep this distinction in mind when scripting cleanup commands.

    Setting Retention and Storage Flags

    The --storage.tsdb.retention.time flag controls how long Prometheus keeps data before deleting old blocks. Fifteen days is a reasonable default for a small deployment, but you can also cap by size with --storage.tsdb.retention.size if disk space is a tighter constraint than time. Both flags can be combined – Prometheus deletes data once either threshold is hit, whichever comes first.

    Adding Scrape Targets

    A Prometheus server that only scrapes itself isn’t very useful. The real value of a prometheus docker compose setup comes from adding scrape targets for the services you actually want to monitor.

    Scraping the Node Exporter

    To collect host-level metrics (CPU, memory, disk, network), add the Node Exporter as a second service in the same Compose file:

    services:
      prometheus:
        image: prom/prometheus:latest
        # ... (as above)
    
      node-exporter:
        image: prom/node-exporter:latest
        container_name: node-exporter
        restart: unless-stopped
        pid: host
        volumes:
          - /proc:/host/proc:ro
          - /sys:/host/sys:ro
          - /:/rootfs:ro
        command:
          - "--path.procfs=/host/proc"
          - "--path.sysfs=/host/sys"
          - "--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)"
        ports:
          - "9100:9100"

    Then add a scrape job to prometheus.yml:

      - job_name: "node"
        static_configs:
          - targets: ["node-exporter:9100"]

    Note that inside the Compose network, Prometheus reaches the exporter by service name (node-exporter), not localhost – Docker Compose’s default bridge network resolves service names as DNS hostnames automatically.

    Scraping Application Metrics

    If your own application exposes a /metrics endpoint (most frameworks have a Prometheus client library for this), add it the same way – as another static_configs block pointing at the service name and port. For dynamic environments with containers that come and go, Prometheus also supports service discovery mechanisms (Docker Swarm, Kubernetes, file-based) instead of hardcoded static targets, though a static config is usually sufficient for a small, stable stack.

    Adding Grafana for Visualization

    Prometheus’s built-in web UI is functional for ad-hoc queries but not built for dashboards. Pairing your prometheus docker compose stack with Grafana is the standard next step.

      grafana:
        image: grafana/grafana:latest
        container_name: grafana
        restart: unless-stopped
        ports:
          - "3000:3000"
        volumes:
          - grafana_data:/var/lib/grafana
        depends_on:
          - prometheus
    
    volumes:
      grafana_data:

    After docker compose up -d, log in to Grafana at http://your-server-ip:3000 (default credentials are admin/admin, and Grafana forces a password change on first login). Add Prometheus as a data source using the internal Docker network address http://prometheus:9090 – not localhost, since Grafana runs in its own container. From there, you can import a community dashboard for Node Exporter metrics or build your own panels against your application’s custom metrics.

    Persisting Data and Handling Backups

    Data persistence is easy to overlook until you lose it. In the Compose file above, both prometheus_data and grafana_data are named volumes, which Docker manages outside the container’s writable layer – they survive docker compose down and container recreation, but not docker compose down -v or a manual docker volume rm.

    For anything you actually care about keeping:

  • Back up named volumes on a schedule with a simple tar job against the volume’s mount point, or a dedicated backup tool.
  • Consider bind-mounting to a known host path instead of a named volume if you want your backup script to reference a stable filesystem location directly.
  • Test your restore process, not just your backup process – an untested backup is just an assumption.
  • If you’re also running a database alongside this stack, the same persistence principles apply – see the guide to Postgres with Docker Compose for a closely related pattern of volume-backed data durability.

    Securing Your Prometheus Docker Compose Deployment

    By default, the Compose file above exposes Prometheus and Grafana directly on the host’s public ports. For anything beyond local testing, that’s a real exposure risk – Prometheus’s UI has no built-in authentication.

  • Don’t publish port 9090 (or 3000) directly to the public internet. Bind it to 127.0.0.1:9090:9090 instead and put a reverse proxy in front with authentication.
  • If you need remote access, terminate TLS and add basic auth at the reverse proxy layer (nginx, Caddy, or Traefik all support this).
  • Keep secrets (Grafana admin passwords, any API tokens exporters need) out of the Compose file itself – see the guide to Docker Compose secrets management for patterns that avoid hardcoding credentials into version-controlled files.
  • Review environment variable handling generally in the Docker Compose env variables guide if you’re passing configuration through .env files.
  • Troubleshooting Common Issues

    A few problems come up repeatedly with prometheus docker compose setups:

    Targets Showing as “Down”

    If a target shows DOWN in the Targets page, the most common cause is using localhost instead of the Docker Compose service name in prometheus.yml. Containers on the same Compose network communicate via service name DNS, not localhost – each container has its own network namespace. Fix the targets: entry to use the service name and re-run docker compose restart prometheus, or check logs with the patterns covered in the Docker Compose logs debugging guide.

    Config Changes Not Taking Effect

    Editing prometheus.yml on the host doesn’t automatically reload the running container’s in-memory config. Either restart the container (docker compose restart prometheus) or, if you started Prometheus with the --web.enable-lifecycle flag, trigger a hot reload via curl -X POST http://localhost:9090/-/reload without a restart.

    Rebuilding After a Compose File Change

    If you change the docker-compose.yml itself (new service, new port mapping), a simple restart isn’t enough – you need to recreate the containers. Run docker compose up -d again, which detects the diff and recreates only the affected services, or see the Docker Compose rebuild guide for a fuller breakdown of when --build or --force-recreate is actually needed.

    Choosing Where to Host Your Monitoring Stack

    A prometheus docker compose stack is lightweight enough to run on a small VPS instance, but disk I/O and available memory both affect how many metrics you can retain and for how long. If you’re provisioning a new server specifically for monitoring, providers like DigitalOcean or Vultr offer small instances that are generally sufficient for a Prometheus + Grafana + a handful of exporters, provided you keep retention windows reasonable and monitor disk usage on the TSDB volume itself.


    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Does Prometheus in Docker Compose persist data across restarts?
    Yes, as long as you use a named volume or bind mount for /prometheus (the TSDB storage path). A plain docker compose restart or down/up cycle preserves the volume; only an explicit docker compose down -v or manual volume deletion removes it.

    Can I run Prometheus and Grafana in the same Compose file as my application?
    Yes – this is a common pattern. Add Prometheus, Grafana, and any exporters as additional services in your application’s existing docker-compose.yml, and they’ll share the same default network, letting Prometheus reach your app’s metrics endpoint by service name.

    How do I add more scrape targets later?
    Edit the scrape_configs section of prometheus.yml and reload Prometheus – either a container restart or a hot reload via the /-/reload endpoint if --web.enable-lifecycle is set. No changes to docker-compose.yml are needed unless the new target is a service you’re also adding to the same Compose stack.

    Is Docker Compose suitable for production Prometheus deployments, or only Kubernetes?
    Docker Compose is a reasonable choice for a single-host production deployment monitoring a small to medium number of targets. Once you need multi-host scaling, high availability, or dynamic service discovery across a cluster, a Kubernetes-based setup (using the Prometheus Operator or similar) becomes more appropriate – see the Kubernetes vs Docker Compose comparison for a broader look at when that step makes sense.

    Conclusion

    A prometheus docker compose setup gives you a reproducible, version-controlled monitoring stack that’s straightforward to extend – start with a single Prometheus service scraping itself, add exporters and application targets, layer Grafana on top for dashboards, and lock down access before exposing anything publicly. The core pattern (one docker-compose.yml, one prometheus.yml, named volumes for persistence) scales from a single small VPS up to a moderately sized fleet of monitored services without needing a different toolset. For further reference, the official Prometheus documentation and Docker Compose documentation cover configuration options and flags in more depth than this guide’s basics.

  • Docker Compose Configuration

    Docker Compose Configuration: A Complete Guide for DevOps Teams

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Getting your docker compose configuration right is the difference between a stack that starts reliably every time and one that breaks the moment you touch a dependency. This guide walks through the structure, syntax, and practical patterns you need to write a clean, maintainable docker compose configuration for real-world projects, from local development to small production deployments.

    Why Docker Compose Configuration Matters

    A docker compose configuration is the single file (or set of files) that describes every service, network, and volume your application needs to run. Instead of remembering a long list of docker run flags, you define everything declaratively in YAML and let Compose handle the orchestration. This matters for a few concrete reasons:

  • Reproducibility: anyone on the team can clone the repo and run docker compose up to get an identical environment.
  • Version control: your infrastructure definition lives next to your application code and gets reviewed in pull requests like everything else.
  • Reduced drift: without a shared docker compose configuration, developers tend to hand-tune containers locally, and environments slowly diverge from each other and from production.
  • Compose is not a replacement for a full orchestrator like Kubernetes in large-scale, multi-node deployments, but for single-host applications, staging environments, and local development it remains one of the fastest ways to get a multi-container application running consistently. If you’re deciding between the two approaches for a given workload, our Kubernetes vs Docker Compose comparison covers the tradeoffs in more depth.

    Anatomy of a docker compose configuration File

    Every docker compose configuration starts with a compose.yaml (or the legacy docker-compose.yml) file at the root of your project. The top-level structure is simple: a services block, and optionally networks, volumes, and configs/secrets blocks.

    services:
      web:
        image: nginx:1.27
        ports:
          - "8080:80"
        depends_on:
          - api
    
      api:
        build: ./api
        environment:
          - NODE_ENV=production
        depends_on:
          - db
    
      db:
        image: postgres:16
        volumes:
          - db_data:/var/lib/postgresql/data
        environment:
          - POSTGRES_PASSWORD=changeme
    
    volumes:
      db_data:

    This minimal docker compose configuration defines three services that depend on each other in sequence, a named volume for persistent database storage, and explicit port mapping for the web-facing service. It’s small enough to read in ten seconds, but it captures the same information that would otherwise require three separate docker run commands with a dozen flags each.

    Services, Networks, and Volumes

    The three core building blocks of any docker compose configuration are services, networks, and volumes:

  • Services define the containers themselves — image or build context, ports, environment, dependencies, and restart behavior.
  • Networks define how services talk to each other. By default, Compose creates a single bridge network per project and every service can reach every other service by name, which is why api in the example above can connect to db using the hostname db rather than an IP address.
  • Volumes define persistent storage that survives container restarts and removals. Anonymous volumes are convenient but hard to manage; named volumes, like db_data above, are easier to inspect, back up, and reason about.
  • If your stack includes a relational database, our Postgres Docker Compose and PostgreSQL Docker Compose guides walk through volume and initialization patterns specific to Postgres, and the same general approach applies to Redis Docker Compose setups.

    The build vs image Decision

    A common early decision in any docker compose configuration is whether a service should use a prebuilt image or a local build context. Using image pulls a tagged image from a registry — fast, predictable, and appropriate for third-party services like databases or message brokers. Using build tells Compose to build from a Dockerfile in your repo, which is what you want for your own application code.

    services:
      app:
        build:
          context: .
          dockerfile: Dockerfile
        image: myapp:latest

    Specifying both build and image together, as shown above, tells Compose to build the image locally and tag it, which is useful when you also want to push that tag to a registry later. For a deeper look at how these two mechanisms relate, see our comparison of Dockerfile vs Docker Compose and the reverse framing in Docker Compose vs Dockerfile.

    Managing Environment Variables in Your docker compose Configuration

    Hardcoding configuration values directly into your docker compose configuration is a common mistake — it makes the file unsafe to commit (if it contains secrets) and inflexible across environments. Compose supports environment variables at two levels: variables injected into the container, and variables used for interpolation inside the YAML file itself.

    # .env file, read automatically by Compose from the project root
    POSTGRES_USER=appuser
    POSTGRES_PASSWORD=supersecret
    API_PORT=3000

    services:
      api:
        build: ./api
        ports:
          - "${API_PORT}:3000"
        environment:
          - DATABASE_USER=${POSTGRES_USER}
          - DATABASE_PASSWORD=${POSTGRES_PASSWORD}

    Compose automatically loads a .env file located next to your compose file and substitutes ${VARIABLE} references at parse time. This keeps secrets and environment-specific values out of the tracked YAML while still letting the docker compose configuration reference them cleanly. For a complete breakdown of interpolation rules, precedence between shell variables and .env files, and per-service env_file directives, see our dedicated guides on Docker Compose Env and Docker Compose Environment Variables.

    Keeping Secrets Out of Plain Environment Variables

    Environment variables are convenient but they show up in docker inspect output and process listings, which isn’t ideal for high-sensitivity values like private keys or production database passwords. Compose’s secrets top-level element lets you mount sensitive values as files instead:

    services:
      api:
        build: ./api
        secrets:
          - db_password
    
    secrets:
      db_password:
        file: ./secrets/db_password.txt

    The application reads the value from /run/secrets/db_password inside the container rather than from an environment variable. This pattern is worth adopting for anything you wouldn’t want to appear in a log or a support ticket. Our Docker Compose Secrets guide covers this in more detail, including how it interacts with Docker Swarm’s native secrets support.

    Multiple Compose Files and Environment Overrides

    Real projects rarely have one docker compose configuration that works identically everywhere. Compose supports layering multiple files together, so you can keep a base configuration and override specific values per environment.

    docker compose -f compose.yaml -f compose.override.yaml up -d

    Compose automatically picks up a file named compose.override.yaml if it exists alongside compose.yaml, so in most projects you don’t even need the -f flags for the local case — this is exactly the default developer workflow. For staging or production, you typically create a separate file such as compose.prod.yaml and pass it explicitly:

    # compose.prod.yaml
    services:
      api:
        restart: always
        environment:
          - NODE_ENV=production
        deploy:
          resources:
            limits:
              memory: 512M

    This layering approach means your base docker compose configuration stays simple and portable, while environment-specific concerns like resource limits, restart policies, or replica counts live in files that are only applied where they’re relevant.

    Rebuilding and Restarting Cleanly

    Once your docker compose configuration changes — a new dependency in the Dockerfile, an updated base image, or a changed build context — Compose needs to rebuild the affected images before the new containers can start.

    docker compose up -d --build

    Adding --build forces Compose to rebuild any service with a build directive before starting containers, which is the safest habit to get into after any Dockerfile or dependency change. If you only need to rebuild without immediately restarting, docker compose build on its own does the same work without touching running containers. Our Docker Compose Rebuild and Docker Compose Up Build guides go through common rebuild pitfalls, like stale layer caching, in more detail, and Docker Compose Build covers build-specific flags and multi-stage build interactions.

    Validating and Debugging Your docker compose Configuration

    Before deploying any change, it’s worth validating the docker compose configuration itself rather than discovering a typo after containers fail to start.

    docker compose config

    This command parses your compose file(s), resolves all environment variable interpolation and file overrides, and prints the fully-resolved configuration Compose would actually use. It’s the fastest way to catch a missing variable, a malformed indentation, or an override that didn’t apply the way you expected.

    When something does go wrong at runtime, logs are your first stop:

    docker compose logs -f api

    Following logs for a specific service, rather than the entire stack, keeps the output readable when you’re debugging one component. See the Docker Compose Logs and Docker Compose Logging guides for filtering by timestamp, tailing multiple services at once, and configuring log drivers, and the Docker Compose Log Command reference for the full flag list.

    Shutting Down Without Losing Data

    Stopping a stack correctly is just as important as starting it. docker compose down stops and removes containers and the default network, but by default it leaves named volumes intact — which is usually what you want, since it preserves your database data between restarts.

    docker compose down          # stops containers, keeps volumes
    docker compose down -v       # stops containers AND deletes volumes

    The -v flag is destructive and will erase persisted volume data, so it should only be used deliberately — for example when tearing down a disposable test environment. Our full Docker Compose Down guide covers the different shutdown modes and when each one is appropriate.

    Choosing Where to Run Your Compose Stack

    A docker compose configuration still needs a host to run on, and the sizing of that host matters more than it might seem. A stack with a database, an API, and a reverse proxy needs enough memory headroom that the kernel’s OOM killer doesn’t start terminating containers under load, and enough disk I/O that volume-backed services like Postgres or Redis don’t become the bottleneck.

    For small to mid-sized production deployments, a general-purpose VPS is usually the right starting point rather than a managed container platform — you get full control over the Docker daemon, storage driver, and networking, at a lower cost than managed alternatives. Providers like DigitalOcean and Hetzner offer straightforward VPS tiers that work well for running a Compose-based stack, and Vultr is another option worth comparing if latency to a specific region matters for your users.

  • Match your container memory limits to the actual VPS RAM, not the other way around — Compose won’t stop you from overcommitting.
  • Use a named volume, not a bind mount, for database storage unless you have a specific reason to inspect files directly on the host.
  • Set explicit restart: unless-stopped (or always) policies so services recover automatically after a host reboot.

  • Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    What’s the difference between docker-compose and docker compose?
    docker-compose (with a hyphen) is the older, standalone Python-based tool. docker compose (as a subcommand, no hyphen) is the newer, Go-based implementation built into the Docker CLI itself. Both read the same docker compose configuration syntax, but the CLI-integrated version is what’s actively maintained and recommended going forward — check the Docker documentation for the current installation and migration guidance.

    Can one docker compose configuration file manage multiple environments?
    Yes, through file layering. Keep a base compose.yaml with shared service definitions, then use compose.override.yaml (applied automatically) for local development and a separate file like compose.prod.yaml (applied explicitly with -f) for production-specific overrides such as resource limits or restart policies.

    Why does my service fail to connect to another service by name?
    Compose only creates DNS entries for services on the same network, and services are only reachable once they’re actually accepting connections — not just once the container has started. Use depends_on with a condition: service_healthy check (paired with a healthcheck block) rather than assuming startup order guarantees readiness.

    Should I commit my .env file to version control?
    No, if it contains secrets. Commit an .env.example with placeholder values documenting which variables your docker compose configuration expects, and keep the real .env file — with actual credentials — out of the repository via .gitignore.

    Conclusion

    A well-structured docker compose configuration is one of the highest-leverage files in a small-to-medium infrastructure project: it documents your entire stack, keeps environments consistent, and turns a multi-step manual setup into a single docker compose up. Start with a clean base file, layer environment-specific overrides rather than duplicating configuration, keep secrets out of plain environment variables where it matters, and validate changes with docker compose config before deploying. For deeper detail on any individual piece — volumes, secrets, environment variables, or rebuild behavior — the linked guides above cover each topic in full, and the official Docker Compose documentation remains the authoritative reference for syntax changes as the tool evolves.

  • Docker Compose Syntax

    Docker Compose Syntax: A Practical Reference Guide

    Getting Docker Compose syntax right is the difference between a stack that starts cleanly and one that fails with a cryptic YAML parsing error at 2 AM. This guide walks through the core structure of a docker-compose.yml file, the most commonly misused directives, and practical patterns for keeping multi-container setups readable and maintainable. Whether you’re defining your first service or debugging an indentation error in a fifty-line file, understanding docker compose syntax thoroughly will save you time and prevent avoidable outages.

    Docker Compose files are YAML, which means whitespace matters and small mistakes compound quickly. This article assumes you already have Docker installed and are comfortable running basic containers, and focuses specifically on the syntax rules and structural conventions that make Compose files predictable and easy to review.

    Why Docker Compose Syntax Matters

    A Compose file isn’t just configuration — it’s the contract between your application’s services. Get the docker compose syntax wrong and you might end up with a service that silently fails to mount a volume, a network alias that never resolves, or an environment variable that’s interpreted as a boolean instead of a string. YAML’s strictness about indentation (spaces only, never tabs) is one of the most common sources of these failures, especially for engineers coming from JSON or INI-based config formats.

    Because Compose files are usually checked into version control and reviewed by teammates, consistent syntax also matters for readability. A file with mixed indentation styles or inconsistent key ordering is harder to diff and harder to trust during a code review.

    The Basic File Structure

    Every Compose file has a small number of top-level keys. The most important is services, which defines the containers that make up your application. Optional top-level keys include volumes, networks, and configs/secrets for resources shared across services.

    services:
      web:
        image: nginx:latest
        ports:
          - "8080:80"
        depends_on:
          - api
    
      api:
        build: ./api
        environment:
          - NODE_ENV=production
        volumes:
          - api_data:/data
    
    volumes:
      api_data:

    Note that older Compose files often included a version: key at the top (e.g. version: "3.8"). The Compose Specification, which is what modern Docker Compose (the docker compose CLI plugin, as opposed to the legacy standalone docker-compose binary) implements, no longer requires this field, and Docker’s own documentation now recommends omitting it.

    Indentation and Key Ordering Rules

    YAML uses indentation to represent nesting, and Compose is no exception. A few rules to internalize:

  • Use two spaces per indentation level; never mix tabs and spaces.
  • Lists (like ports or volumes entries) are denoted with a leading dash and a single space.
  • Mapping keys under a service (like image, build, environment) must all be indented at the same level directly under the service name.
  • Quoting is optional in most cases, but strings that look like numbers, booleans, or contain a colon (such as port mappings) should be quoted to avoid YAML misinterpreting them.
  • A very common docker compose syntax mistake is writing ports: 8080:80 instead of a proper list item. Compose expects a sequence here, so the correct form is always a dash-prefixed entry under the ports key.

    Defining Services: Core Directives

    The services block is where most of your Compose file lives. Each service maps to a running container, and Docker Compose handles building images, creating networks, and starting containers in dependency order.

    Image vs. Build

    You’ll typically choose one of two ways to source a container image for a service:

    services:
      cache:
        image: redis:7-alpine
    
      app:
        build:
          context: .
          dockerfile: Dockerfile.prod
          args:
            NODE_VERSION: "20"

    image pulls a pre-built image from a registry. build tells Compose to construct the image locally from a Dockerfile, and you can pass build arguments through the args key. If you’re unsure when to reach for one over the other, the Dockerfile vs Docker Compose comparison covers the distinction between building images and orchestrating containers in more depth, and the related Docker Compose vs Dockerfile guide walks through concrete examples of each.

    Ports, Volumes, and Networks

    These three directives handle how your service is exposed and how data persists:

    services:
      db:
        image: postgres:16
        ports:
          - "5432:5432"
        volumes:
          - db_data:/var/lib/postgresql/data
        networks:
          - backend
    
    volumes:
      db_data:
    
    networks:
      backend:

    Port mappings follow the HOST:CONTAINER format. Volumes can be named (as above, backed by a Docker-managed volume) or bind-mounted to a host path using an absolute or relative path string like ./config:/etc/app/config. For a deeper walkthrough of volume-specific syntax and common pitfalls, see the Docker Compose Volumes guide.

    Environment Variables

    Environment variables can be declared inline as a list, as a mapping, or pulled from an external file:

    services:
      api:
        image: myapp/api:latest
        environment:
          DATABASE_URL: postgres://user:pass@db:5432/appdb
          LOG_LEVEL: info
        env_file:
          - .env

    Both the list form (- KEY=value) and the mapping form (KEY: value) are valid docker compose syntax for the environment key — pick one convention and use it consistently across your files. The Docker Compose Env guide and the more detailed Docker Compose Environment Variables reference both go further into variable substitution, default values, and precedence rules when the same variable is set in multiple places.

    Multi-Document and Override Files

    A single docker-compose.yml is fine for small projects, but most real deployments split configuration across a base file and one or more overrides — for example, docker-compose.yml plus docker-compose.override.yml for local development, or a separate docker-compose.prod.yml for production.

    docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

    Compose merges these files in the order given on the command line, with later files overriding matching keys from earlier ones. Understanding merge behavior is part of understanding docker compose syntax as a whole, because a misplaced key in an override file can silently shadow a setting you expect to be active.

    Common Merge Pitfalls

    A few things trip people up when working with multiple Compose files:

  • Lists (like ports or command) are replaced entirely by an override, not merged item-by-item.
  • Mappings (like environment when written as key-value pairs) are merged key-by-key.
  • depends_on entries can be extended, but the format must match between files (short list form vs. long form with condition).
  • If a merged stack behaves unexpectedly, check the output of docker compose config, which prints the fully resolved configuration after all files and variable substitutions are applied — often the fastest way to spot a syntax or merge issue.

    Extension Fields and YAML Anchors

    For larger files with repeated blocks, YAML anchors and the Compose Specification’s extension fields (keys prefixed with x-) help avoid duplication:

    x-common-env: &common-env
      TZ: UTC
      LOG_LEVEL: info
    
    services:
      worker:
        image: myapp/worker:latest
        environment:
          <<: *common-env
          QUEUE_NAME: default
    
      scheduler:
        image: myapp/scheduler:latest
        environment:
          <<: *common-env
          QUEUE_NAME: scheduled

    This isn’t Compose-specific — it’s standard YAML — but it’s a useful pattern once your service definitions start repeating the same environment variables, labels, or logging configuration across multiple services.

    Dependency Ordering and Health Checks

    depends_on controls startup order but, by default, only waits for the dependency’s container to start, not for the application inside it to be ready. For services like databases where “started” and “ready to accept connections” are different moments, pair depends_on with a healthcheck:

    services:
      db:
        image: postgres:16
        healthcheck:
          test: ["CMD-SHELL", "pg_isready -U postgres"]
          interval: 5s
          timeout: 5s
          retries: 5
    
      api:
        build: .
        depends_on:
          db:
            condition: service_healthy

    This long-form depends_on syntax, using condition: service_healthy, tells Compose to wait until the healthcheck passes before starting the dependent service. This is one of the more important pieces of docker compose syntax to get right in any stack involving a database, since a race between an application container and its database is a frequent source of intermittent startup failures. If you’re specifically working with Postgres, the Postgres Docker Compose setup guide and the related PostgreSQL Docker Compose guide walk through healthcheck configuration in more detail, and the same pattern applies well to Redis Docker Compose setups.

    Logging and Debugging Compose Files

    Once your file is syntactically valid, the next challenge is often understanding what’s happening at runtime. docker compose config validates and prints the resolved file, which is the first thing to run when a service isn’t behaving as the raw YAML suggests. For runtime issues, docker compose logs -f <service> streams output from a running container.

    Validating Before You Deploy

    Running docker compose config --quiet in a CI pipeline is a cheap way to catch docker compose syntax errors before they reach a server. It exits non-zero if the file fails to parse or references an undefined variable without a default, which makes it easy to wire into a pre-deploy check.

    docker compose config --quiet && echo "Compose file is valid"

    For deeper debugging once containers are running, the Docker Compose Logs guide and the related Docker Compose Logging reference cover log drivers, tailing multiple services at once, and filtering output — useful once you’ve confirmed the syntax itself isn’t the problem.

    Secrets and Sensitive Configuration

    Avoid putting credentials directly in environment blocks committed to version control. Compose supports a dedicated secrets top-level key, which is especially relevant when running in Swarm mode but is also usable to reference files that Compose mounts into containers at runtime:

    services:
      api:
        image: myapp/api:latest
        secrets:
          - db_password
    
    secrets:
      db_password:
        file: ./secrets/db_password.txt

    This keeps sensitive values out of the environment block and out of docker inspect output in cases where that distinction matters for your security posture. The Docker Compose Secrets guide covers this pattern in more depth, including how it differs from plain environment variables and bind-mounted config files.

    FAQ

    Does docker compose syntax require a version key at the top of the file?
    No. Modern Docker Compose implements the Compose Specification, which does not require a version field. If present, it’s largely ignored by current versions of the CLI. New files can omit it entirely.

    What’s the difference between the short and long syntax for depends_on?
    The short form is a plain list of service names (depends_on: [db, cache]) and only waits for the dependency container to start. The long form uses a mapping with a condition key (e.g. condition: service_healthy) and can wait for a healthcheck to pass, which is generally the more reliable choice for databases and other stateful dependencies.

    Can I use environment variables inside a Compose file itself, not just inside containers?
    Yes. Compose supports variable substitution using ${VARIABLE_NAME} syntax, sourced from the shell environment or a .env file in the same directory as the Compose file. You can also provide defaults inline, like ${PORT:-3000}.

    Why does my Compose file fail with an indentation error even though it looks correctly formatted in my editor?
    This is almost always a tabs-vs-spaces issue, since YAML forbids tabs for indentation. Many editors render tabs and spaces identically, so the mismatch is invisible until a YAML parser rejects it. Configuring your editor to insert spaces for the Compose file type (and to visualize whitespace) usually resolves it.

    Conclusion

    Docker Compose syntax is straightforward once you internalize a handful of rules: consistent two-space indentation, correct use of lists versus mappings, and an understanding of how multiple files merge together. Most real-world Compose problems trace back to one of a small set of causes — a tab where a space was expected, an unquoted string that YAML parses differently than intended, or a depends_on that doesn’t actually wait for readiness. Running docker compose config early and often, keeping secrets out of plain environment blocks, and splitting configuration across base and override files as your stack grows will keep even fairly large multi-service applications maintainable. For the authoritative and continuously updated reference on every available key, the Docker Compose file reference on docs.docker.com and the underlying Compose Specification on GitHub are worth bookmarking alongside this guide.

  • Graylog Docker Compose

    Graylog Docker Compose: A Complete Self-Hosted Setup Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Setting up Graylog Docker Compose is one of the fastest ways to get a working centralized logging stack on a single VPS or bare-metal box. This guide walks through a real, working Graylog Docker Compose configuration, explains each service’s role, and covers the operational details you need to keep the stack healthy long after the first docker compose up.

    Graylog is a log management platform built on top of OpenSearch (or Elasticsearch, depending on version) and MongoDB. Running it via Docker Compose means you don’t need to hand-install three separate services with three separate init scripts — you define the whole stack in one file, start it with one command, and tear it down just as cleanly. This article assumes basic familiarity with Docker and Docker Compose but no prior Graylog experience.

    Why Use Graylog Docker Compose Instead of a Manual Install

    Manually installing Graylog means installing a JVM, MongoDB, and OpenSearch separately, matching compatible versions, and hand-writing systemd units for each one. A Graylog Docker Compose setup collapses all of that into a single declarative file. Version pinning is explicit, networking between services is handled automatically by Compose’s internal DNS, and the entire stack can be reproduced on a new host in minutes.

    There are a few concrete advantages worth calling out:

  • Reproducibility — the same docker-compose.yml produces the same stack on any host with Docker installed.
  • Isolation — Graylog, MongoDB, and OpenSearch each run in their own container with their own filesystem, reducing dependency conflicts.
  • Easier upgrades — bumping an image tag and running docker compose up -d is far less risky than an in-place OS package upgrade.
  • Simple teardown — a broken test stack can be removed with docker compose down -v without leaving stray files across your filesystem.
  • If you already run other services under Compose — for example if you’ve set up a Postgres Docker Compose stack or a Redis Docker Compose instance for other applications — this workflow will feel familiar. The same mental model of “one YAML file, one command” applies here.

    Core Components of the Graylog Stack

    Before writing the Compose file, it helps to understand what each container is actually doing. A Graylog Docker Compose deployment typically has three services.

    MongoDB: Configuration Storage

    MongoDB stores Graylog’s configuration data: users, dashboards, input definitions, alert rules, and index set metadata. It does not store your actual log messages — that’s OpenSearch’s job. MongoDB’s storage requirements stay relatively small and predictable even as log volume grows, because it only holds metadata, not the log events themselves.

    OpenSearch: Log Storage and Search

    OpenSearch (the search engine Graylog uses in current releases, having replaced Elasticsearch in the officially supported stack) is where the actual log data lives and gets indexed. This is almost always the most resource-hungry container in the stack — it needs enough heap memory to index incoming messages and enough disk to retain them for your chosen retention window. Undersizing this container is the single most common cause of a sluggish Graylog Docker Compose deployment.

    Graylog Server: Ingestion, Processing, and UI

    The Graylog server container handles everything else: accepting incoming log messages over various inputs (GELF, Syslog, Beats, raw TCP/UDP), running extractors and pipeline rules against them, and serving the web interface you use to search and build dashboards. It talks to MongoDB for configuration and to OpenSearch for storing and querying messages.

    Writing the Graylog Docker Compose File

    Here is a minimal, working docker-compose.yml for a single-node Graylog Docker Compose stack. It’s intentionally close to the configuration documented in Graylog’s own installation guide, with values you should adjust for your environment.

    version: "3.8"
    
    services:
      mongodb:
        image: mongo:6.0
        restart: unless-stopped
        volumes:
          - mongo_data:/data/db
    
      opensearch:
        image: opensearchproject/opensearch:2.15.0
        restart: unless-stopped
        environment:
          - "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
          - "discovery.type=single-node"
          - "DISABLE_SECURITY_PLUGIN=true"
          - "action.auto_create_index=false"
        ulimits:
          memlock:
            soft: -1
            hard: -1
        volumes:
          - opensearch_data:/usr/share/opensearch/data
    
      graylog:
        image: graylog/graylog:6.1
        restart: unless-stopped
        environment:
          - GRAYLOG_PASSWORD_SECRET=changeme_use_a_long_random_string
          - GRAYLOG_ROOT_PASSWORD_SHA2=changeme_sha256_hash_of_your_password
          - GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000/
        entrypoint: /usr/bin/tini -- wait-for-it opensearch:9200 -- /docker-entrypoint.sh
        depends_on:
          - mongodb
          - opensearch
        ports:
          - "9000:9000"
          - "1514:1514/udp"
          - "12201:12201/udp"
        volumes:
          - graylog_data:/usr/share/graylog/data
    
    volumes:
      mongo_data:
      opensearch_data:
      graylog_data:

    A few notes on this file worth understanding before you deploy it:

  • GRAYLOG_PASSWORD_SECRET must be a long, random string — Graylog uses it to salt stored passwords.
  • GRAYLOG_ROOT_PASSWORD_SHA2 is the SHA-256 hash of your admin password, not the plaintext password itself.
  • DISABLE_SECURITY_PLUGIN=true disables OpenSearch’s built-in authentication, which is fine for a single-node stack behind a firewall but not appropriate for a publicly exposed instance.
  • Port 12201/udp is the default GELF UDP input port, and 1514/udp is a common Syslog input port — you’ll enable the corresponding inputs from the Graylog UI after first boot.
  • Generating the Root Password Hash

    Graylog requires the root password as a SHA-256 hash, not plaintext, in the Compose environment variables. You can generate it directly from the command line:

    echo -n "your-strong-password" | sha256sum

    Copy the resulting hash (excluding the trailing filename marker) into GRAYLOG_ROOT_PASSWORD_SHA2.

    Starting the Stack

    With the file saved as docker-compose.yml, bring the stack up in detached mode:

    docker compose up -d
    docker compose ps
    docker compose logs -f graylog

    Graylog can take a minute or two to become reachable on first boot while it waits for OpenSearch to finish initializing. If you need to debug a slow or failing startup, the same log-reading techniques covered in this site’s Docker Compose logs debugging guide apply directly here — docker compose logs -f graylog is your primary tool for watching the ingestion and startup sequence in real time.

    Configuring Inputs After First Boot

    Once the Graylog Docker Compose stack is running, log in to the web UI at http://<your-host>:9000 with the root username and the plaintext password you hashed earlier. The first practical task is enabling an input so Graylog actually starts receiving logs.

    Enabling a GELF UDP Input

    GELF (Graylog Extended Log Format) is Graylog’s own structured logging format and is usually the simplest input to start with, especially if your applications already log via a GELF driver.

    1. Navigate to System → Inputs.
    2. Select GELF UDP from the dropdown and click Launch new input.
    3. Set the bind address to 0.0.0.0 and the port to 12201 (matching the port mapping in the Compose file).
    4. Save, and the input should show as running.

    You can test it immediately from any host with nc:

    echo '{"version": "1.1", "host": "test-host", "short_message": "hello from gelf"}' | nc -u -w1 localhost 12201

    If everything is wired correctly, this message appears in the Graylog search view within a few seconds.

    Enabling Docker Container Logging via GELF

    If the goal is to centralize logs from other Docker Compose stacks on the same or different hosts, point each container’s logging driver at Graylog directly instead of relying on docker compose logs. This can be set per-service in any other Compose file on your infrastructure:

    services:
      app:
        image: my-app:latest
        logging:
          driver: gelf
          options:
            gelf-address: "udp://your-graylog-host:12201"
            tag: "my-app"

    This is a natural next step once your Postgres Docker Compose or n8n self-hosted stacks are running — instead of docker compose logs on each host individually, every container ships its logs to one central Graylog instance where you can search across all of them at once.

    Persistence, Backups, and Data Volumes

    The Compose file above defines three named volumes — mongo_data, opensearch_data, and graylog_data. Understanding what lives in each matters before you script backups or plan a migration.

  • mongo_data holds all Graylog configuration: dashboards, users, roles, alert conditions, and index set definitions. This is small but critical — losing it means rebuilding your entire configuration by hand.
  • opensearch_data holds the actual indexed log messages. This grows continuously and is the volume you need to size and monitor most closely.
  • graylog_data holds Graylog server-specific state, including its internal node ID and some cached configuration.
  • A simple, scriptable backup approach for a single-node Graylog Docker Compose stack is to stop the stack briefly and archive the volume directories, or use docker run with a throwaway container to tar each volume without downtime:

    docker run --rm 
      -v mongo_data:/data 
      -v "$(pwd)":/backup 
      alpine tar czf /backup/mongo_data_backup.tar.gz -C /data .

    Repeat the same pattern for graylog_data if you want to preserve node identity across a restore. opensearch_data backups are usually handled separately via OpenSearch’s own snapshot API for larger deployments, since a raw tar of a live index can be inconsistent under write load.

    Scaling and Production Considerations

    A single-node Graylog Docker Compose file is fine for development, small teams, or moderate log volume, but a few things change as usage grows.

    Memory Sizing for OpenSearch

    The OPENSEARCH_JAVA_OPTS heap size in the example above (1GB) is a starting point, not a production recommendation. OpenSearch generally performs best when given roughly half of the host’s available RAM as heap, up to a ceiling of around 32GB (beyond which Java’s compressed object pointers stop being effective — a detail documented in the OpenSearch documentation). If searches feel slow or ingestion starts lagging, check heap usage before assuming the problem is elsewhere.

    Disk and Retention

    Log data is inherently disk-hungry. Configure Graylog’s index rotation and retention strategy under System → Indices to automatically delete or close old indices before disk fills up. Running out of disk space on the OpenSearch volume is one of the most common ways a Graylog Docker Compose stack becomes unresponsive, since OpenSearch enforces disk watermarks and will refuse writes when free space drops too low.

    Choosing Where to Host the Stack

    Because OpenSearch and Graylog together are memory- and disk-intensive, undersized VPS instances tend to struggle once real log volume arrives. If you’re standing up a dedicated logging host, providers like DigitalOcean and Hetzner offer VPS tiers with enough RAM and SSD-backed storage to run a single-node Graylog Docker Compose stack comfortably without needing to shard across multiple nodes.

    Networking and TLS

    For anything beyond local testing, put the Graylog web interface behind a reverse proxy with TLS termination rather than exposing port 9000 directly. This also lets you enforce authentication at the proxy layer as an additional safeguard, similar to the reverse-proxy patterns commonly used for other self-hosted Compose stacks like n8n.

    Troubleshooting Common Issues

    A few problems come up repeatedly with Graylog Docker Compose deployments, and most trace back to the same handful of causes.

  • Graylog container restarts in a loop — usually means it can’t reach OpenSearch yet, or the GRAYLOG_PASSWORD_SECRET/GRAYLOG_ROOT_PASSWORD_SHA2 values are malformed. Check docker compose logs graylog for the specific exception.
  • OpenSearch container exits immediately — often a vm.max_map_count kernel setting that’s too low on the host. This needs to be raised at the host OS level (sysctl -w vm.max_map_count=262144), not inside the container.
  • No logs appearing in search — confirm the relevant input is actually running under System → Inputs, and that the port mapping in your Compose file matches the port the input is bound to.
  • Web UI unreachable — check that GRAYLOG_HTTP_EXTERNAL_URI matches how you’re actually accessing the instance; a mismatch here can cause redirect loops in the browser.
  • If you’re new to debugging multi-container stacks generally, the techniques in this site’s guide on rebuilding Docker Compose services are useful for cleanly forcing a container to pick up a config change without tearing down the whole stack.


    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Does Graylog Docker Compose require Elasticsearch or OpenSearch specifically?
    Current Graylog releases officially support OpenSearch as the search backend. Older Graylog versions supported Elasticsearch, but new Graylog Docker Compose deployments should use OpenSearch to stay aligned with the officially supported and tested configuration.

    Can I run Graylog Docker Compose on a single small VPS?
    Yes, for low-to-moderate log volume. The main constraint is OpenSearch’s memory requirement — a stack with too little RAM will feel slow or become unresponsive under load, so size the host based on expected ingestion volume rather than guessing.

    How do I update Graylog to a newer version in this Compose setup?
    Change the image tag for the graylog service (and, if needed, the compatible OpenSearch/MongoDB versions per Graylog’s compatibility matrix), then run docker compose pull followed by docker compose up -d. Always check the release notes for breaking changes before upgrading a production instance.

    Is a Graylog Docker Compose setup suitable for production, or only for testing?
    A single-node setup is commonly used in production for smaller environments, but larger deployments typically move to a multi-node OpenSearch cluster and dedicated Graylog server nodes rather than a single Compose file, since a one-node stack has no built-in redundancy.

    Conclusion

    A Graylog Docker Compose stack gives you a fully working centralized logging platform — ingestion, storage, search, and dashboards — defined in a single reproducible file. The setup shown here covers MongoDB for configuration, OpenSearch for storage and search, and the Graylog server itself, along with the inputs, volumes, and sizing considerations that determine whether the stack stays healthy as log volume grows. Start with the minimal configuration, confirm ingestion works with a simple GELF test message, and then tune memory, retention, and networking as your actual log volume and team size demand. For further detail on the underlying container orchestration model this stack relies on, the official Docker Compose documentation is the authoritative reference.

  • Docker Compose Environment Variables: Complete Guide

    Docker Compose Environment Variables: A Complete Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever hardcoded a database password directly into a docker-compose.yml file and then panicked when you realized it got pushed to a public GitHub repo, you already understand why managing a docker compose environment correctly matters. Environment variables are the standard way to inject configuration into containers without baking secrets or environment-specific values into your images. Get this wrong and you end up with leaked credentials, brittle deployments, or containers that behave differently in staging than in production.

    This guide walks through every practical way to set, override, and debug environment variables in Docker Compose, plus the precedence rules that trip up almost everyone the first time.

    Why Environment Variables Matter in a Docker Compose Setup

    Environment variables let you decouple your application’s configuration from its code. Instead of rebuilding an image every time a database URL or API key changes, you pass that value in at runtime. This is the core idea behind the 12-Factor App methodology, and Docker Compose was built with it in mind.

    A well-structured docker compose environment gives you:

  • Portable images that work identically across dev, staging, and production
  • A single source of truth for configuration per environment
  • The ability to keep secrets out of version control
  • Easier debugging, since you can inspect exactly what values a container received
  • If you’re new to Compose in general, it’s worth reading our Docker Compose networking guide first, since environment variables and networking often intersect (service discovery, hostnames, ports).

    Setting Variables with the environment Key

    The most direct way to pass environment variables into a service is the environment key inside your docker-compose.yml. You can write it as a list or a mapping:

    services:
      web:
        image: node:20-alpine
        environment:
          - NODE_ENV=production
          - API_PORT=3000
          - DEBUG=false

    or the map form:

    services:
      web:
        image: node:20-alpine
        environment:
          NODE_ENV: production
          API_PORT: "3000"
          DEBUG: "false"

    Both are functionally identical. The map form is easier to read and merge in overlay files, so most teams standardize on it. Numeric and boolean-looking values should be quoted, because YAML will otherwise try to parse 3000 or false as their native types instead of strings, which can cause subtle bugs in some applications.

    You can also pull values from your shell into the compose file:

    services:
      web:
        environment:
          - API_KEY=${API_KEY}

    Running API_KEY=abc123 docker compose up will inject abc123 into the container. This is useful for CI pipelines where secrets are already exported as shell variables.

    Using env_file for Larger Configuration Sets

    Once you have more than a handful of variables, listing them inline gets unwieldy. The env_file directive lets you point to a plain text file instead:

    services:
      web:
        image: node:20-alpine
        env_file:
          - .env.production

    The file itself is just KEY=VALUE pairs, one per line:

    NODE_ENV=production
    API_PORT=3000
    DATABASE_URL=postgres://user:pass@db:5432/appdb

    A few rules to know:

  • Lines starting with # are treated as comments
  • Blank lines are ignored
  • Values are not shell-expanded inside the file (no $HOME substitution)
  • You can list multiple env_file entries; later files override earlier ones
  • This is the cleanest option when you’re managing dozens of variables across multiple services, and it keeps your docker-compose.yml readable. Just make sure any file containing real secrets is listed in .gitignore — a mistake we cover in more depth in our container secrets management guide.

    The Special .env File and Variable Substitution

    Docker Compose treats a file literally named .env in the same directory as your docker-compose.yml differently from other env files. It isn’t injected into containers automatically — instead, Compose reads it to substitute ${VARIABLE} placeholders inside the compose file itself, before the file is even parsed.

    # .env
    POSTGRES_VERSION=16
    POSTGRES_PASSWORD=supersecret
    COMPOSE_PROJECT_NAME=myapp

    services:
      db:
        image: postgres:${POSTGRES_VERSION}
        environment:
          POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

    This distinction confuses a lot of people: .env drives compose-file templating, while env_file: drives container environment injection. You can use both together — a .env file for values referenced in the YAML structure (image tags, project name, ports) and env_file: entries for values that need to land inside the running container.

    You can verify what Compose actually resolved with:

    docker compose config

    This prints the fully rendered configuration with all variables substituted, which is the fastest way to catch a typo before it reaches production.

    Using Multiple Compose Files to Override Environment Values

    Docker Compose supports merging multiple compose files together, which is handy for maintaining a base environment configuration and layering environment-specific overrides on top:

    docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

    # docker-compose.prod.yml
    services:
      web:
        environment:
          NODE_ENV: production
          LOG_LEVEL: warn

    Values in the later file override matching keys in the base file, following the same merge rules as top-level list-vs-map behavior in the compose spec. This pattern is especially useful for keeping a shared docker-compose.yml for local development and a slim docker-compose.prod.yml that only overrides the handful of variables that actually differ — log level, replica counts, or a production database URL. Compose also automatically loads a file named docker-compose.override.yml if it exists in the same directory, without needing the -f flag at all, which is convenient for keeping personal local overrides out of version control.

    Precedence: Which Value Actually Wins

    When the same variable is set in multiple places, Compose applies a strict precedence order, from highest to lowest priority:

    1. Variables set with docker compose run -e VAR=value
    2. Variables defined under the environment key in the compose file
    3. Variables from files listed under env_file
    4. Variables set in the shell environment before running docker compose up
    5. Variables defined in a .env file (used only for compose-file substitution, not container injection)

    In practice, this means an environment: entry will always override a matching key from env_file:, even if env_file is listed. If your container isn’t picking up the value you expect, check every one of these layers — this is the single most common source of “why isn’t my environment variable working” issues reported on the official Docker Compose documentation and forums.

    Managing Secrets Safely

    Environment variables are convenient, but they aren’t a secure secrets store by themselves — anything passed via environment: or .env is visible to anyone who can run docker inspect or docker compose config against the running stack. For sensitive values in production, consider:

  • Using Docker Compose’s native secrets: block, which mounts values as files rather than environment variables
  • Pulling secrets at runtime from a vault (HashiCorp Vault, AWS Secrets Manager, Doppler)
  • Restricting file permissions on .env files (chmod 600 .env)
  • Never committing .env or env_file targets to git — add them to .gitignore immediately after creating them
  • Using separate .env files per environment (.env.dev, .env.staging, .env.production) and loading the correct one explicitly
  • If you’re deploying these stacks on your own infrastructure, make sure the host itself is locked down too — a misconfigured environment variable is a much smaller risk than an exposed Docker socket on a public-facing VPS. Providers like DigitalOcean make it straightforward to spin up a hardened droplet specifically for container workloads, with private networking so your secrets never traverse the public internet.

    Once your environment variables are correctly configured, the next challenge is observability — knowing when a misconfigured variable causes a service to silently fail. Tools like BetterStack can monitor container logs and alert you the moment a service crashes from a missing environment variable, which saves hours of manual debugging in production.

    Debugging a Broken Environment

    When a container isn’t seeing the variable you expect, work through this checklist:

    # See what Compose resolved before starting anything
    docker compose config
    
    # Check the environment inside a running container
    docker compose exec web env
    
    # Check a stopped/crashed container's environment
    docker inspect <container_id> --format '{{.Config.Env}}'

    Common causes of mismatched values:

  • A typo in the variable name (DATABSE_URL vs DATABASE_URL)
  • Quoting issues in the .env file (Compose does not strip surrounding quotes consistently across versions)
  • An environment: entry silently overriding an env_file: value
  • Forgetting to restart the container after changing .env — Compose doesn’t hot-reload environment changes; you need docker compose up -d --force-recreate
  • A Real-World Multi-Service Example

    Here’s how these pieces fit together in a typical stack with a web app, a database, and Redis cache. Assume the following directory structure:

    myapp/
    ├── docker-compose.yml
    ├── docker-compose.prod.yml
    ├── .env
    ├── .env.production
    └── app/

    The base docker-compose.yml stays generic:

    services:
      web:
        build: ./app
        env_file:
          - .env.production
        environment:
          REDIS_URL: redis://cache:6379
        depends_on:
          - db
          - cache
      db:
        image: postgres:${POSTGRES_VERSION}
        environment:
          POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
          POSTGRES_DB: ${POSTGRES_DB}
        volumes:
          - db-data:/var/lib/postgresql/data
      cache:
        image: redis:7-alpine
    
    volumes:
      db-data:

    Notice the layering: ${POSTGRES_VERSION} and ${POSTGRES_PASSWORD} come from the auto-loaded .env file and get substituted into the YAML structure itself, while REDIS_URL is hardcoded because it never changes between environments, and the application’s own secrets live in .env.production, loaded via env_file:. This split keeps infrastructure-level values (image versions, resource names) separate from application-level configuration (API keys, connection strings), which makes the compose file easier to audit at a glance.

    Common Mistakes to Avoid

  • Committing .env files with real secrets to version control instead of using .env.example as a template
  • Assuming docker compose up automatically picks up changes to .env — you need to recreate containers
  • Mixing up .env (compose-file substitution) with env_file: (container injection) and expecting them to behave the same way
  • Quoting numeric or boolean values inconsistently, which causes type mismatches inside the application
  • Hardcoding environment-specific values like database hostnames directly into the image instead of injecting them at runtime
  • Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: What’s the difference between .env and env_file in Docker Compose?
    A: .env is read by Compose itself to substitute ${VAR} placeholders inside the docker-compose.yml file before parsing. env_file injects variables directly into a container’s runtime environment. They serve different purposes and can be used together.

    Q: Can I use multiple .env files with different names?
    A: Compose only auto-loads a file literally named .env. To use a differently named file for substitution, pass it explicitly with docker compose --env-file .env.staging up.

    Q: Does environment: override env_file: if both set the same variable?
    A: Yes. Values under the environment: key always take precedence over matching keys loaded from env_file:.

    Q: Are environment variables secure enough for passwords and API keys?
    A: For low-stakes local development, generally fine. For production secrets, use Docker’s secrets: block, a vault service, or your cloud provider’s secrets manager instead, since environment variables are visible via docker inspect.

    Q: Why isn’t my .env file being picked up?
    A: The most common cause is the file not being in the same directory as docker-compose.yml, or a typo in the filename (it must be exactly .env, not env or .env.txt).

    Q: How do I pass a variable from the shell into a container at runtime?
    A: Export it in your shell and reference it in the compose file, e.g. environment: - API_KEY=${API_KEY}, then run API_KEY=abc123 docker compose up.

    Wrapping Up

    A clean docker compose environment setup comes down to picking the right mechanism for the job: environment: for a handful of values, env_file: for larger sets, and .env for compose-file-level substitution. Once you understand the precedence order and keep secrets out of version control, most of the “why isn’t this variable working” headaches disappear. For a deeper look at structuring multi-service stacks around these variables, check out our complete Docker Compose guide. Test your setup locally with docker compose config before every deploy, and you’ll catch most environment-related surprises long before they reach production.

  • Docker Compose Build: Complete Guide to Building Images

    Docker Compose Build: A Practical Guide to Building Images Correctly

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever run docker compose up and wondered why your code changes weren’t reflected in the container, the answer almost always traces back to how docker compose build works. This command is the backbone of any local development workflow that uses custom images instead of pulling pre-built ones from a registry, and understanding its mechanics will save you hours of confused debugging.

    This guide covers everything from the basic syntax to advanced caching strategies, build arguments, multi-stage builds, and the most common pitfalls developers run into when building images through Compose.

    What docker compose build Actually Does

    When you define a build key in your docker-compose.yml, Compose doesn’t just run a generic build — it reads the context, Dockerfile, and any build-time configuration you’ve specified, then hands the job off to the Docker build engine (BuildKit, by default on modern installs). The resulting image is tagged according to your service name and project, unless you override it with an explicit image key.

    Here’s a minimal example:

    services:
      web:
        build:
          context: .
          dockerfile: Dockerfile
        ports:
          - "3000:3000"

    Running the build is straightforward:

    docker compose build

    This builds every service in the file that has a build section. If you only want to rebuild one service, target it by name:

    docker compose build web

    Difference Between build and up

    A common point of confusion is the relationship between docker compose build and docker compose up --build. Running docker compose up alone will use existing images if they exist and won’t rebuild automatically, even if your Dockerfile changed. Adding --build forces a rebuild before starting containers:

    docker compose up --build

    In practice, most developers alias this command or bake it into a Makefile because forgetting --build after editing a Dockerfile is one of the most common sources of “why isn’t my change showing up” bug reports. If you’re running a multi-service stack, this is also worth combining with a solid Docker Compose networking setup so services can reach each other reliably after rebuilds.

    Build Caching and Why It Matters

    Docker layers each instruction in your Dockerfile, and it caches those layers so subsequent builds are faster — as long as nothing upstream of a given instruction has changed. docker compose build respects this same caching behavior. The order of instructions in your Dockerfile directly affects build speed:

    FROM node:20-alpine
    WORKDIR /app
    COPY package*.json ./
    RUN npm ci
    COPY . .
    CMD ["node", "server.js"]

    By copying package.json before the rest of the source code, npm ci only reruns when dependencies actually change, not on every source edit. This pattern alone can cut build times from minutes to seconds during active development.

    If you need to bypass the cache entirely — for example, after a base image security patch — use:

    docker compose build --no-cache

    For a deeper dive into layer caching and image size reduction, the official Docker build cache documentation is worth reading in full, especially the section on cache invalidation triggers.

    Using Build Arguments

    Build arguments let you pass values into the build process without hardcoding them into the Dockerfile. This is especially useful for things like version pinning or environment-specific configuration that shouldn’t be baked into a production image.

    services:
      api:
        build:
          context: .
          args:
            NODE_ENV: production
            APP_VERSION: 1.4.2

    Your Dockerfile then references these with ARG:

    FROM node:20-alpine
    ARG NODE_ENV
    ARG APP_VERSION
    ENV NODE_ENV=${NODE_ENV}
    LABEL version=${APP_VERSION}
    WORKDIR /app
    COPY . .
    RUN npm ci --omit=dev
    CMD ["node", "server.js"]

    You can also override args at build time from the CLI:

    docker compose build --build-arg APP_VERSION=1.5.0 api

    Keep in mind that build args are visible in the image history unless you use secrets mounts, so never pass credentials or API keys this way — that’s a common security misstep. If you’re handling secrets in your build pipeline, pair this with proper Docker security hardening practices to avoid leaking sensitive values into image layers.

    Multi-Stage Builds With Compose

    Multi-stage builds are one of the most effective ways to keep production images small while still having full build tooling available during compilation. Compose handles multi-stage Dockerfiles natively — you just need to tell it which target to build.

    # Build stage
    FROM node:20 AS builder
    WORKDIR /app
    COPY package*.json ./
    RUN npm ci
    COPY . .
    RUN npm run build
    
    # Production stage
    FROM node:20-alpine AS production
    WORKDIR /app
    COPY --from=builder /app/dist ./dist
    COPY --from=builder /app/node_modules ./node_modules
    CMD ["node", "dist/server.js"]

    In your docker-compose.yml, specify the target stage:

    services:
      app:
        build:
          context: .
          target: production

    This approach routinely shrinks final image sizes by 60-80% compared to single-stage builds that ship build tools and dev dependencies into production.

    Parallel Builds and Performance

    By default, docker compose build builds services sequentially. If you have multiple independent services, you can speed things up significantly by building in parallel:

    docker compose build --parallel

    This only helps when services don’t depend on each other’s build output. If one service’s Dockerfile copies artifacts from another service’s build context, parallel builds won’t help and could even cause race conditions — structure your Dockerfiles to avoid cross-service file dependencies where possible.

    Common Build Errors and How to Fix Them

    A few errors show up constantly in docker compose build logs:

  • “failed to compute cache key” — usually means a file referenced in COPY or ADD doesn’t exist in the build context. Double-check your .dockerignore isn’t excluding something you need.
  • “context deadline exceeded” — often a network timeout pulling a base image or dependency. Retry, or check if you’re behind a corporate proxy that needs Docker daemon configuration.
  • “no space left on device” — your local Docker storage is full of dangling images and build cache. Run docker system prune to reclaim space (use -a to also remove unused images, but be careful on shared machines).
  • Stale dependency errors after editing package.json — usually means the layer cache wasn’t invalidated correctly. Force a clean rebuild with --no-cache for that one service.
  • Build succeeds locally but fails in CI — almost always an environment difference, like a missing .env file or a platform architecture mismatch (Apple Silicon vs. x86 CI runners).
  • For the platform mismatch issue specifically, you can pin the target platform explicitly:

    services:
      app:
        build:
          context: .
          platforms:
            - linux/amd64

    Debugging a Failing Build Interactively

    When a build fails partway through and the error message isn’t clear, it helps to build up to the failing layer and inspect the intermediate state:

    docker build --target production -t debug-image .
    docker run -it debug-image sh

    This drops you into a shell inside the partially built image so you can manually run the commands that failed and see the actual output, rather than guessing from truncated CI logs.

    Optimizing Your Compose Build Workflow

    A few practical habits make a real difference for teams running Compose day to day:

  • Pin base image versions (node:20.11-alpine instead of node:latest) to avoid surprise breakage when upstream images update.
  • Use .dockerignore aggressively to keep build contexts small — excluding node_modules, .git, and test fixtures can cut context upload time dramatically on large repos.
  • Separate docker-compose.yml and docker-compose.override.yml so local dev overrides (bind mounts, debug ports) don’t leak into production configs.
  • Tag images explicitly with image: alongside build: so you can push the same image you tested locally without an unnecessary rebuild.
  • Run builds in CI with --no-cache periodically (e.g., weekly) to catch cache-related drift that wouldn’t otherwise surface.
  • If your team is deploying these built images to a VPS or cloud instance rather than just running locally, it’s worth comparing providers before committing to one. DigitalOcean offers straightforward Droplets that work well for small-to-medium Compose-based deployments, and their managed container registry integrates cleanly with docker compose push workflows if you want to skip building images repeatedly on the target server.

    For teams running builds and deployments at higher frequency, keeping an eye on server resource usage during build spikes matters too — CPU and memory can spike hard during compilation-heavy multi-stage builds, so monitoring your host is worth setting up before it becomes a problem. Once your images are built and running, pairing them with proper container health checks and monitoring will help you catch build-related regressions before they hit production.

    Choosing Infrastructure for Your Build Pipeline

    If you’re self-hosting your CI runners or build servers, the underlying hardware matters more than people expect — slow disk I/O alone can double build times on image-heavy Dockerfiles. Hetzner is a solid budget option for dedicated build servers with fast NVMe storage, which noticeably speeds up layer extraction and cache writes compared to standard cloud block storage.

    Frequently Asked Questions

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: What’s the difference between docker build and docker compose build?
    A: docker build builds a single image from a Dockerfile and context you specify manually on the command line. docker compose build reads your docker-compose.yml and builds every service with a build: section, using the context, args, and target defined in that file — it’s essentially a batch wrapper that keeps your build configuration version-controlled alongside your service definitions.

    Q: Why doesn’t docker compose up rebuild my image automatically?
    A: Compose intentionally avoids rebuilding on every up to save time — it assumes the existing image is still valid unless told otherwise. Use docker compose up --build to force a rebuild, or run docker compose build separately before starting containers.

    Q: How do I rebuild only one service instead of the whole stack?
    A: Pass the service name directly: docker compose build <service-name>. This only rebuilds that service’s image, leaving others untouched and speeding up iteration in multi-service projects.

    Q: Can I use build secrets with docker compose build without exposing them in the image?
    A: Yes. Use BuildKit secret mounts in your Dockerfile with RUN --mount=type=secret,id=mysecret, and reference the secret in your Compose file’s build.secrets section. This keeps sensitive values out of the final image layers and build history entirely.

    Q: Why is my build cache not being used even though nothing changed?
    A: Check whether your build context includes files that change on every run, like log files or generated artifacts not excluded in .dockerignore. Even a timestamp file in the context can invalidate cache for every subsequent COPY . . instruction.

    Q: Does docker compose build push images to a registry?
    A: No, build only builds and tags images locally. To push, tag the service with an image: field pointing to your registry path, then run docker compose push separately after a successful build.

    Getting comfortable with docker compose build — its caching behavior, build args, and multi-stage targets — pays off quickly once you’re managing more than a couple of services. Start by auditing your existing Dockerfiles for cache-friendly instruction ordering, since that’s usually the fastest win available with zero risk.