Author: admin_ts

  • Docker Compose Logs: The Complete Debugging Guide

    Docker Compose Logs: A Practical Guide to Debugging Multi-Container Apps

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    When a container silently dies or a service starts throwing errors at 2 AM, docker compose logs is usually the first command you reach for. It’s the fastest way to see what’s actually happening inside your stack without SSHing into individual containers or digging through log files scattered across the filesystem.

    If you’re running Compose stacks in production — whether that’s a media server setup, a CI runner, or a full application backend — understanding log output is non-negotiable. Let’s get into it.

    Why docker compose logs Matters

    Unlike docker logs, which only works on a single container, docker compose logs aggregates output from every service defined in your docker-compose.yml. This matters because most real-world failures aren’t isolated — a database connection timeout in your API service might actually be caused by the database container failing to start, and you won’t see that correlation unless you’re looking at both logs side by side.

    If you’re new to container orchestration in general, our Docker networking fundamentals guide is a good companion piece before diving deeper into log debugging.

    Basic Syntax and Usage

    The core command is straightforward:

    docker compose logs

    Run this from the directory containing your docker-compose.yml, and it dumps the combined stdout/stderr output from every running service, prefixed by service name and color-coded in your terminal.

    To target a single service:

    docker compose logs web

    To target multiple specific services:

    docker compose logs web db redis

    This is useful when you have a stack with 10+ services (common in microservice architectures) and only care about the ones involved in a specific request path.

    Following Logs in Real Time

    By default, docker compose logs prints everything and exits. To keep the stream open and watch new log lines as they arrive — similar to tail -f — add the -f or --follow flag:

    docker compose logs -f

    This is the flag you’ll use most often during active debugging sessions. Combine it with a service filter to watch just one component while you reproduce a bug:

    docker compose logs -f api

    Press Ctrl+C to stop following without stopping the containers themselves.

    Limiting Output with Tail

    On long-running production services, log history can be enormous. Dumping the entire history to your terminal is slow and mostly useless. Use --tail to limit output to the most recent N lines:

    docker compose logs --tail=100 api

    Combine --tail with --follow to get recent context plus a live stream:

    docker compose logs --tail=50 -f api db

    This pattern — recent history plus live tail — is the single most useful log-debugging habit you can build.

    Adding Timestamps

    By default, Compose logs don’t show timestamps, which makes it hard to correlate events across services or figure out how much time elapsed between two log lines. Add the -t or --timestamps flag:

    docker compose logs -t --tail=100 api

    Output looks like this:

    api_1  | 2026-07-02T14:22:01.123456789Z Starting server on port 3000
    api_1  | 2026-07-02T14:22:03.987654321Z Connected to database

    Timestamps are essential when you’re trying to line up an error in your app logs with a corresponding event in your reverse proxy or database logs.

    Filtering by Time Range

    If you know roughly when an incident occurred, you don’t need to scroll through hours of noise. Use --since and --until to bound the output:

    docker compose logs --since 2026-07-02T14:00:00 --until 2026-07-02T14:30:00 api

    You can also use relative durations:

    docker compose logs --since 30m api

    This pulls only logs from the last 30 minutes — extremely useful when triaging a fresh incident without wading through days of accumulated output.

    Disabling the Log Prefix

    When you’re only looking at one service and want cleaner output (for piping into grep or another tool), drop the service-name prefix with --no-log-prefix:

    docker compose logs --no-log-prefix api | grep ERROR

    This is handy when scripting log analysis or feeding output into external tools.

    Piping and Searching Logs

    docker compose logs output is just text, so standard Unix tools work fine on top of it. A few practical patterns:

  • Search for errors across all services: docker compose logs | grep -i error
  • Count occurrences of a specific warning: docker compose logs | grep -c "connection refused"
  • Save a snapshot for later analysis: docker compose logs --no-color > incident-2026-07-02.log
  • Watch only 500 errors in a web service: docker compose logs -f web | grep "500"
  • Combine tail and search for a fast triage loop: docker compose logs --tail=200 db | grep -i "deadlock"
  • These one-liners solve 90% of the ad-hoc debugging you’ll do day to day, without needing a dedicated log aggregation tool.

    Common Problems and How Logs Help Diagnose Them

    Container Restart Loops

    If a container keeps restarting, docker compose ps will show a high restart count, but docker compose logs tells you why. Run it without --tail truncation on a small window right after the crash:

    docker compose logs --since 5m --tail=200 worker

    Look for the last few lines before the process exits — that’s almost always where the actual error (missing environment variable, failed migration, out-of-memory kill) shows up.

    Silent Startup Failures

    Sometimes a service reports as “running” but isn’t actually healthy — for example, a database that’s accepting connections but hasn’t finished initializing. Following logs during startup catches this:

    docker compose up -d
    docker compose logs -f db

    Watch for the specific “ready to accept connections” message your database emits before assuming dependent services can connect.

    Cross-Service Correlation

    When your API returns a 500 error, you often need to see what the database or cache was doing at the exact same millisecond. Use timestamps and pull both services together:

    docker compose logs -t --since 10m api redis

    Having both streams interleaved by timestamp makes root-cause analysis dramatically faster than tailing them in separate terminal windows.

    For a deeper dive into container health checks that complement log-based debugging, see our Docker health check configuration guide.

    Beyond docker compose logs: When to Use a Real Logging Stack

    The built-in logging command is great for local development and quick production triage, but it has real limits: no persistent storage past the log driver’s retention, no full-text search across weeks of history, and no alerting. Once you’re running more than a handful of services, it’s worth shipping logs to a dedicated platform.

    For teams that don’t want to run their own Elasticsearch or Loki stack, a hosted monitoring and log management service removes the operational overhead. BetterStack offers log management and uptime monitoring that integrates cleanly with Docker Compose setups via simple log-driver configuration, so you get searchable, retained logs and alerting without babysitting another piece of infrastructure.

    If you’re hosting your Compose stack yourself, the underlying VPS matters too — slow disk I/O can bottleneck log writes on high-throughput services. Providers like DigitalOcean and Hetzner both offer NVMe-backed instances that hold up well under heavy container logging workloads, and either is a solid choice for a self-managed Compose deployment.

    Configuring the Logging Driver

    By default, Compose uses the json-file driver, which can grow unbounded if you don’t set limits. Add size and rotation limits directly in your docker-compose.yml:

    services:
      api:
        image: myapp:latest
        logging:
          driver: "json-file"
          options:
            max-size: "10m"
            max-file: "3"

    This caps each log file at 10MB and keeps a maximum of 3 rotated files per container, preventing a chatty service from filling your disk.

    Quick Reference: Useful Flags

  • -f, --follow — stream logs continuously
  • --tail=N — show only the last N lines
  • -t, --timestamps — prepend ISO 8601 timestamps
  • --since / --until — bound output by time
  • --no-log-prefix — remove service name prefix
  • --no-color — strip ANSI color codes (useful when redirecting to a file)
  • Memorize --tail and -f first — they cover the majority of day-to-day debugging. Add -t and --since once you’re troubleshooting timing-sensitive issues across multiple services.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: What’s the difference between docker logs and docker compose logs?
    A: docker logs <container> targets a single container by name or ID. docker compose logs operates on an entire Compose project, aggregating and interleaving output from all defined services (or a filtered subset) in one command.

    Q: Why don’t I see any output when I run docker compose logs?
    A: This usually means the containers aren’t running, or they’re writing logs to a file inside the container instead of stdout/stderr. Docker only captures stdout/stderr by default — check your app’s logging configuration to make sure it logs to the console, not a file.

    Q: How do I clear or reset the logs for a service?
    A: Compose doesn’t have a built-in “clear logs” command. You typically need to either restart the container (which starts a fresh log file depending on your driver) or manually truncate the underlying log file on the host if you have access to it.

    Q: Can I export logs to a file for sharing with a teammate?
    A: Yes. Run docker compose logs --no-color > output.log to save a clean, ANSI-free text file that’s easy to share or attach to a ticket.

    Q: Does --tail work with --follow at the same time?
    A: Yes, and it’s a very common combination. docker compose logs --tail=100 -f api shows the last 100 lines immediately, then continues streaming new lines as they arrive.

    Q: Why are my logs missing timestamps even in the raw JSON log file?
    A: The json-file driver does store timestamps internally, but docker compose logs only displays them when you pass -t/--timestamps. Without that flag, Compose strips them from the terminal output for readability.

    Wrapping Up

    docker compose logs is deceptively simple but covers most of what you need for day-to-day container debugging: filtering by service, following in real time, bounding by time range, and piping into standard Unix tools for search. Combine it with sensible logging-driver limits and, once your stack grows, a proper log aggregation service to avoid losing critical history.

    For related deployment topics, check out our Docker Compose production checklist to make sure your logging setup fits into a broader operational strategy.

  • Docker Compose Down: Full Guide to Stopping Stacks

    Docker Compose Down: The Complete Guide to Stopping and Cleaning Up Your Stack

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you run multi-container applications, you already know docker compose up is the easy part. The command that actually trips people up — and occasionally wipes out data they didn’t mean to lose — is docker compose down. It looks simple on the surface, but the flags you attach to it determine whether you get a clean, reversible stop or a full teardown that deletes volumes, networks, and images.

    This guide covers exactly what docker compose down does under the hood, every major flag, how it differs from docker compose stop, and the mistakes that cause the most support tickets and Stack Overflow posts. We’ll also touch on how this fits into a broader DevOps deployment workflow if you’re managing production stacks.

    What Does docker compose down Actually Do?

    When you run:

    docker compose down

    Compose does the following, in order:

  • Stops all running containers defined in your docker-compose.yml
  • Removes those containers (not just stops them — they’re gone)
  • Removes any networks created by Compose for the project
  • Leaves named volumes and images untouched by default
  • This is the critical distinction from docker compose stop, which only pauses containers without deleting them. If you want to preserve container state for a quick restart, stop is the safer choice. If you want a clean slate — containers and networks gone, but data intact — down is what you want.

    # Just pause everything, keep containers on disk
    docker compose stop
    
    # Stop AND remove containers + networks
    docker compose down

    Why Containers Get Removed But Volumes Don’t

    Docker’s design philosophy treats containers as disposable and volumes as durable. Containers hold your application process; volumes hold your data (database files, uploaded media, logs). docker compose down respects that separation by default — it tears down the disposable layer and leaves the durable layer alone unless you explicitly tell it not to.

    This matters enormously if you’re running something like PostgreSQL, MySQL, or a media server with persistent libraries. Running plain docker compose down and back up with docker compose up -d will not touch your database contents, because the volume survives the teardown.

    The Flags That Actually Matter

    -v / --volumes: Deleting Named and Anonymous Volumes

    This is the flag that causes the most accidental data loss:

    docker compose down -v

    Adding -v removes named volumes declared in the volumes: section of your compose file, along with anonymous volumes attached to containers. If your Postgres data lives in a named volume like pgdata:, this command deletes it permanently. There’s no confirmation prompt — it just happens.

    Rule of thumb: never run docker compose down -v on a production stack unless you have a verified backup. Test this in a scratch environment first, and consider backing up with a tool like Restic or a scheduled pg_dump before any teardown that touches volumes.

    --rmi: Removing Images Too

    If you also want to reclaim disk space by deleting images built or pulled for the stack:

    docker compose down --rmi all

    Options here are all (remove every image used by any service) or local (only remove images that don’t have a custom tag, i.e., ones Compose built itself). This is useful in CI pipelines where you rebuild from scratch every run, but it’s overkill for a local dev loop where rebuilding images repeatedly wastes time and bandwidth.

    --remove-orphans: Cleaning Up Renamed or Deleted Services

    When you remove a service from your docker-compose.yml but its container is still running from a previous up, Compose calls it an “orphan.” By default, down won’t touch containers it doesn’t recognize from the current file. Force it with:

    docker compose down --remove-orphans

    This is especially common after refactoring a compose file — renaming a service from web to frontend, for example, leaves the old web container orphaned and still consuming resources.

    -t / --timeout: Controlling Shutdown Grace Period

    By default, Compose sends SIGTERM and waits 10 seconds before force-killing with SIGKILL. For services that need longer to flush writes or close connections gracefully (databases, message queues), increase the timeout:

    docker compose down -t 30

    For stacks with fast-shutdown stateless services, you can shrink this to speed up CI teardown:

    docker compose down -t 2

    Combining Flags for a Full Reset

    When you genuinely want to nuke everything — containers, networks, volumes, images, and orphans — in a disposable dev or CI environment:

    docker compose down -v --rmi all --remove-orphans

    This is the command CI pipelines commonly run at the end of an integration test job to guarantee no leftover state pollutes the next run.

    Common Scenarios and How to Handle Them

    Scenario 1: Restarting a Stack Without Losing Data

    docker compose down
    docker compose up -d

    Safe. Networks and containers are recreated fresh, but named volumes persist.

    Scenario 2: Wiping a Dev Environment Completely

    docker compose down -v --remove-orphans

    Use this when your local Postgres or Redis data has gotten into a weird state and you’d rather start clean than debug it.

    Scenario 3: Targeting a Specific Project When You Have Multiple Stacks

    If you run several Compose projects on the same host, specify the project name explicitly to avoid tearing down the wrong stack:

    docker compose -p myproject down

    This matters a lot on shared VPS environments where multiple teams or apps run side by side — mixing up project names is an easy way to take down someone else’s containers. If you’re managing several stacks on a single VPS, it’s worth reading up on Docker network isolation strategies to keep projects from stepping on each other.

    Scenario 4: down Hangs or Times Out

    If a container refuses to stop gracefully within the timeout window, Compose force-kills it, but you might see:

    ERROR: for web  Container did not stop within 10 seconds

    Fix it by increasing the timeout or checking whether your app’s entrypoint script is trapping SIGTERM correctly. Some Node.js and Python apps swallow the signal without exiting, which forces the timeout every time. Adding proper signal handling in your entrypoint (or using tini) usually resolves this. For a deep dive on signal handling inside containers, the official Docker docs on stopping containers are worth reading.

    docker compose down vs Related Commands

  • docker compose stop — pauses containers, keeps them on disk for a fast restart
  • docker compose down — stops and removes containers + networks, keeps volumes/images by default
  • docker compose rm — removes stopped containers without touching networks (rarely needed once you use down)
  • docker compose kill — force-kills containers immediately, skipping graceful shutdown entirely
  • If your goal is a quick pause-and-resume during development, stop plus start is faster since it skips network and container recreation. Reserve down for when you actually want to reset the project’s runtime state.

    Automating Safe Teardowns in Scripts

    If you’re scripting deployments or CI jobs, wrap the teardown in something explicit rather than relying on defaults:

    #!/usr/bin/env bash
    set -euo pipefail
    
    echo "Tearing down stack (volumes preserved)..."
    docker compose down --remove-orphans -t 15
    
    echo "Done."

    Keeping volume deletion out of your default script prevents a copy-paste accident from wiping production data. Make volume deletion a deliberate, separate command that requires a human to type it out.

    Hosting Considerations for Compose Stacks

    Where you host your Compose stack affects how forgiving these commands are. On a resource-constrained VPS, an orphaned container silently eating RAM can push you into swap and degrade every other service on the box. If you’re outgrowing a budget VPS and need predictable CPU/RAM for Docker workloads, providers like DigitalOcean offer straightforward droplet sizing built for exactly this kind of container hosting, and Hetzner is a solid budget option for dev/staging stacks where you’re tearing things down and rebuilding often. For monitoring container health so a bad down/up cycle doesn’t go unnoticed, BetterStack gives you uptime and log monitoring without much setup overhead.

    If your stack sits behind a domain and you want to keep DNS and CDN caching predictable while you restart backend containers, Cloudflare is worth having in front of anything public-facing — it also means a down/up cycle doesn’t immediately expose a connection-refused error to visitors while containers spin back up.

    For teams optimizing how these guides and docs get found organically, tracking keyword rankings with a tool like SE Ranking helps you see which Docker/DevOps topics are actually driving traffic before you invest more writing time in them.

    Checking What Would Be Removed Before Running It

    Compose doesn’t have a true dry-run flag for down, but you can inspect what’s currently active before tearing anything down:

    docker compose ps
    docker compose config --services
    docker volume ls

    Running these three commands first tells you exactly which containers, services, and volumes exist under the current project name, so down -v doesn’t surprise you.

    Troubleshooting Checklist

  • Network still exists after down — another running project may share the network name; check with docker network ls
  • Volume wasn’t deleted despite -v — the volume might be marked external: true in your compose file, which Compose deliberately never deletes
  • “No such service” errors — you’re likely running down from a directory without the matching docker-compose.yml, or with a mismatched -f file path
  • Containers reappear after down — a process manager like systemd or a restart: always policy combined with an external supervisor may be recreating them; check for a wrapping service definition
  • If you’re still getting familiar with the broader Compose command set beyond teardown, our guide to essential Docker Compose commands covers up, logs, exec, and ps in the same practical style.

    Recommended: Ready to put this into practice? SE Ranking is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Does docker compose down delete my data?
    Not by default. It removes containers and networks but preserves named volumes. Data loss only happens if you explicitly add the -v or --volumes flag.

    What’s the difference between docker compose down and docker compose stop?
    stop pauses containers without removing them, so a subsequent start is fast. down removes the containers and networks entirely, requiring Compose to recreate them on the next up.

    How do I remove only some services instead of the whole stack?
    docker compose down doesn’t support targeting individual services — it’s all-or-nothing for the project. To stop a single service without touching the rest, use docker compose stop <service_name> or docker compose rm -s -f <service_name>.

    Why didn’t my volume get deleted even with -v?
    If the volume is declared as external: true in your compose file, Docker treats it as managed outside the project’s lifecycle and never deletes it automatically, regardless of flags.

    Is it safe to run docker compose down -v in production?
    Only if you have a verified, tested backup of every volume involved. There is no undo. Many teams explicitly ban -v from production deployment scripts and require it to be run manually with confirmation.

    How do I remove orphaned containers left over from an old compose file?
    Run docker compose down --remove-orphans, which detects and removes containers that no longer match any service defined in the current docker-compose.yml.

    Wrapping Up

    docker compose down is safer than people assume once you understand its default behavior: containers and networks go, volumes and images stay. The danger comes entirely from the extra flags — -v and --rmi all — which turn a routine teardown into a destructive one. Keep those flags out of your default scripts, reserve them for deliberate resets, and always check docker volume ls before you run anything with -v against a stack that matters.

    Get comfortable with stop for quick pauses, down for clean resets, and down -v --remove-orphans --rmi all for full nukes in disposable environments, and you’ll rarely be surprised by what Compose does on teardown.

  • Cloudflare Pages Hosting: A Practical Guide to Deploying Static and Full-Stack Sites

    Cloudflare Pages Hosting: A Practical Guide to Deploying Static and Full-Stack Sites

    Cloudflare Pages hosting has become a common choice for teams that want fast, globally distributed static site delivery without managing origin servers or a separate CDN layer. This guide walks through how Cloudflare Pages hosting works, how to set it up, and what to consider when comparing it against other deployment platforms.

    What Is Cloudflare Pages Hosting?

    Cloudflare Pages hosting is a platform for deploying static sites and JAMstack applications directly onto Cloudflare’s global edge network. Instead of running a traditional web server, your build output — HTML, CSS, JavaScript, and static assets — is pushed to Cloudflare’s infrastructure, and requests are served from the nearest edge location to the visitor.

    The core appeal of Cloudflare Pages hosting is the combination of a git-based deployment workflow with a CDN that was already built for scale. When you connect a repository, Cloudflare watches for commits, runs your build command, and publishes the output automatically. There’s no need to separately configure a CDN in front of the site, because the hosting layer and the edge network are the same system.

    Cloudflare Pages hosting also supports Functions, which let you run server-side logic (written against a Workers-compatible runtime) alongside your static assets. This means a project that started as a purely static site can grow into something with API routes, redirects, and dynamic rendering without switching platforms.

    Static Sites vs. Full-Stack Deployments

    For a purely static site — documentation, a marketing page, a blog generated with a static site generator — Cloudflare Pages hosting requires almost no configuration beyond a build command and an output directory. For full-stack use cases, Pages Functions add server-side capability, but it’s worth understanding the execution model: Functions run in a V8 isolate environment, not a traditional Node.js server process, so not every npm package or Node API will work unchanged. If your app depends heavily on Node-specific APIs, review compatibility before committing to the platform. See the Node.js documentation for reference on what a typical Node runtime provides, and compare that against what Cloudflare’s runtime supports.

    Setting Up Cloudflare Pages Hosting for Your Project

    Getting a project onto Cloudflare Pages hosting generally follows the same pattern regardless of framework:

  • Push your project to a Git provider (GitHub or GitLab).
  • Connect the repository in the Cloudflare dashboard under Workers & Pages.
  • Define a build command and an output directory.
  • Set any required environment variables.
  • Trigger the first deployment.
  • Most popular frameworks — static generators, React-based frameworks, and Vue-based frameworks — are auto-detected, which pre-fills sensible build settings. If you’re migrating an existing project, it’s worth reviewing your CI/CD pipeline fundamentals first, since Cloudflare Pages hosting effectively replaces part of that pipeline with its own build-and-deploy step.

    Connecting a Repository

    When you link a repository, Cloudflare asks for a production branch (commonly main) and, optionally, additional branches for preview deployments. Every push to a non-production branch generates a unique preview URL, which is useful for reviewing changes before they go live. This is one of the more underrated features of Cloudflare Pages hosting — preview environments come free with the git integration and don’t require separate infrastructure.

    Environment Variables and Secrets

    Build-time and runtime environment variables are configured separately in the dashboard, and it’s important not to conflate the two. Build-time variables affect how your static assets are generated; runtime variables are read by Pages Functions when handling a request. Secrets (API keys, tokens) should always be added as encrypted environment variables rather than committed to the repository — a mistake that’s easy to make when moving quickly. If you manage secrets across multiple services, it’s worth reading up on environment variable management practices to keep conventions consistent.

    Build Configuration and Deployment Workflow

    A typical Cloudflare Pages hosting project defines its build in the dashboard, but you can also drive it from the command line using Wrangler, Cloudflare’s CLI tool. This is particularly useful for local testing of Functions or for scripting deployments outside the git-triggered flow.

    # Install Wrangler globally
    npm install -g wrangler
    
    # Authenticate with your Cloudflare account
    wrangler login
    
    # Build your static site (example for a generic build step)
    npm run build
    
    # Deploy the build output to Cloudflare Pages
    wrangler pages deploy ./dist --project-name=my-site

    A minimal configuration file can also define build settings if you prefer infrastructure-as-code over dashboard clicks:

    # wrangler.toml (partial example)
    name = "my-site"
    pages_build_output_dir = "dist"
    
    [env.production]
    vars = { NODE_ENV = "production" }

    Rollbacks and Deployment History

    Every successful deployment to Cloudflare Pages hosting is retained and can be promoted back to production instantly if a new release introduces a regression. This is a meaningful advantage over manually managed servers, where rollback usually means redeploying an older build artifact or reverting a commit and waiting for a fresh build. With Cloudflare Pages hosting, the previous build is already sitting on the edge network, ready to be re-promoted.

    Handling Redirects and Routing Rules

    Static hosts often need a way to define redirects, custom 404 pages, and routing rules without touching application code. Cloudflare Pages hosting supports a _redirects file and a _headers file, both placed in the build output directory, for exactly this purpose. These are read at deploy time and applied at the edge, so redirect logic doesn’t add latency the way an application-level redirect would.

    Custom Domains, SSL, and DNS

    Attaching a custom domain to a Cloudflare Pages hosting project is straightforward if your domain’s DNS is already managed through Cloudflare — you add the domain in the Pages project settings, and the necessary DNS record is created automatically. If your domain lives elsewhere, you’ll need to add a CNAME record pointing to your .pages.dev subdomain.

    SSL certificates for custom domains are issued and renewed automatically, which removes a common source of expired-certificate incidents. If your team currently handles certificate renewal manually elsewhere in your stack, it’s worth comparing that process against how SSL certificate automation works on platforms like this one, since manual renewal is one of the more preventable causes of outages.

    DNS Considerations for Multi-Service Domains

    If a domain already routes some traffic elsewhere — an API on a different host, a subdomain pointing to another CDN — plan the DNS changes carefully before switching the root domain to Cloudflare Pages hosting. Mixing providers for different subdomains is common and generally fine, but it’s easy to introduce a conflicting record accidentally. Reviewing your DNS management best practices before making the cutover reduces the chance of an unplanned outage during migration.

    Performance, Caching, and Edge Functions

    Because Cloudflare Pages hosting serves assets directly from Cloudflare’s edge network, static content benefits from caching close to the visitor by default. This is different from a traditional setup where a CDN is layered in front of an origin server after the fact — with Pages, there is no separate origin to fall out of sync with the cache.

    Cache behavior for static assets is largely automatic, but Pages Functions give you control over caching for dynamic responses using standard HTTP cache headers. If you’re building an API layer on top of Cloudflare Pages hosting, treat cache headers deliberately rather than relying on defaults, since a dynamic response cached too aggressively can serve stale data to users.

    Using Functions for Server-Side Logic

    Pages Functions live in a functions/ directory in your project and map to routes based on file paths, similar to file-based routing conventions in several modern frameworks. A file at functions/api/hello.js becomes available at /api/hello. This makes it possible to add lightweight server-side logic — form handling, authentication checks, API proxying — without standing up a separate backend service. For projects that need more control over the runtime environment than Functions provide, it’s worth reading about containerized deployment patterns as an alternative or complement.

    Comparing Cloudflare Pages Hosting to Other Static Hosts

    Cloudflare Pages hosting is often evaluated alongside other JAMstack-focused platforms. The comparison generally comes down to a few factors:

  • Edge network reach: Cloudflare’s network is one of the more widely distributed CDNs, which benefits geographically diverse audiences.
  • Runtime model: Functions run on a Workers-based runtime rather than a Node.js server, which affects compatibility with some libraries.
  • Pricing structure: Bandwidth and request-based pricing differ across providers, so it’s worth modeling expected traffic before committing.
  • Build minutes and concurrency: Every platform has limits on concurrent builds and total build time, which matters for teams with frequent deploys.
  • Integration with existing DNS: If your domain already sits on Cloudflare DNS, Pages integrates more seamlessly than platforms that require a separate DNS provider.
  • None of these factors makes one platform universally better — the right choice depends on your existing infrastructure, your team’s familiarity with the runtime model, and how much server-side logic your application actually needs. Teams already using Cloudflare for DNS or security features often find Cloudflare Pages hosting a natural extension, while teams deeply invested in a Node.js backend may prefer a platform with a closer match to that runtime. For general context on choosing a deployment strategy, see the JAMstack deployment guide and Cloudflare’s own Pages documentation.

    FAQ

    Does Cloudflare Pages hosting support server-side rendering?
    Yes, through Pages Functions, which run on Cloudflare’s Workers runtime. This supports server-side rendering for compatible frameworks, though you should verify that any server-side dependencies work within that runtime rather than assuming full Node.js compatibility.

    Is Cloudflare Pages hosting free to use?
    Cloudflare offers a free tier for Pages that covers many small to medium projects, with paid tiers available for higher build concurrency, more build minutes, or additional Functions usage. Check current plan details on Cloudflare’s site before committing to a paid tier.

    Can I use Cloudflare Pages hosting alongside an existing backend API?
    Yes. A common pattern is to host the static frontend on Cloudflare Pages hosting while the backend API remains on a separate server or platform. You can either call the external API directly from the frontend or proxy requests through Pages Functions.

    How do preview deployments work on Cloudflare Pages hosting?
    Every branch other than your designated production branch automatically gets its own preview URL on push. This lets you review changes in a live environment before merging, without any additional configuration.

    Conclusion

    Cloudflare Pages hosting combines a git-driven deployment workflow with Cloudflare’s edge network, making it a solid option for static sites and JAMstack applications that want fast global delivery without managing origin infrastructure. Its strengths — automatic previews, instant rollbacks, integrated SSL, and optional server-side Functions — cover a wide range of common use cases, though teams with heavy Node.js-specific backend requirements should evaluate runtime compatibility before migrating. As with any hosting decision, the right fit depends on your existing DNS setup, your team’s deployment habits, and how much dynamic logic your application actually needs at the edge.

  • How to Build AI Agents With n8n: Step-by-Step Guide

    How to Build AI Agents With n8n

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Building AI agents used to mean stitching together Python scripts, cron jobs, and a pile of API keys. n8n changes that. It’s an open-source workflow automation tool that now ships a native AI Agent node, letting you wire large language models, tools, and memory into a visual canvas instead of a code editor. This guide walks through everything from a local Docker install to a production deployment on a VPS you actually control.

    We’ll build a working agent — not a toy demo — that can call external APIs, remember conversation context, and run on a schedule or webhook trigger. If you’re already comfortable with Docker, you can skip straight to the workflow-building section below.

    What Is n8n and Why Use It for AI Agents

    n8n (pronounced “n-eight-n”) is a fair-code workflow automation platform. Unlike Zapier or Make, you can self-host it, inspect every line of its source on GitHub, and extend it with custom JavaScript or Python code nodes when the built-in nodes aren’t enough.

    Since version 1.19, n8n ships an AI Agent node built on top of LangChain’s agent framework, but exposed through n8n’s drag-and-drop interface. That means you get:

  • Native support for OpenAI, Anthropic, Google Gemini, and local models via Ollama
  • Built-in memory nodes (buffer, window, vector store-backed)
  • Tool nodes that let the agent call HTTP endpoints, run code, query databases, or trigger other workflows
  • Full execution logs for every agent decision, which matters a lot when something goes wrong in production
  • n8n vs LangChain vs Custom Code

    If you’ve evaluated raw LangChain or LlamaIndex for agent building, the tradeoff is straightforward: those frameworks give you more flexibility but require you to own the infrastructure, error handling, retry logic, and deployment pipeline yourself. n8n gives you 80% of that flexibility with a visual debugger, built-in credential storage, and one-click deployment via Docker. For internal tooling, support bots, and workflow automation, that tradeoff usually favors n8n. For research-grade custom agent architectures, you’ll still want raw code.

    Prerequisites

    Before starting, make sure you have:

  • Docker and Docker Compose installed (see our Docker Compose beginner’s guide if you’re new to it)
  • An API key from OpenAI, Anthropic, or a self-hosted Ollama instance
  • Basic familiarity with webhooks and REST APIs
  • A VPS if you plan to deploy beyond local testing (we cover that below)
  • Setting Up n8n Locally With Docker

    The fastest way to get n8n running is Docker Compose. Create a project directory and drop in the following file.

    # docker-compose.yml
    version: "3.8"
    
    services:
      n8n:
        image: docker.n8n.io/n8nio/n8n:latest
        restart: unless-stopped
        ports:
          - "5678:5678"
        environment:
          - N8N_BASIC_AUTH_ACTIVE=true
          - N8N_BASIC_AUTH_USER=admin
          - N8N_BASIC_AUTH_PASSWORD=change_this_password
          - N8N_HOST=localhost
          - N8N_PORT=5678
          - N8N_PROTOCOL=http
          - GENERIC_TIMEZONE=America/New_York
          - N8N_ENCRYPTION_KEY=replace_with_a_long_random_string
        volumes:
          - n8n_data:/home/node/.n8n
    
    volumes:
      n8n_data:

    Bring it up with:

    docker compose up -d

    Then open http://localhost:5678 and log in with the credentials you set above. n8n will prompt you to create an owner account on first launch — do that before anything else, since the basic auth env vars only gate the login screen, not the app itself in newer versions.

    Installing n8n via Docker Compose Behind a Reverse Proxy

    If you want HTTPS locally or plan to test webhook triggers from third-party services, put Nginx or Caddy in front of n8n. Our Nginx reverse proxy for Docker guide covers the certificate and config details. For local development, an ngrok tunnel is usually faster:

    ngrok http 5678

    Use the generated HTTPS URL as your webhook endpoint when testing integrations with services that require public callbacks (Slack, Telegram, Twilio, etc.).

    Building Your First AI Agent Workflow

    Once n8n is running, create a new workflow and add these nodes in order: Chat Trigger, then AI Agent, then the tools attached to that agent.

    Configuring the AI Agent Node

    1. Add a Chat Trigger node — this gives you a built-in chat UI for testing without building a frontend.
    2. Add an AI Agent node and connect it to the trigger.
    3. Under “Chat Model,” select OpenAI Chat Model (or Anthropic, or Ollama for local inference) and paste your credential.
    4. Set the system prompt. Be specific — vague prompts produce agents that hallucinate tool usage:

    You are a DevOps assistant. You have access to a server-status tool and a
    ticket-creation tool. Always check server status before creating a ticket.
    Respond concisely and cite which tool you used.

    5. Under “Memory,” attach a Window Buffer Memory node if you want the agent to remember the last N messages in a conversation. For anything long-running or multi-session, use a vector-store-backed memory instead (Postgres with pgvector, Qdrant, or Pinecone all have native n8n nodes).

    Adding Tools to Your Agent

    Tools are what separate an “agent” from a chatbot. In n8n, any node can become a tool by toggling “Use as Tool” or by using the dedicated Tool node wrappers. Common patterns:

  • HTTP Request Tool — expose an internal API (server metrics, ticketing system, inventory database) as a callable function the LLM can invoke
  • Code Tool — run a JavaScript or Python snippet for calculations the model shouldn’t attempt itself
  • Workflow Tool — call another n8n workflow as a sub-agent, useful for splitting complex logic into smaller, testable pieces
  • Vector Store Tool — give the agent retrieval-augmented generation (RAG) access to a document store
  • Here’s an example HTTP Request Tool configuration for checking server uptime, wired as a tool the agent can call autonomously:

    {
      "method": "GET",
      "url": "https://api.yourservice.com/v1/status",
      "authentication": "genericCredentialType",
      "toolDescription": "Returns current uptime and CPU/memory usage for the production server. Call this before creating any incident ticket."
    }

    The toolDescription field matters more than any other setting here — it’s the text the LLM reads to decide whether and when to call the tool. Vague descriptions lead to agents that either never use the tool or use it constantly for no reason.

    Testing and Debugging Agent Decisions

    Every AI Agent execution in n8n logs its full reasoning chain: which tools it considered, what parameters it passed, and the raw model response at each step. Open the execution list after a test run and click into any node to see this. This is the single biggest advantage over building agents in raw code without a visual debugger — you can see exactly why an agent chose (or refused) to call a tool, without adding print statements everywhere.

    Adding Retrieval-Augmented Generation (RAG)

    If your agent needs to answer questions from internal documentation, wire up a vector store pipeline separately from the live agent workflow:

    1. A trigger workflow that watches a folder or endpoint for new documents
    2. A Text Splitter node to chunk long documents
    3. An Embeddings node (OpenAI or a local Ollama embedding model)
    4. A Vector Store node (Qdrant, Pinecone, or Postgres/pgvector) to store the chunks

    Then in your agent workflow, add a Vector Store Tool pointed at the same collection, with a description like “Search internal documentation for policy and configuration questions.” The agent will query it automatically when a user’s question matches that description.

    Deploying n8n to Production

    Running n8n on your laptop is fine for prototyping, but agents that call real APIs and handle real users need a stable host. A small VPS is enough for most single-team deployments — n8n itself is lightweight, and the actual LLM inference happens on OpenAI’s or Anthropic’s servers, not yours.

    For this, DigitalOcean droplets are a solid default — a $12/month droplet handles n8n plus a Postgres database comfortably, and their managed database add-on removes one more thing to babysit. If you want more compute per dollar and don’t mind Europe-based infrastructure, Hetzner is consistently cheaper for the same specs.

    Once deployed, put Cloudflare in front of your n8n instance for free SSL termination and DDoS protection — this matters because webhook-triggered agents are publicly reachable by design. And since an agent that silently stops responding is worse than one that never worked, wire up BetterStack for uptime monitoring on your webhook endpoints and the n8n health check route (/healthz).

    A minimal production-ready docker-compose.yml swaps SQLite for Postgres and adds persistent volumes:

    version: "3.8"
    
    services:
      postgres:
        image: postgres:16
        restart: unless-stopped
        environment:
          - POSTGRES_USER=n8n
          - POSTGRES_PASSWORD=change_this_password
          - POSTGRES_DB=n8n
        volumes:
          - postgres_data:/var/lib/postgresql/data
    
      n8n:
        image: docker.n8n.io/n8nio/n8n:latest
        restart: unless-stopped
        depends_on:
          - postgres
        ports:
          - "5678:5678"
        environment:
          - DB_TYPE=postgresdb
          - DB_POSTGRESDB_HOST=postgres
          - DB_POSTGRESDB_DATABASE=n8n
          - DB_POSTGRESDB_USER=n8n
          - DB_POSTGRESDB_PASSWORD=change_this_password
          - N8N_ENCRYPTION_KEY=replace_with_a_long_random_string
          - WEBHOOK_URL=https://n8n.yourdomain.com/
          - N8N_PROTOCOL=https
        volumes:
          - n8n_data:/home/node/.n8n
    
    volumes:
      postgres_data:
      n8n_data:

    Deploy it the same way — docker compose up -d — then point your reverse proxy or Cloudflare Tunnel at port 5678.

    Common Pitfalls When Building n8n Agents

  • Overloaded system prompts — one agent trying to do ten things will underperform ten narrow agents chained together via Workflow Tool nodes
  • Missing tool descriptions — the model can’t guess what a tool does; write descriptions as if explaining to a new hire
  • No rate limiting on webhook triggers — a public Chat Trigger without authentication is an open invitation for abuse and runaway API bills
  • Storing API keys in plain environment variables instead of n8n’s credential vault — always use the built-in credentials system so keys are encrypted at rest
  • Skipping memory pruning — window buffer memory left unbounded will eventually blow past your model’s context window on long conversations
  • Monitoring and Iterating on Agent Performance

    Shipping the first version of an agent is the easy part. The harder part is catching failure modes before users do — an agent that occasionally calls the wrong tool, loops on a task, or returns a confident but wrong answer won’t always throw an error n8n can catch automatically. Export execution data regularly and review a sample of transcripts by hand, especially in the first few weeks after launch. Pay attention to:

  • How often the agent calls a tool it wasn’t supposed to need
  • Average response latency, which climbs fast once you chain multiple tool calls per turn
  • Token usage per conversation, since a single runaway loop can burn through your OpenAI budget in minutes
  • Cases where the agent apologizes or hedges excessively — often a sign the system prompt is too restrictive
  • Set up alerting on failed executions using n8n’s built-in error workflow feature: any workflow can have a dedicated error handler that fires on failure, letting you route exceptions to Slack, email, or a ticketing system instead of discovering them days later in the logs. Combined with BetterStack‘s uptime checks on your webhook endpoint, this gives you two independent signals — one for “the server is down” and one for “the server is up but the agent is misbehaving” — which is the distinction that actually matters once agents are handling real traffic.

    Wrapping Up

    n8n won’t replace a custom-built agent framework for every use case, but for teams that need production AI agents shipped in days instead of months, it’s one of the fastest paths available. Start local with Docker, prototype your tool descriptions carefully, and move to a proper Postgres-backed deployment once the workflow proves useful. If you’re also managing the underlying infrastructure, our self-hosting guide for VPS deployments covers hardening steps worth applying before you expose any agent publicly.

    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Do I need to know how to code to build AI agents in n8n?
    No. The AI Agent node, tool nodes, and memory nodes are all configured visually. Code nodes are optional and only needed for custom logic that no built-in node covers.

    Can I run n8n AI agents without paying for OpenAI or Anthropic?
    Yes. n8n’s Chat Model node supports Ollama, which runs open models like Llama 3 or Mistral locally. Performance depends on your hardware, but it works well for testing and low-traffic production use.

    How is an n8n AI Agent different from a regular n8n workflow?
    A regular workflow follows a fixed, predetermined sequence of steps. An AI Agent node lets the LLM decide dynamically which tools to call and in what order, based on the conversation and system prompt, rather than following a hardcoded path.

    Is n8n secure enough for production agent deployments?
    It can be, if you follow standard practices: use the credential vault instead of plain env vars, put a reverse proxy with HTTPS in front of it, restrict webhook access where possible, and keep the Docker image updated. n8n itself is open source and auditable on GitHub.

    What’s the difference between window buffer memory and vector store memory?
    Window buffer memory keeps the last N messages in a conversation in raw form — simple and fast, but limited by context window size. Vector store memory embeds and stores conversation history (or documents) for semantic retrieval, letting the agent recall relevant information from much further back or from an external knowledge base.

    Can one n8n agent call another n8n agent?
    Yes, via the Workflow Tool node. This lets you build a “manager” agent that delegates subtasks to specialized “worker” agent workflows, which is often more reliable than one agent trying to handle everything with a single sprawling system prompt.

  • Cloudflare Page Rules: Complete Setup & Optimization Guide

    Cloudflare Page Rules: A Practical Guide for Developers and Sysadmins

    If you manage a domain behind Cloudflare, sooner or later you’ll need finer-grained control than the global dashboard settings allow. That’s where Cloudflare Page Rules come in. They let you override Cloudflare’s default behavior — caching, SSL, redirects, security level — for specific URL patterns, without touching your origin server’s configuration.

    This guide walks through what Page Rules actually do, how the priority system works (a common source of confusion), and practical configurations you can copy directly into your Cloudflare dashboard.

    What Are Cloudflare Page Rules?

    A Page Rule is a URL-pattern match paired with one or more settings that apply only when a request matches that pattern. Free plans get 3 rules, Pro gets 20, Business gets 50, and Enterprise can have hundreds. Each rule can bundle up to a handful of settings — cache level, browser cache TTL, forwarding URL, security level, SSL mode, and more.

    Think of them as if this URL matches, then do this statements layered on top of Cloudflare’s edge network. They’re evaluated before the request reaches your origin, which makes them powerful for performance tuning and access control alike.

    Common Use Cases

  • Force HTTPS on specific subdomains or paths
  • Bypass cache for admin panels or API endpoints (/wp-admin/*, /api/*)
  • Set aggressive caching on static assets (/assets/*, /images/*)
  • Create 301 redirects for old URLs after a migration
  • Disable security features (like Rocket Loader) on pages where they break JavaScript
  • Enable Always Online for specific high-traffic pages
  • If you’re running a Dockerized app behind Cloudflare, Page Rules pair well with the reverse proxy setup covered in our Docker Compose reverse proxy guide — Cloudflare handles edge caching while Nginx or Traefik handles routing internally.

    Setting Up Your First Page Rule

    Log into the Cloudflare dashboard, select your domain, and navigate to Rules > Page Rules. Click Create Page Rule.

    A rule has three parts:

    1. URL pattern — supports wildcards (*)
    2. Settings — one or more actions to apply
    3. Order/priority — determined by the rule’s position in the list

    Here’s a basic example that caches everything under /static/ aggressively:

    URL: example.com/static/*
    Settings:
      Cache Level: Cache Everything
      Edge Cache TTL: a month
      Browser Cache TTL: a month

    And one that bypasses cache entirely for an API path:

    URL: example.com/api/*
    Settings:
      Cache Level: Bypass
      Disable Performance (Rocket Loader, Auto Minify)

    Using the API Instead of the Dashboard

    For infrastructure-as-code workflows, you’ll want to manage Page Rules via the Cloudflare API rather than clicking through the UI every time. Here’s a curl example creating a cache rule:

    curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/pagerules" 
      -H "Authorization: Bearer $CF_API_TOKEN" 
      -H "Content-Type: application/json" 
      --data '{
        "targets": [
          {
            "target": "url",
            "constraint": {
              "operator": "matches",
              "value": "example.com/static/*"
            }
          }
        ],
        "actions": [
          { "id": "cache_level", "value": "cache_everything" },
          { "id": "edge_cache_ttl", "value": 2592000 }
        ],
        "priority": 1,
        "status": "active"
      }'

    You can find your ZONE_ID on the domain overview page, and generate a scoped API token under My Profile > API Tokens with Zone.Page Rules edit permissions. Full parameter reference is available in the Cloudflare API documentation.

    If you’d rather manage rules declaratively, Terraform’s Cloudflare provider supports the cloudflare_page_rule resource, which is worth adopting once you have more than a handful of rules across multiple zones.

    Priority Order: The Part Everyone Gets Wrong

    Page Rules don’t evaluate in the order you’d expect from most rule-based systems. Cloudflare processes rules top to bottom in the order they appear in your dashboard list, and the first matching rule wins — subsequent matching rules are ignored, even if they contain different settings.

    This trips people up constantly. If you have a broad rule like example.com/* sitting above a specific rule like example.com/blog/*, the broad rule will always win and the specific one never fires.

    Correct Ordering Example

    1. example.com/blog/feed/*        -> Cache Level: Bypass
    2. example.com/blog/*             -> Cache Level: Cache Everything
    3. example.com/*                  -> Security Level: Medium

    Most specific rules go first, broadest rules go last. You can drag rules to reorder them in the dashboard, or set the priority field explicitly via the API.

    Debugging Priority Conflicts

  • List all rules and their order before troubleshooting
  • Use the Cache Status header (cf-cache-status) in response headers to verify which behavior is actually being applied
  • Test with curl -I against the live URL, not a browser (browser cache can mask the real edge behavior)
  • curl -I https://example.com/blog/some-post/

    Look for cf-cache-status: HIT, MISS, BYPASS, or DYNAMIC to confirm the rule that actually applied.

    Page Rules vs. Cloudflare Rulesets (Transform/Cache Rules)

    Cloudflare has been migrating functionality out of the legacy Page Rules system into newer, more granular products: Cache Rules, Transform Rules, and Redirect Rules, all part of the Rulesets engine. These are available on all plans (including Free, with limits) and are generally the better choice going forward because:

  • They support more complex logic (AND/OR conditions, not just URL wildcards)
  • They separate concerns (cache behavior vs. redirects vs. header rewrites) instead of bundling everything into one rule type
  • They don’t count against your legacy Page Rule limit
  • If you’re starting fresh in 2026, lean toward Cache Rules and Redirect Rules for new configurations, and reserve legacy Page Rules for settings that haven’t been migrated yet (like Forwarding URL redirects on older plans, or Always Online). Our DNS records explained article covers how these interact with your zone’s DNS setup if you’re still getting your domain fully onboarded.

    Redirect Example (Legacy Page Rule)

    URL: example.com/old-page
    Setting: Forwarding URL (301 - Permanent Redirect)
    Destination: https://example.com/new-page

    For bulk redirects (hundreds of URLs after a migration), use Bulk Redirects instead — Page Rules cap out fast and aren’t meant for large redirect maps.

    Security-Focused Page Rule Configurations

    Beyond caching, Page Rules are useful for tightening security on sensitive paths:

  • Set Security Level: High on login and admin URLs to trigger more aggressive challenge behavior
  • Enable Browser Integrity Check on form submission endpoints
  • Force Always Use HTTPS across the entire zone if you haven’t already enabled it globally
  • Disable Email Obfuscation on paths where it conflicts with your app’s JS (rare, but it happens)
  • If you’re running self-hosted services and want an alternative to exposing an origin IP entirely, look at Cloudflare Tunnel — we cover the tradeoffs in Cloudflare Tunnel vs. VPN for homelab and small production setups.

    Combining Page Rules With Origin Server Config

    Page Rules operate at the edge, but they work best when your origin server headers agree with them. For example, if you set Cache Level: Cache Everything at the edge but your origin sends Cache-Control: no-store, Cloudflare will generally respect the more restrictive origin header unless you also set an explicit Edge Cache TTL in the rule to override it. Always test both layers together rather than assuming the edge rule is the final word.

    Monitoring the Impact of Your Rules

    After deploying Page Rules, verify they’re actually improving performance rather than just trusting the dashboard. Check:

  • Analytics > Traffic in Cloudflare for cache hit ratio changes
  • Origin server request logs to confirm reduced load (fewer requests reaching your backend is the real signal a caching rule worked)
  • Uptime and latency from an external monitor — a tool like BetterStack gives you synthetic checks from multiple regions so you can see whether edge caching actually improved response times for real users, not just from your own location
  • If your infrastructure lives on a VPS, pairing Cloudflare’s edge layer with a solid host matters just as much as the rules themselves — see our comparison in best VPS providers for self-hosting if you’re still choosing where to run the origin.

    FAQ

    Q: How many free Page Rules does Cloudflare give me?
    A: The Free plan includes 3 Page Rules per zone. Pro gets 20, Business gets 50, and Enterprise plans can negotiate higher limits or use the Rulesets engine instead, which has separate, more generous limits.

    Q: Why isn’t my Page Rule working even though the URL matches?
    A: The most common cause is rule ordering — a broader rule listed above your specific rule will match first and suppress it. Reorder so the most specific URL patterns are at the top of the list.

    Q: Do Page Rules affect DNS-only (grey-clouded) records?
    A: No. Page Rules only apply to traffic proxied through Cloudflare (orange-clouded). If a DNS record is set to “DNS only,” none of your Page Rules, Cache Rules, or security settings will apply to it.

    Q: Should I use Page Rules or Cache Rules for new caching configurations?
    A: Use Cache Rules going forward. They’re more flexible, available on all plans, and don’t consume your limited legacy Page Rule count. Reserve legacy Page Rules for settings not yet available in the newer Rulesets products.

    Q: Can Page Rules replace a CDN cache-busting strategy for a Docker-deployed app?
    A: Partially. Page Rules control edge caching behavior, but you’ll still want cache-busting via versioned filenames or query strings in your build pipeline. Combine both for the most reliable results.

    Q: Will changing a Page Rule take effect immediately?
    A: Changes propagate across Cloudflare’s edge network within seconds to a few minutes globally. If you don’t see the change reflected, check for browser cache or an overriding rule higher in priority.

    Wrapping Up

    Cloudflare Page Rules are a small but sharp tool: three or four well-ordered rules can meaningfully cut origin load, tighten security on sensitive paths, and clean up legacy URLs after a migration. The biggest pitfall is priority ordering — get specific rules above broad ones, verify behavior with curl -I and the cf-cache-status header, and migrate new configurations to Cache Rules and Redirect Rules where possible since that’s where Cloudflare’s development effort is focused going forward.

    If you manage Cloudflare across multiple domains or want deeper analytics and rule automation, a Cloudflare Pro or Business plan is worth evaluating — the added Page Rule slots and Cache Rule flexibility pay for themselves quickly once you’re managing more than a couple of sites.

  • Unmanaged VPS Hosting: A Practical Guide for Devs

    Unmanaged VPS Hosting: What It Is and How to Run One Without Getting Burned

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever spun up a cloud instance and realized you were entirely on your own for OS patches, firewall rules, and backups, you’ve already experienced unmanaged VPS hosting. It’s the default mode for most cloud providers — you get root access and a blank slate, and everything after that is your responsibility.

    This guide covers what unmanaged VPS hosting actually means, when it makes sense over a managed plan, which providers are worth your money, and how to configure a new server so it isn’t compromised in the first 48 hours.

    What Is Unmanaged VPS Hosting?

    An unmanaged VPS (Virtual Private Server) gives you a virtual machine with root or administrator access and nothing else. The provider handles the physical hardware, the hypervisor, and network uptime. Everything above that layer — OS updates, security patching, firewall configuration, monitoring, backups, and application stack — is entirely your job.

    Compare that to a managed VPS, where the hosting company typically handles OS patching, security hardening, control panel installation (like cPanel or Plesk), and often 24/7 support for server-level issues. Managed plans cost significantly more, sometimes 3-5x, because you’re paying for someone else’s sysadmin time.

    For developers and DevOps engineers who already know their way around a Linux shell, unmanaged VPS hosting is usually the better deal. You’re not paying for hand-holding you don’t need, and you get full control to install exactly the stack you want — Docker, Kubernetes, a custom kernel module, whatever.

    Unmanaged vs Managed: The Real Tradeoffs

    The decision usually comes down to three factors: cost, control, and time.

  • Cost: Unmanaged VPS plans start as low as $4-6/month for a basic instance; managed equivalents with similar specs often start at $20-40/month.
  • Control: With unmanaged hosting you choose your own OS image, kernel, container runtime, and security tooling — no restrictions from a control panel.
  • Time: You own the operational burden. If you don’t have time to patch a CVE the same day it drops, that’s a real risk you’re accepting.
  • If your team already runs Docker in production and has existing playbooks for patching and monitoring, unmanaged hosting is a straightforward win. If you’re a solo founder without ops experience, a managed plan or a PaaS like Render or Railway might save you more time than it costs.

    Who Should Actually Use Unmanaged VPS Hosting

    Unmanaged hosting makes the most sense for:

  • Developers comfortable with SSH, systemd, and basic Linux administration
  • Teams running containerized workloads who want infrastructure-as-code control
  • Anyone hosting side projects, staging environments, or personal streaming/media servers where uptime SLAs aren’t business-critical
  • Cost-sensitive startups that can absorb some ops overhead in exchange for lower monthly spend
  • It’s a poor fit if you have no one on the team who can respond to a security incident, or if your compliance requirements mandate a managed, audited environment.

    Best Unmanaged VPS Providers in 2026

    Most major cloud providers sell unmanaged VPS instances by default — you just have to know what you’re buying.

    DigitalOcean is the easiest on-ramp for developers new to VPS management. Droplets are cheap, the dashboard is simple, and their documentation library is genuinely excellent for learning Linux fundamentals.

    Hetzner is the price-to-performance leader, especially for EU-based workloads — their Cloud instances often deliver double the CPU/RAM of comparable US providers at a lower price point.

    Both are solid, but if you want a quick reference for evaluating either one:

  • DigitalOcean: best documentation, simplest UI, good for beginners
  • Hetzner: best raw performance per dollar, slightly steeper learning curve
  • Both: fully unmanaged by default, root access from the first boot
  • 👉 Spin up a DigitalOcean Droplet if you want the gentlest learning curve for your first unmanaged server.

    👉 Check Hetzner Cloud pricing if raw compute-per-dollar matters more than a polished dashboard.

    Setting Up Your First Unmanaged VPS Securely

    The biggest risk with unmanaged hosting isn’t the provider — it’s the first hour after your server boots, before you’ve locked anything down. Automated bots scan public IP ranges constantly, so treat every fresh VPS as hostile territory until it’s hardened.

    Step 1: Create a Non-Root User and Disable Password Login

    Never run your daily workload as root. Create a sudo user immediately and switch to key-based SSH authentication.

    # On the fresh VPS, logged in as root
    adduser deploy
    usermod -aG sudo deploy
    
    # Copy your local public key to the new user
    rsync --archive --chown=deploy:deploy ~/.ssh /home/deploy

    Then edit /etc/ssh/sshd_config to disable root login and password auth:

    PermitRootLogin no
    PasswordAuthentication no
    PubkeyAuthentication yes

    Restart SSH to apply:

    sudo systemctl restart sshd

    Step 2: Configure a Firewall

    On Ubuntu or Debian, ufw is the fastest way to lock down ports. Only allow what you actually need.

    sudo ufw default deny incoming
    sudo ufw default allow outgoing
    sudo ufw allow OpenSSH
    sudo ufw allow 80/tcp
    sudo ufw allow 443/tcp
    sudo ufw enable

    If you’re running Docker, be aware that Docker manipulates iptables directly and can bypass ufw rules for published container ports. Check the official Docker documentation before exposing containers publicly, and consider binding container ports to 127.0.0.1 and proxying through Nginx instead.

    Step 3: Install fail2ban to Block Brute-Force Attempts

    sudo apt update && sudo apt install fail2ban -y
    sudo systemctl enable --now fail2ban

    The default SSH jail is usually enough for a small server, but you can tune ban time and retry thresholds in /etc/fail2ban/jail.local.

    Step 4: Set Up Unattended Security Updates

    Since nobody’s patching this server for you, automate at least the security updates:

    sudo apt install unattended-upgrades -y
    sudo dpkg-reconfigure --priority=low unattended-upgrades

    This won’t cover major version upgrades, but it closes the gap on critical CVEs between your manual maintenance windows.

    Step 5: Install Docker (If That’s Your Stack)

    Most unmanaged VPS workloads these days are container-based. The official convenience script works fine for a quick setup:

    curl -fsSL https://get.docker.com -o get-docker.sh
    sudo sh get-docker.sh
    sudo usermod -aG docker deploy

    Log out and back in for the group change to take effect, then verify:

    docker run hello-world

    From here, most teams layer on a reverse proxy (Nginx, Caddy, or Traefik) with automatic TLS via Let’s Encrypt, and a docker-compose.yml to manage their application stack. If you’re new to container networking, our guide to Docker networking basics walks through bridge networks, port publishing, and common pitfalls.

    Monitoring an Unmanaged Server

    Without a managed provider watching your back, you need your own alerting. At minimum, set up:

  • Uptime monitoring on your public-facing services
  • Disk space alerts (a full disk silently kills more VPS instances than attacks do)
  • CPU/memory monitoring to catch runaway processes early
  • Log aggregation or at least log rotation so /var/log doesn’t fill your disk
  • A lightweight option is htop plus ncdu for manual checks, but for anything running in production you want real alerting. BetterStack offers uptime and log monitoring with a generous free tier and clean incident alerting via Slack, email, or SMS — worth setting up before you actually need it, not after an outage.

    👉 Try BetterStack monitoring to get paged before your users notice downtime, not after.

    Common Pitfalls With Unmanaged VPS Hosting

    A few mistakes show up repeatedly with teams new to unmanaged hosting:

  • No backup strategy. Snapshots from your provider aren’t a substitute for application-level backups (database dumps, volume backups) stored somewhere else.
  • Skipping the firewall because “it’s just a test server.” Test servers get compromised just as often as production ones — sometimes faster, since they’re less watched.
  • Running everything as root. It’s tempting for convenience, but a single compromised process running as root can take over the entire box.
  • Ignoring kernel and Docker version drift. An outdated Docker Engine can carry known container-escape vulnerabilities. Keep it current.
  • No monitoring at all. Most outages on unmanaged servers go unnoticed for hours because nothing was watching.
  • If you’re also using Cloudflare in front of your VPS for DDoS protection and DNS, make sure your origin IP isn’t exposed elsewhere — a surprising number of “protected” servers are still directly reachable because of a stray DNS record or a leaked IP in an old deployment script.

    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Is unmanaged VPS hosting good for beginners?
    It can be, if you’re willing to learn basic Linux administration. Start with a provider that has strong documentation, like DigitalOcean, and follow a hardening checklist before deploying anything real.

    How much does unmanaged VPS hosting typically cost?
    Basic instances start around $4-6/month for 1 vCPU and 1-2GB RAM. Costs scale with CPU, RAM, storage, and bandwidth, but stay well below managed hosting for equivalent specs.

    Can I switch from unmanaged to managed hosting later?
    Usually yes, either by migrating to a managed plan with the same provider or moving your workload to a new host entirely. Containerizing your applications early makes this migration much less painful.

    Do I need a firewall if my provider already has one?
    Yes. Provider-level firewalls (like cloud security groups) are a good first layer, but you should still configure host-level rules with ufw or iptables as defense in depth.

    What’s the biggest security risk with unmanaged VPS hosting?
    Unpatched software and weak SSH configuration. Automated bots scan for exposed SSH ports and outdated services constantly, so hardening your server in the first hour matters more than almost anything else you’ll do.

    Is Docker required for unmanaged VPS hosting?
    No, but it’s strongly recommended. Containers isolate your applications, simplify dependency management, and make it far easier to reproduce your setup if you ever need to migrate providers.

    Final Thoughts

    Unmanaged VPS hosting rewards developers who already know Linux fundamentals with lower costs and full control. The tradeoff is real: you own every patch, every firewall rule, and every 3 a.m. incident. If that sounds manageable — and for most technical teams it is — the savings and flexibility are hard to beat. Start with a hardened base image, automate what you can, and monitor everything you can’t.

  • n8n YouTube Automation: Self-Hosted Workflow Guide

    N8N YouTube Automation: Building Self-Hosted Video Workflows with Docker

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’re managing a YouTube channel alongside a streaming or cord-cutting content business, you already know the manual overhead: uploading videos, writing metadata, cross-posting to socials, and tracking analytics. n8n YouTube automation lets you replace that manual grind with self-hosted, version-controlled workflows that you fully own — no per-task pricing, no vendor lock-in.

    This guide walks through deploying n8n via Docker, connecting it to the YouTube Data API, and building real workflows: auto-publishing metadata, sending upload alerts to Slack/Discord, and archiving video stats for reporting.

    Why n8n for YouTube Automation

    n8n is an open-source, node-based workflow automation tool — think Zapier or Make, but self-hostable and free of per-execution billing once you run your own instance. For teams already managing Docker infrastructure, n8n slots in as just another container in your stack.

    Compared to closed SaaS automation platforms, self-hosted n8n gives you:

  • Full control over data — video metadata, API keys, and analytics never leave your infrastructure
  • No execution caps or surprise billing tiers
  • Native support for custom JavaScript/Python code nodes for edge cases
  • Direct integration with the YouTube Data API v3 via HTTP Request nodes or the built-in YouTube node
  • Easy chaining into other tools — Slack, Discord, Notion, Google Sheets, S3-compatible storage
  • The tradeoff is that you’re responsible for hosting, updates, and security — which is exactly the kind of operational work this site normally covers for streaming infrastructure, so it’s a natural fit.

    Prerequisites

    Before you start, you’ll need:

  • A VPS or dedicated server with Docker and Docker Compose installed
  • A domain or subdomain (e.g., n8n.yourdomain.com) with DNS pointed at your server
  • A Google Cloud project with the YouTube Data API v3 enabled and OAuth2 credentials generated
  • Basic familiarity with docker-compose and reverse proxies
  • If you haven’t set up a production-ready VPS yet, our guide to hardening a Linux VPS covers firewall rules and SSH lockdown before you expose any web service.

    Deploying n8n with Docker Compose

    The fastest reliable path to a persistent, production-usable n8n instance is Docker Compose with a Postgres backend (SQLite works for testing but isn’t recommended once you have real workflows running).

    version: "3.8"
    
    services:
      postgres:
        image: postgres:16-alpine
        restart: unless-stopped
        environment:
          POSTGRES_USER: n8n
          POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
          POSTGRES_DB: n8n
        volumes:
          - postgres_data:/var/lib/postgresql/data
    
      n8n:
        image: docker.n8n.io/n8nio/n8n:latest
        restart: unless-stopped
        ports:
          - "5678:5678"
        environment:
          - N8N_HOST=n8n.yourdomain.com
          - N8N_PROTOCOL=https
          - N8N_PORT=5678
          - DB_TYPE=postgresdb
          - DB_POSTGRESDB_HOST=postgres
          - DB_POSTGRESDB_USER=n8n
          - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
          - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
          - GENERIC_TIMEZONE=America/New_York
        volumes:
          - n8n_data:/home/node/.n8n
        depends_on:
          - postgres
    
    volumes:
      postgres_data:
      n8n_data:

    Save your secrets in a .env file rather than hardcoding them:

    echo "POSTGRES_PASSWORD=$(openssl rand -hex 16)" >> .env
    echo "N8N_ENCRYPTION_KEY=$(openssl rand -hex 24)" >> .env

    Bring the stack up:

    docker compose up -d
    docker compose logs -f n8n

    Put Nginx or Caddy in front of it for TLS termination. A minimal Caddy config:

    n8n.yourdomain.com {
        reverse_proxy localhost:5678
    }

    Once it’s live, visit https://n8n.yourdomain.com, create your owner account, and you’re ready to build workflows.

    Connecting n8n to the YouTube Data API

    YouTube automation in n8n runs through Google’s OAuth2 credential flow:

    1. In Google Cloud Console, enable the YouTube Data API v3 for your project.
    2. Create an OAuth 2.0 Client ID (Web application type), and add https://n8n.yourdomain.com/rest/oauth2-credential/callback as an authorized redirect URI.
    3. In n8n, go to Credentials → New → YouTube OAuth2 API, paste in your Client ID and Secret, and complete the consent screen flow.
    4. Test the connection with a simple YouTube node action, like listing your channel’s recent uploads.

    Once authenticated, n8n can read and write almost anything the API exposes: video metadata, playlist items, comments, captions, and channel analytics (via the separate YouTube Analytics API, which you can call through an HTTP Request node using the same OAuth2 credential).

    Building a Practical Workflow: Auto-Publish and Alert

    A common first workflow: when a new video finishes uploading (or a scheduled trigger fires), update its metadata from a template and post a notification to Discord.

    Trigger: Schedule node (e.g., every 15 minutes) or a Webhook node if you’re pushing events from an external upload script.

    Step 1 — Fetch the video:

    {
      "node": "YouTube",
      "operation": "get",
      "resource": "video",
      "videoId": "={{ $json.videoId }}"
    }

    Step 2 — Update metadata via HTTP Request node (useful when you need fields the built-in node doesn’t expose):

    curl -X PUT 
      "https://www.googleapis.com/youtube/v3/videos?part=snippet" 
      -H "Authorization: Bearer $ACCESS_TOKEN" 
      -H "Content-Type: application/json" 
      -d '{
        "id": "'"$VIDEO_ID"'",
        "snippet": {
          "title": "'"$TITLE"'",
          "description": "'"$DESCRIPTION"'",
          "categoryId": "28"
        }
      }'

    In n8n, this becomes an HTTP Request node with the OAuth2 credential attached and the body built from expressions referencing your Google Sheet or database row (title, description, tags pulled dynamically per video).

    Step 3 — Notify your team:

    {
      "node": "Discord",
      "webhookUrl": "={{ $env.DISCORD_WEBHOOK }}",
      "content": "✅ Published: {{ $json.snippet.title }} — https://youtu.be/{{ $json.id }}"
    }

    Chain a Google Sheets or Postgres node afterward to log the publish event for reporting.

    Advanced Automation Ideas

    Once the basic pipeline works, extend it:

  • Pull daily view/watch-time stats via the YouTube Analytics API and store them in Postgres for a self-hosted dashboard
  • Auto-generate SEO-friendly descriptions using an LLM node, then require manual approval via a Slack interactive message before publishing
  • Trigger a workflow on new comments containing specific keywords (e.g., support requests) and route them to a ticketing system
  • Cross-post new uploads automatically to Twitter/X and Mastodon using their respective HTTP APIs
  • Archive final video renders to S3-compatible object storage as a backup step before deleting local files
  • Each of these is a small, composable workflow — the strength of n8n is chaining these together rather than building one monolithic automation.

    Securing and Scaling Your n8n Instance

    Because n8n holds OAuth tokens and API keys for your YouTube channel, treat it like any other production secret store:

  • Enable n8n’s built-in basic auth or SSO if you’re not already behind a VPN
  • Keep the container updated — subscribe to n8n’s release notes and patch regularly
  • Back up the Postgres volume on a schedule; workflow definitions and credentials live there
  • Use BetterStack or a similar uptime/log monitoring service to alert you if the container crashes or the API quota gets exhausted
  • If you’re running this on a budget VPS, DigitalOcean droplets and Hetzner cloud instances both handle an n8n + Postgres stack comfortably on their smallest tiers
  • For teams pushing this into real production use — multiple workflows, webhook triggers from external services, higher execution volume — consider a queue-mode n8n deployment with Redis, which decouples the webhook listener from the worker processes and scales horizontally.

      redis:
        image: redis:7-alpine
        restart: unless-stopped

    Add EXECUTIONS_MODE=queue and QUEUE_BULL_REDIS_HOST=redis to your n8n environment variables to enable this mode once you outgrow a single-instance setup.

    A Note on YouTube API Quotas

    The YouTube Data API enforces a default daily quota of 10,000 units, and different operations cost different amounts — a videos.update call costs 50 units, while a simple videos.list costs 1. Design your workflows to batch reads where possible and avoid polling on tight schedules; a 15-minute schedule trigger checking a single playlist is cheap, but looping through hundreds of videos on every run will burn through your quota fast. You can request a quota increase from Google Cloud Console if your automation genuinely needs it, but most single-channel workflows never come close to the default limit.

    If you’re scaling this system alongside other DevOps tooling, it’s worth reading our guide to monitoring self-hosted services to keep tabs on container health, API error rates, and disk usage on the same VPS.

    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Is n8n free to use for YouTube automation?
    Yes — n8n is fair-code licensed and free to self-host with no execution limits. You only pay for the server it runs on. n8n also offers a paid cloud version if you’d rather not manage infrastructure yourself.

    Do I need coding experience to use n8n for YouTube workflows?
    No. Most workflows are built visually by connecting nodes. Code nodes (JavaScript/Python) are optional and only needed for custom logic beyond what built-in nodes support.

    Can n8n actually upload videos to YouTube, or just manage metadata?
    n8n can perform full uploads via the YouTube Data API’s videos.insert endpoint using an HTTP Request node with multipart upload, though most users automate metadata, scheduling, and post-publish tasks rather than the upload itself, since large file uploads are often handled by dedicated encoding pipelines.

    What happens if my YouTube API quota runs out?
    Requests will fail with a 403 quota-exceeded error until the daily quota resets (midnight Pacific Time). Design workflows to fail gracefully and alert you rather than retrying aggressively, which can worsen the problem.

    Is self-hosted n8n secure enough for storing YouTube OAuth credentials?
    Yes, provided you follow basic hardening: keep it behind TLS, enable authentication, restrict network access, and keep the container patched. n8n encrypts stored credentials at rest using the encryption key you configure.

    Can I run n8n alongside my existing Docker services on the same VPS?
    Yes. n8n is lightweight and runs fine alongside other containers as long as you allocate enough RAM for Postgres and give it its own subdomain and reverse proxy block.

    Wrapping Up

    Self-hosted n8n turns YouTube channel management from a checklist of manual steps into a set of reliable, auditable workflows you control end to end. Start with one simple automation — metadata updates or publish alerts — before layering in analytics archiving and cross-posting. The Docker Compose setup above gets you a production-capable instance in under fifteen minutes, and from there it’s just a matter of wiring nodes together.

  • Automated SEO: A DevOps Pipeline for Site Monitoring

    Automated SEO: Building a DevOps Pipeline for Technical SEO Monitoring

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Most articles about automated SEO are written for marketers clicking around in a dashboard. This one isn’t. If you’re a developer or sysadmin who got handed “make the site rank better” as a side quest, you don’t need another SaaS subscription — you need a pipeline. This guide shows you how to build automated SEO checks into your existing DevOps workflow using Docker, cron, shell scripts, and a couple of free APIs.

    The goal is simple: catch broken canonical tags, missing meta descriptions, 404s, slow TTFB, and crawl errors before Google does, using the same containerized, version-controlled approach you already use for infrastructure.

    What Automated SEO Actually Means for Engineers

    Forget keyword stuffing. For a technical audience, automated SEO is really three things:

  • Automated auditing — scripts that crawl your site and flag technical issues (broken links, duplicate titles, missing alt text, bad status codes)
  • Automated monitoring — uptime, response time, and Core Web Vitals tracked continuously, not checked manually once a quarter
  • Automated reporting — results piped into Slack, email, or a dashboard so issues surface without anyone opening a spreadsheet
  • All three fit naturally into a container-based workflow. If you already run a Docker monitoring stack for your infrastructure, adding SEO checks is a small incremental step, not a new discipline.

    Why Manual SEO Audits Don’t Scale

    A marketer running Screaming Frog once a month on a 50-page site is fine. A DevOps team running a 5,000-page content platform across staging and production environments is not fine doing that manually. Issues compound: a bad canonical tag pushed in a deploy can silently deindex hundreds of pages before anyone notices in Google Search Console.

    The fix is treating SEO health like any other observability metric — something you scrape, store, and alert on. Google’s own Search Central documentation is explicit that crawl efficiency and technical health directly affect indexing, which is exactly the kind of thing a cron job should be watching, not a human.

    Building Block One: A Containerized Crawler

    The simplest automated SEO setup is a scheduled crawl using an open-source tool inside Docker. Here’s a minimal setup using xml-sitemap parsing plus curl to check status codes and response times for every URL in your sitemap:

    #!/usr/bin/env bash
    # seo-audit.sh - basic automated SEO health check
    set -euo pipefail
    
    SITEMAP_URL="https://thinkstreamtv.com/sitemap.xml"
    REPORT_FILE="/reports/seo-audit-$(date +%F).log"
    
    mkdir -p /reports
    echo "Starting SEO audit: $(date)" > "$REPORT_FILE"
    
    curl -s "$SITEMAP_URL" 
      | grep -oP '(?<=<loc>)[^<]+' 
      | while read -r url; do
          status=$(curl -o /dev/null -s -w '%{http_code}' "$url")
          ttfb=$(curl -o /dev/null -s -w '%{time_starttransfer}' "$url")
          if [ "$status" -ge 400 ]; then
            echo "BROKEN [$status]: $url" >> "$REPORT_FILE"
          fi
          if (( $(echo "$ttfb > 1.0" | bc -l) )); then
            echo "SLOW [${ttfb}s]: $url" >> "$REPORT_FILE"
          fi
        done
    
    echo "Audit complete: $(date)" >> "$REPORT_FILE"

    Wrap that in a Dockerfile so it runs identically in any environment:

    FROM alpine:3.19
    RUN apk add --no-cache curl bash grep bc pcre-tools
    COPY seo-audit.sh /usr/local/bin/seo-audit.sh
    RUN chmod +x /usr/local/bin/seo-audit.sh
    ENTRYPOINT ["/usr/local/bin/seo-audit.sh"]

    Build and run it:

    docker build -t seo-audit:latest .
    docker run --rm -v "$(pwd)/reports:/reports" seo-audit:latest

    This container becomes a reusable artifact — the same one referenced in our technical SEO audit checklist — that you can run locally, in CI, or on a schedule.

    Building Block Two: Scheduling and Alerting

    A script that runs once is a manual task with extra steps. Automated SEO means it runs on a schedule and tells someone when something breaks. Add it to cron on your VPS:

    # /etc/cron.d/seo-audit
    0 6 * * * root docker run --rm -v /opt/seo-reports:/reports seo-audit:latest

    Then pipe failures into Slack using a webhook so nobody has to SSH in and read a log file:

    #!/usr/bin/env bash
    # notify-seo-issues.sh
    REPORT="/opt/seo-reports/seo-audit-$(date +%F).log"
    WEBHOOK_URL="https://hooks.slack.com/services/XXX/YYY/ZZZ"
    
    if grep -qE 'BROKEN|SLOW' "$REPORT"; then
      ISSUES=$(grep -cE 'BROKEN|SLOW' "$REPORT")
      curl -s -X POST -H 'Content-type: application/json' 
        --data "{"text": "⚠️ $ISSUES SEO issues found today. Check $REPORT"}" 
        "$WEBHOOK_URL"
    fi

    This is exactly the kind of check that pairs well with a dedicated uptime and monitoring service. If you don’t want to babysit your own alerting infrastructure, BetterStack can absorb the uptime and incident-alerting side while your custom script handles the SEO-specific logic — giving you a single pane of glass instead of a pile of cron jobs nobody remembers exists.

    Building Block Three: Structured Data and Metadata Validation

    Broken JSON-LD structured data is one of the most common silent SEO killers, especially after a CMS migration or template change. Validate it automatically using Google’s structured data guidelines referenced in schema.org as your source of truth, and script a check like this:

    curl -s https://thinkstreamtv.com/some-article/ 
      | grep -oP '(?<=<script type="application/ld+json">).*?(?=</script>)' 
      | python3 -m json.tool > /dev/null 
      && echo "Valid JSON-LD" || echo "INVALID JSON-LD - fix immediately"

    Run this against every published URL nightly, and you’ll catch schema regressions the same day they ship instead of two weeks later when rich snippets quietly disappear from search results.

    Building Block Four: Hosting That Doesn’t Fight You

    Automated SEO checks are only useful if the underlying infrastructure is stable enough that “slow” or “down” readings mean something real, not noise from an overloaded shared host. If you’re running these audit containers alongside your production site, a VPS with predictable, dedicated resources matters. Providers like DigitalOcean and Hetzner are common choices for teams running this kind of always-on Docker workload because pricing is predictable and provisioning a new droplet or server for testing audits is trivial to script via API.

    If your team eventually outgrows shell scripts and wants keyword rank tracking, backlink monitoring, and competitor analysis bundled into the same automated pipeline, a platform like SE Ranking can plug into the reporting side via API, letting you keep the crawling and infrastructure checks custom while outsourcing the keyword-tracking data layer.

    Putting It All Together

    A realistic automated SEO stack for a small-to-medium content site looks like this:

  • A Docker container that crawls the sitemap nightly and logs status codes, response times, and broken links
  • A cron job (or systemd timer) that triggers the crawl on a schedule
  • A notification script that posts failures to Slack or email
  • A structured data validator that runs against new and changed pages
  • An external uptime/monitoring service for the parts you don’t want to hand-roll
  • Optional integration with a paid rank-tracking API once you need keyword-level insight
  • None of this replaces good content strategy. Automated SEO catches technical regressions — it doesn’t write your articles or pick your keywords. But it does mean the technical foundation stays solid while your team focuses on content, which is the whole point of running this like infrastructure instead of a manual chore.

    If you’re setting this up for the first time, start small: get the crawler container running locally, confirm it flags a deliberately broken link, then wire up cron and Slack. Once that loop works, everything else — structured data checks, uptime monitoring, rank tracking — is additive.

    Recommended: Ready to put this into practice? SE Ranking is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: Is automated SEO a replacement for tools like Screaming Frog or Ahrefs?
    A: Not entirely. Commercial tools give you polished dashboards and deep keyword data. Automated scripts are best for continuous, low-noise technical checks — status codes, structured data, response times — that you don’t want to run manually every day.

    Q: How often should an automated SEO crawl run?
    A: Daily for actively updated sites, weekly for mostly static ones. Run it more frequently right after deploys, since template changes are the most common source of sitewide SEO regressions.

    Q: Can I run this audit pipeline in CI instead of on a cron schedule?
    A: Yes — many teams add a lightweight version of the crawl as a CI step post-deploy, so broken canonical tags or 404s block a release before they hit production.

    Q: What’s the biggest technical SEO mistake automated checks catch that manual audits miss?
    A: Silent regressions from CMS or template updates — a single bad conditional in a template can strip meta descriptions or canonical tags from thousands of pages at once, and nobody notices until traffic drops weeks later.

    Q: Do I need Kubernetes for this, or is a single VPS enough?
    A: A single VPS is enough for the vast majority of sites. This is a lightweight, scheduled job, not a service that needs to scale horizontally.

    Q: How do I validate structured data without hitting rate limits on Google’s tools?
    A: Validate JSON-LD syntax locally with a JSON parser first (as shown above), and only spot-check a sample of pages against Google’s Rich Results Test rather than every page on every run.

    Automated SEO, done this way, is just another observability pipeline — one more thing your infrastructure watches so your team doesn’t have to remember to check it manually. Treat it like uptime monitoring or log aggregation: version it, containerize it, schedule it, and let the alerts do the work.

  • How to Build Agentic AI: A Developer’s Guide

    How to Build Agentic AI: A Practical Developer’s Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Agentic AI isn’t a single library or API call — it’s an architecture pattern where a language model plans, calls tools, evaluates the results, and iterates until it solves a task on its own, without a human clicking “next” at every step. If you’ve been shipping simple chatbot wrappers around an LLM and want to move up to something that actually takes actions in the real world, this guide covers how to build agentic ai systems from first principles: the reasoning loop, tool calling, memory, and the infrastructure decisions that separate a weekend demo from something you can run in production.

    We’ll build a minimal agent in Python, add tool calling, containerize it with Docker, and cover the hosting, monitoring, and security choices that matter once you’re running this for real users instead of just yourself in a notebook.

    What Is Agentic AI, Really?

    Most people use “agentic AI” loosely to describe anything that calls an LLM in a loop. A more precise definition: an agentic system is one where the model itself decides what to do next based on the current state of the world, rather than following a hardcoded sequence of steps written by a developer. The model observes, reasons about the observation, picks an action (usually a tool call), executes it, observes the result, and repeats until it decides the task is done.

    This is fundamentally different from a chatbot, which just responds to the last message, or a fixed pipeline, which runs steps A → B → C regardless of what happens along the way.

    Agents vs. Chatbots vs. Pipelines

    It helps to draw a hard line between three patterns that get conflated constantly:

  • Chatbot: stateless (or lightly stateful) request/response. User asks, model answers. No tool use, no planning.
  • Pipeline / workflow: a fixed sequence of LLM calls and deterministic steps, defined by the developer ahead of time. Reliable, but can’t adapt to novel situations.
  • Agent: the control flow itself is decided by the model at runtime. It chooses which tool to call, whether to retry, when to stop, and how to recover from errors.
  • Agents are more powerful but also less predictable and harder to debug — every extra decision the model makes is a new surface for it to get things wrong. That tradeoff should inform whether you actually need an agent or whether a simpler pipeline would do the job with far less operational risk.

    The Core Components of an Agentic System

    Every agentic AI implementation, regardless of framework, is built from the same handful of pieces.

    1. The Reasoning Loop

    The heart of an agent is the ReAct-style loop: Reason, Act, Observe, repeat. The model is prompted to think through the problem, choose an action, and the system executes that action and feeds the result back in. This continues until the model emits a “final answer” signal or a hard iteration limit is hit — that limit matters, because without it a buggy agent will happily loop forever, burning API credits.

    2. Tool Calling

    Tools are how an agent affects anything outside its own context window: searching the web, querying a database, running a shell command, hitting an internal API. Modern LLM providers (OpenAI, Anthropic, and others) expose native function-calling APIs, so the model returns structured JSON describing which tool to call and with what arguments, instead of you parsing free text.

    3. Memory and State

    Agents that run multi-step tasks need to remember what they’ve already tried. Short-term memory is usually just the running conversation/action history in the context window. Long-term memory — for agents that need to recall facts across sessions — typically uses a vector database or a simple key-value store. Don’t reach for a vector database on day one; most agents get by fine with a plain list of prior steps until you’ve proven you need more.

    Building a Minimal Agent From Scratch

    You don’t need a heavyweight framework to understand how agentic AI works. Here’s a minimal ReAct-style loop in Python using the OpenAI SDK’s function-calling interface:

    import json
    from openai import OpenAI
    
    client = OpenAI()
    
    def get_weather(city: str) -> str:
        # Stand-in for a real API call
        return f"It's 72F and sunny in {city}."
    
    TOOLS = [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get current weather for a city",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                },
            },
        }
    ]
    
    AVAILABLE_FUNCTIONS = {"get_weather": get_weather}
    
    def run_agent(user_prompt: str, max_steps: int = 5) -> str:
        messages = [{"role": "user", "content": user_prompt}]
    
        for _ in range(max_steps):
            response = client.chat.completions.create(
                model="gpt-4o",
                messages=messages,
                tools=TOOLS,
            )
            message = response.choices[0].message
            messages.append(message)
    
            if not message.tool_calls:
                return message.content
    
            for call in message.tool_calls:
                fn = AVAILABLE_FUNCTIONS[call.function.name]
                args = json.loads(call.function.arguments)
                result = fn(**args)
                messages.append({
                    "role": "tool",
                    "tool_call_id": call.id,
                    "content": result,
                })
    
        return "Max steps reached without a final answer."
    
    if __name__ == "__main__":
        print(run_agent("What's the weather in Lisbon?"))

    That’s the whole pattern: loop, check for tool calls, execute them, feed results back, repeat. Everything else — memory, planning strategies, multi-agent orchestration — is built on top of this core.

    Adding Real Tools: Shell Execution and Web Search

    Once the loop works, the next step is giving the agent tools that do something meaningful. A shell-execution tool is common for DevOps-focused agents:

    import subprocess
    
    def run_shell(command: str) -> str:
        result = subprocess.run(
            command, shell=True, capture_output=True, text=True, timeout=30
        )
        return result.stdout + result.stderr

    Be deliberate here: giving a model direct shell access is a serious security decision, not a convenience feature. At minimum, run it inside an isolated container with no access to secrets or the host filesystem, and whitelist the specific commands the agent is allowed to run rather than passing arbitrary strings to shell=True in a production system. Treat every tool the agent can call as an attack surface, the same way you’d treat a public API endpoint.

    Containerizing Your Agent with Docker

    Once your agent works locally, package it so it runs the same way everywhere. If you’re new to multi-service setups, our Docker Compose guide for beginners covers the fundamentals this builds on.

    FROM python:3.12-slim
    
    WORKDIR /app
    COPY requirements.txt .
    RUN pip install --no-cache-dir -r requirements.txt
    
    COPY . .
    
    ENV PYTHONUNBUFFERED=1
    USER nobody
    
    CMD ["python", "agent.py"]

    Running as a non-root user (USER nobody) matters more here than in a typical web app, because an agent with shell or filesystem tools is effectively remote code execution by design — you want the container’s own permissions to be your last line of defense if a tool call goes wrong. Build and run it:

    docker build -t my-agent .
    docker run --rm -e OPENAI_API_KEY=$OPENAI_API_KEY my-agent

    For anything beyond a single container, wire it into a docker-compose.yml alongside a vector database (like Qdrant or Chroma) and any supporting services your agent depends on.

    Deploying and Monitoring in Production

    A local Docker container is fine for development, but production agents need a real host, uptime monitoring, and a way to catch runaway loops before they burn through your API budget.

    For hosting, a DigitalOcean droplet is a solid, low-friction choice for running a containerized agent — you get a public IP, predictable pricing, and enough control to lock down the box the way an agent with tool access requires. If you’re comparing providers for this kind of workload, our guide to picking a VPS for Docker workloads walks through the tradeoffs in more depth.

    Once it’s running, you need to know immediately if the agent process dies or starts erroring on every request — an agent stuck in a retry loop can fail silently for hours otherwise. BetterStack gives you uptime checks and log aggregation without having to stand up your own Prometheus/Grafana stack for a single service. If your agent exposes an HTTP endpoint for other systems to call, put Cloudflare in front of it for basic DDoS protection and rate limiting — an unauthenticated agent endpoint is an expensive target if someone finds it and starts hammering it with requests that each trigger a paid LLM call.

    Rate Limiting and Cost Controls

    Agentic loops are the easiest way to accidentally run up a four-figure API bill overnight. At minimum:

  • Cap the number of loop iterations per task (5–10 is usually plenty).
  • Set a hard token budget per request and abort if it’s exceeded.
  • Log every tool call with its cost so you can audit spend after the fact.
  • Use a cheaper model for the reasoning/routing steps and reserve the expensive model for the final synthesis step, if your framework supports mixed models.
  • Common Pitfalls When Building Agentic AI

    A few mistakes show up in almost every agent project:

  • No iteration cap. Without a hard max-steps limit, a confused agent will loop indefinitely.
  • Overly broad tool permissions. Giving an agent full shell or filesystem access “to be safe” is backwards — narrow, purpose-built tools are both safer and easier for the model to use correctly.
  • Ignoring tool call failures. If a tool call errors out, feed the error back into the loop so the model can adapt, don’t just crash the whole run.
  • Skipping observability. If you can’t see the full trace of reasoning steps and tool calls after the fact, you can’t debug why the agent did something wrong.
  • Treating agents as a drop-in replacement for a deterministic pipeline. If the task has a fixed, known sequence of steps, a pipeline will be more reliable and cheaper than an agent every time.
  • Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Do I need LangChain or a similar framework to build agentic AI?
    No. Frameworks like LangChain, LlamaIndex, and CrewAI save time on boilerplate (tool schemas, memory stores, multi-agent orchestration), but the core reasoning loop is simple enough to write from scratch, as shown above. Many production teams start with a framework and later rip it out once they need finer control over the loop.

    What’s the difference between an agent and a RAG pipeline?
    RAG (retrieval-augmented generation) is a fixed pipeline: retrieve relevant documents, stuff them into the prompt, generate an answer. An agent can decide whether to retrieve, what to search for, and whether the results are good enough to answer with — RAG can be one of the tools an agent has available.

    How do I stop an agent from getting stuck in a loop?
    Set a hard maximum number of iterations, and consider adding a secondary check — like a separate, cheap model call — that flags when the agent is repeating the same action without making progress.

    Which LLM is best for building agentic AI?
    Models with strong native function-calling support (OpenAI’s GPT-4o, Anthropic’s Claude models) are the most reliable choice, since they return structured tool calls instead of you having to parse free-form text for intent.

    Is it safe to give an agent shell access?
    Only inside an isolated, disposable container with no access to secrets, and ideally with a whitelist of allowed commands rather than arbitrary execution. Never grant shell access to an agent running on the same host as production data or credentials.

    How much does it cost to run an agent in production?
    It depends entirely on iteration count and model choice, since each reasoning step is a separate API call. Logging token usage per tool call from day one is the only reliable way to catch cost blowouts before they show up on your bill.

    Building agentic AI isn’t about finding the right framework — it’s about understanding the loop, being deliberate about what tools you expose, and treating the infrastructure around the agent (containers, monitoring, rate limits) with the same rigor you’d apply to any other production service. Start with the minimal loop above, add one tool at a time, and only reach for memory stores, multi-agent orchestration, or a heavier framework once you’ve hit a concrete limitation the simple version can’t handle.

  • How to Create an AI Agent: A Developer’s Guide

    How to Create an AI Agent: A Practical Guide for Developers

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve spent any time reading DevOps forums or Hacker News in the last year, you’ve seen the term “AI agent” thrown around constantly. Most of the explanations stop at theory. This guide doesn’t. We’re going to build a working AI agent in Python, wrap it in Docker, and deploy it to a real Linux server — the same way you’d ship any other production service.

    By the end of this article you’ll understand what an AI agent actually is (beyond the marketing buzzword), how to structure one, how to give it tool-use capabilities, and how to run it reliably in production — including logging, monitoring, and basic security hardening.

    What Is an AI Agent, Really?

    An AI agent is a program that uses a large language model (LLM) as its reasoning engine, combined with a loop that lets it take actions, observe results, and decide what to do next — without a human manually scripting every step.

    The key difference between an AI agent and a simple chatbot wrapper is autonomy through iteration. A chatbot takes input and returns output once. An agent can:

  • Break a goal into subtasks
  • Call external tools or APIs (search, databases, shell commands, other services)
  • Evaluate whether the result moved it closer to the goal
  • Loop again if it didn’t
  • This is why agents are useful for DevOps and infrastructure work specifically — tasks like “check if this container is healthy, and restart it if not” or “summarize last night’s error logs and file a ticket” are naturally iterative and benefit from an LLM’s ability to reason over unstructured text.

    Core Components of an Agent

    Every functional AI agent, regardless of framework, has the same four pieces:

  • A model — the LLM doing the reasoning (OpenAI’s GPT models, Anthropic’s Claude, or a self-hosted model via Ollama)
  • A memory/state store — tracks conversation history and intermediate results
  • Tools — functions the agent can call (web search, file I/O, shell commands, database queries)
  • A control loop — the code that decides when to call the model, when to call a tool, and when to stop
  • You can build this from scratch in under 150 lines of Python, which is exactly what we’ll do below.

    Prerequisites

    Before you start, make sure you have:

  • Python 3.11+ installed locally
  • Docker installed (see our Docker installation guide if you’re starting fresh)
  • An API key from OpenAI, Anthropic, or a local model server
  • A Linux VPS if you plan to deploy the agent (we use Ubuntu 22.04 in this guide)
  • If you don’t already have a VPS, DigitalOcean is a solid choice for running small agent workloads — their basic droplets are cheap enough to experiment with without committing to a large monthly bill.

    Building the Agent Step by Step

    Setting Up the Environment

    Start with a clean virtual environment and the minimal dependencies. We’re deliberately avoiding a heavyweight framework for the first pass so you understand exactly what’s happening under the hood.

    mkdir ai-agent && cd ai-agent
    python3 -m venv venv
    source venv/bin/activate
    pip install openai python-dotenv

    Create a .env file to hold your API key — never hardcode credentials into your source:

    echo "OPENAI_API_KEY=sk-your-key-here" > .env

    Writing the Agent Core Loop

    This is the heart of the system: a loop that sends the current state to the model, checks whether it wants to call a tool, executes that tool if so, and feeds the result back in.

    # agent.py
    import os
    import json
    from dotenv import load_dotenv
    from openai import OpenAI
    
    load_dotenv()
    client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
    
    def get_disk_usage():
        import shutil
        total, used, free = shutil.disk_usage("/")
        return {
            "total_gb": round(total / (2**30), 2),
            "used_gb": round(used / (2**30), 2),
            "free_gb": round(free / (2**30), 2),
        }
    
    TOOLS = [
        {
            "type": "function",
            "function": {
                "name": "get_disk_usage",
                "description": "Return current disk usage stats for the root filesystem.",
                "parameters": {"type": "object", "properties": {}},
            },
        }
    ]
    
    AVAILABLE_FUNCTIONS = {"get_disk_usage": get_disk_usage}
    
    def run_agent(user_goal, max_iterations=5):
        messages = [
            {"role": "system", "content": "You are an infrastructure monitoring agent. Use tools when needed."},
            {"role": "user", "content": user_goal},
        ]
    
        for _ in range(max_iterations):
            response = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=messages,
                tools=TOOLS,
            )
            msg = response.choices[0].message
    
            if msg.tool_calls:
                messages.append(msg)
                for call in msg.tool_calls:
                    fn = AVAILABLE_FUNCTIONS[call.function.name]
                    result = fn()
                    messages.append({
                        "role": "tool",
                        "tool_call_id": call.id,
                        "content": json.dumps(result),
                    })
                continue
    
            return msg.content
    
        return "Max iterations reached without a final answer."
    
    if __name__ == "__main__":
        result = run_agent("Check disk usage and tell me if we're at risk of running out of space.")
        print(result)

    Run it:

    python agent.py

    The model will call get_disk_usage, receive real data from your machine, and respond with an actual assessment — not a hallucinated guess. This tool-calling pattern is the foundation every serious agent framework (LangChain, LangGraph, CrewAI) builds on top of.

    Adding More Tools

    Once the loop works, expanding capability is just a matter of adding functions and registering them:

  • A tool to tail recent log entries from a service
  • A tool to query a Prometheus endpoint for container metrics
  • A tool to restart a Docker container via the Docker SDK
  • A tool to post a summary to Slack or a ticketing system
  • Each tool should do one narrow thing and return structured data — resist the temptation to have a single “do everything” tool, since the model reasons much better with clearly scoped functions.

    Dockerizing the Agent

    Once the core loop works, package it so it runs the same way everywhere. Here’s a minimal production Dockerfile:

    FROM python:3.11-slim
    
    WORKDIR /app
    
    COPY requirements.txt .
    RUN pip install --no-cache-dir -r requirements.txt
    
    COPY agent.py .
    
    ENV PYTHONUNBUFFERED=1
    
    CMD ["python", "agent.py"]

    Build and run it:

    docker build -t ai-agent:latest .
    docker run --rm --env-file .env ai-agent:latest

    For anything beyond a one-off script, mount a volume for logs and pass secrets via environment variables rather than baking them into the image. If you’re new to container networking and volumes, our Docker networking deep dive covers the patterns you’ll need before going further.

    Deploying the Agent to a VPS

    Once containerized, deployment is straightforward. SSH into your server, pull the image (or build it there), and run it as a systemd-managed service or via docker run --restart unless-stopped for simplicity:

    ssh user@your-vps-ip
    docker pull yourregistry/ai-agent:latest
    docker run -d 
      --name ai-agent 
      --restart unless-stopped 
      --env-file /opt/ai-agent/.env 
      yourregistry/ai-agent:latest

    For anything running continuously (rather than a one-shot script), wrap the loop in a scheduler or a long-running process that sleeps between iterations, and make sure docker logs ai-agent gives you enough context to debug failures without SSHing in and guessing.

    Monitoring and Logging

    An agent that silently fails is worse than no agent at all — you need visibility into what it’s doing and why. At minimum:

  • Log every tool call and its result to stdout (captured automatically by Docker)
  • Log every model response, especially final decisions
  • Set up alerting for repeated failures or max-iteration timeouts
  • For production deployments, a dedicated uptime and log monitoring service pays for itself the first time your agent silently stops working at 3 a.m. BetterStack is worth evaluating here — their log aggregation and uptime monitoring integrate cleanly with containerized services and will page you before your users notice something’s wrong.

    Security Considerations

    Giving an LLM the ability to execute code or call APIs is powerful and dangerous in equal measure. Follow these rules:

  • Never let the agent execute arbitrary shell commands generated by the model without a strict allowlist
  • Sandbox tool execution — run agents in containers with minimal privileges, not on your host directly
  • Validate and sanitize any input that flows from the model into a database query, file path, or shell command
  • Rate-limit and log every external API call the agent makes
  • Put the agent’s public-facing endpoints (if any) behind a WAF and DDoS protection — Cloudflare is a reasonable default if you’re exposing a webhook or dashboard for the agent
  • Treat every tool the agent can call as a potential attack surface, the same way you’d treat a user-facing API endpoint.

    Scaling Beyond a Single Agent

    Once your first agent works reliably, the natural next step is running several agents for different tasks — one for log triage, one for cost reporting, one for deployment verification. At this point:

  • Isolate each agent in its own container with its own resource limits
  • Centralize logging so you’re not tailing five separate containers by hand
  • Consider a message queue (Redis, RabbitMQ) if agents need to hand off tasks to each other
  • Running multiple containers reliably means your underlying infrastructure needs to keep up. If you’re outgrowing a single small droplet, DigitalOcean‘s managed Kubernetes or larger droplet tiers are a straightforward upgrade path without re-architecting everything from scratch.

    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Do I need LangChain or CrewAI to build an AI agent?
    No. Those frameworks are useful once you have many tools and complex branching logic, but you can build a fully functional agent with plain Python and the OpenAI or Anthropic SDK, as shown above. Frameworks add convenience, not capability.

    Which LLM should I use for an agent?
    For tool-calling reliability, GPT-4o-mini, GPT-4o, and Claude models all support structured function calling well. Start with a cheaper model for development and upgrade only if you see reasoning failures in production.

    Can I run an AI agent without paying for API access?
    Yes — tools like Ollama let you run open models locally, though tool-calling support and reasoning quality vary by model. It’s a good option for prototyping or privacy-sensitive workloads.

    How do I stop an agent from looping forever?
    Always set a max_iterations cap in your control loop, as shown in the example above. Never let an agent run in an unbounded while loop in production.

    Is it safe to let an agent execute shell commands?
    Only with strict guardrails — an explicit allowlist of commands, no direct shell interpolation of model output, and execution inside an isolated container with minimal permissions.

    How is an AI agent different from a cron job with an API call?
    A cron job runs a fixed set of steps. An agent decides its own steps based on what it observes, and can adapt its plan mid-execution — that’s the core distinction.

    Wrapping Up

    Building an AI agent isn’t magic — it’s a control loop, a handful of well-scoped tools, and disciplined engineering around logging, security, and deployment. Start small: one tool, one clear goal, a hard iteration cap. Containerize it, deploy it to a real server, and monitor it like you would any other production service. Once that foundation is solid, adding more tools and more agents is incremental work, not a redesign.

    If you’re setting this up on your own infrastructure, revisit our Docker basics guide and Docker networking deep dive to make sure your container setup is production-ready before you put an agent in charge of anything that matters.