ThinkStreamTV — Blog

Category: Без рубрики

  • Generative AI vs Agentic AI: A DevOps Guide

    Generative AI vs Agentic AI: What DevOps Teams Actually Need to Know

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Every vendor pitch deck now claims to ship “agentic AI,” but half of them are still just wrapping a chatbot in a new label. If you’re the person actually responsible for uptime, deployments, and the 3am pager, you need a clear technical definition — not marketing copy. This article breaks down generative AI vs agentic AI in concrete terms, with real code, so you can decide which one belongs in your pipeline.

    If you’re evaluating hosting for either type of workload, providers like DigitalOcean offer straightforward GPU and CPU droplets that work well for both prototyping generative models and running long-lived agentic workers.

    What Is Generative AI?

    Generative AI refers to models that produce new content — text, code, images, audio — from a prompt. It’s fundamentally a single-shot transformation: input goes in, output comes out, and the model has no memory of what happens next unless you build that memory yourself.

    Examples you already use:

  • ChatGPT or Claude answering a question
  • GitHub Copilot autocompleting a function
  • DALL-E or Midjourney generating an image from a text prompt
  • An LLM summarizing a log file you paste in

    • The defining trait is statelessness by default. The model doesn’t decide to check your server’s disk usage before answering — it just generates the most probable continuation of your prompt based on training data and context window.

    How Generative AI Fits Into DevOps Pipelines

    In practice, generative AI shows up in DevOps as a text-in, text-out utility bolted onto existing tooling. A common pattern is piping logs or diffs into an API and getting back a summary or suggested fix:

    #!/usr/bin/env bash
# summarize-deploy-log.sh - send last 200 lines of a deploy log to an LLM for triage

LOG_FILE="/var/log/deploy/latest.log"

curl -s https://api.anthropic.com/v1/messages 
  -H "x-api-key: $ANTHROPIC_API_KEY" 
  -H "anthropic-version: 2023-06-01" 
  -H "content-type: application/json" 
  -d @- <<EOF
{
  "model": "claude-sonnet-5",
  "max_tokens": 500,
  "messages": [
    {"role": "user", "content": "Summarize the likely cause of failure in this deploy log:n$(tail -n 200 "$LOG_FILE")"}
  ]
}
EOF

    That’s it. The script sends data, gets a response, prints it to stdout. Nothing calls itself, nothing decides to restart a container, nothing loops. A human reads the summary and takes action. This is generative AI doing exactly one job well: compressing information for a person to act on.

    What Is Agentic AI?

    Agentic AI adds a control loop around the model: observe, decide, act, repeat — without a human in the loop for every step. The model doesn’t just answer a question; it’s given tools (shell access, an API, a database connection) and a goal, and it iterates until the goal is met or it hits a stopping condition.

    The core components of an agentic system:

  • A goal or task description (“keep this container’s memory under 80%”)
  • Tool access — functions the model can call, like run_shell, query_metrics, restart_service
  • A loop that feeds tool results back into the model’s context
  • Guardrails — timeouts, approval gates, or budget limits that stop runaway behavior

    • Agentic AI in Action: Autonomous Infrastructure Management

    Here’s a minimal (but real, runnable-shape) Python agent loop that monitors Docker container memory usage and restarts a container if it’s misbehaving — the kind of thing tools like LangChain or the Anthropic API are commonly used to build:

    import subprocess
import json
import time

def get_container_stats(name: str) -> dict:
    out = subprocess.check_output(
        ["docker", "stats", name, "--no-stream", "--format", "{{json .}}"]
    )
    return json.loads(out)

def restart_container(name: str) -> str:
    subprocess.run(["docker", "restart", name], check=True)
    return f"Restarted {name}"

def agent_loop(container: str, mem_threshold: float = 80.0, max_iterations: int = 20):
    for i in range(max_iterations):
        stats = get_container_stats(container)
        mem_pct = float(stats["MemPerc"].strip("%"))
        print(f"[iteration {i}] {container} memory: {mem_pct}%")

        if mem_pct > mem_threshold:
            print(restart_container(container))
            time.sleep(30)  # let it stabilize before re-checking
        else:
            time.sleep(10)

if __name__ == "__main__":
    agent_loop("streaming-transcoder")

    This is a toy example, but it demonstrates the key architectural difference: the loop itself makes decisions across multiple steps without a human approving each one. A production agent would replace the hardcoded threshold logic with an LLM call that reasons about why memory is climbing, checks recent deploys, and decides whether restarting is even the right move versus rolling back a release.

    Generative AI vs Agentic AI: Key Differences

    | Trait | Generative AI | Agentic AI |
    |—|—|—|
    | Execution | Single request/response | Multi-step loop |
    | Memory | None (unless you add it) | Maintains state across steps |
    | Tool use | Usually none | Calls APIs, shells, databases |
    | Human role | Reviews every output | Sets goals, reviews outcomes |
    | Failure mode | Bad text output | Bad actions taken autonomously |
    | Typical use | Drafting, summarizing, code suggestions | Monitoring, remediation, orchestration |

    The risk profile is the real differentiator. A generative model that hallucinates gives you a wrong paragraph. An agentic system that hallucinates a wrong conclusion can restart the wrong container, delete a volume, or scale down production during peak traffic. If you’re running anything agentic, you need real-time observability — this is where a service like BetterStack pairs well, since agent-driven infrastructure changes need the same alerting and uptime tracking you’d put around a human on-call engineer.

    Choosing Between Generative and Agentic AI for Your Stack

    A few practical rules we use when deciding which pattern fits a task:

  • Use generative AI when the output is consumed by a human who makes the final call (log summaries, PR descriptions, incident postmortem drafts).
  • Use agentic AI only when the loop is bounded, auditable, and reversible — e.g., scaling a stateless service, not deleting data.
  • Never let an agent hold destructive permissions by default. Scope its tool access to the minimum required (read-only metrics, restart — not rm, not docker system prune).
  • Log every tool call the agent makes, the same way you’d log an SSH session. If you can’t reconstruct what an agent did after the fact, you shouldn’t have let it run unsupervised.
  • Set hard iteration and time limits. A loop with no max_iterations is a denial-of-service bug waiting to happen against your own infrastructure.

    • If you’re building this into an existing Docker-based deployment pipeline, our guide on Docker Compose best practices covers how to structure services so an agent (or a human) can safely restart individual components without taking down the whole stack. And if you’re exposing any of this tooling behind a public endpoint, make sure it sits behind proper protection — Cloudflare in front of an agent’s control API is a sane default so you’re not accepting unauthenticated requests that trigger infrastructure changes.

    Where Streaming Infrastructure Uses Both

    For a site like ours that deals with transcoding pipelines and CDN edge caches, the split looks like this in practice: generative AI drafts release notes and summarizes CDN error logs; agentic AI watches transcoder queue depth and autoscales worker containers when a live stream spikes. Keeping those responsibilities separate — generation for humans, agency for bounded, reversible infrastructure actions — is what keeps an AI-assisted pipeline from becoming a liability. For monitoring the servers underneath either workload, see our self-hosted monitoring tools comparison for options that integrate cleanly with both generative summarization scripts and agentic control loops.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Is agentic AI just generative AI with more steps?
    Not exactly. Agentic AI typically uses a generative model as its reasoning engine, but the defining feature is the surrounding loop — tool access, state tracking, and multi-step decision-making — not the underlying model itself.

    Can I build an agentic system without an LLM?
    Yes. Classic rule-based automation (cron jobs, Kubernetes autoscalers, Ansible playbooks with conditionals) is agentic in the broad sense — it observes and acts autonomously. What’s new is using an LLM as the decision-making core instead of hardcoded rules, which adds flexibility but also unpredictability.

    Which one is riskier to run in production?
    Agentic AI, by a wide margin. A generative model’s worst case is a bad text output a human catches before acting on it. An agentic system’s worst case is an autonomous action — a restart, a deletion, a scale-down — taken without review.

    Do I need agentic AI if I already have good CI/CD automation?
    Probably not urgently. Traditional CI/CD is deterministic and auditable. Only reach for an agentic layer when the decision space is too fuzzy for fixed rules — for example, correlating multiple noisy signals before deciding whether to roll back a deploy.

    What’s the simplest way to start experimenting safely?
    Run the agent against a staging environment first, give it read-only access initially, and log every proposed action without executing it. Once you trust its reasoning, promote it to execute low-risk, reversible actions only.

    Are generative and agentic AI mutually exclusive in one pipeline?
    No — most mature setups use both. Generative AI handles reporting and human-facing summaries; agentic AI handles the narrow, bounded operational loop. Treating them as separate concerns, rather than one blob of “AI,” makes the system easier to reason about and secure.

    Generative AI and agentic AI solve different problems, and conflating them is how teams end up either underusing a powerful tool or handing too much autonomy to something that isn’t ready for it. Start with generative AI for anything a human reviews, and only graduate to agentic loops once you’ve defined tight guardrails, logging, and reversible failure modes.

    Related articles:

  • Generative AI vs. Agentic AI: A DevOps Guide

    Generative AI vs. Agentic AI: What DevOps Teams Need to Know

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Every infrastructure team is now being asked the same question: should we bolt a chatbot onto our stack, or build something that can actually act on its own? That question boils down to generative ai vs. agentic ai, and the distinction matters a lot more than marketing decks suggest. One produces content. The other executes tasks, calls APIs, and makes decisions with minimal human oversight. If you’re the one who has to containerize, deploy, and monitor these systems, the difference changes your entire architecture.

    This post breaks down the real technical differences, shows deployment patterns for each using Docker, and gives you a framework for deciding which one (or both) belongs in your stack.

    What Generative AI Actually Does

    Generative AI models — think GPT-4 class LLMs, Stable Diffusion, or Whisper — take an input and produce an output in a single pass or a short conversational loop. They generate text, images, audio, or code based on patterns learned during training. Critically, a generative model has no persistent goal, no memory of state beyond its context window, and no ability to independently decide to take an action unless something outside the model (your application code) tells it to.

    In practice, most “AI features” shipped today are generative AI wrapped in a thin application layer:

  • A support widget that summarizes a ticket
  • A code assistant that autocompletes a function
  • A tool that rewrites your changelog into release notes

    • These are stateless-per-call, deterministic to deploy, and easy to reason about from an ops perspective. You send a prompt, you get a completion, you log it, you move on.

    Deploying a Generative AI Service with Docker

    A typical generative AI microservice is just an API wrapper around a model endpoint. Here’s a minimal example using FastAPI and the OpenAI SDK, containerized for production:

    FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

    # main.py
from fastapi import FastAPI
from pydantic import BaseModel
from openai import OpenAI

app = FastAPI()
client = OpenAI()

class PromptRequest(BaseModel):
    prompt: str

@app.post("/generate")
def generate(req: PromptRequest):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": req.prompt}]
    )
    return {"output": response.choices[0].message.content}

    Build and run it like any other stateless service:

    docker build -t gen-ai-service .
docker run -d -p 8000:8000 -e OPENAI_API_KEY=$OPENAI_API_KEY gen-ai-service

    Scaling this horizontally behind a load balancer is trivial — there’s no shared state between replicas. If you’re already running services on a VPS, this pattern slots directly into the same Docker Compose stack you use for self-hosted apps.

    What Agentic AI Actually Does

    Agentic AI systems wrap a generative model in a loop that includes planning, tool use, memory, and self-correction. Instead of one prompt-in, one response-out, an agent decomposes a goal into steps, calls external tools (APIs, shell commands, databases), evaluates the results, and decides what to do next — often without a human approving each step.

    Frameworks like LangChain’s agent tooling or AutoGPT-style architectures exemplify this. An agent tasked with “reduce our AWS bill” might:

  • Query cost-explorer APIs
  • Identify idle EC2 instances
  • Draft a termination plan
  • Execute the plan (or open a PR for review)
  • Report back with a summary

    • That’s fundamentally different from a chatbot. The agent maintains state across steps, has access to real infrastructure, and can take destructive actions if you’re not careful. This is why agentic ai vs generative ai isn’t just an academic distinction — it’s a threat-model distinction.

    Deploying an Agentic AI Worker with Docker

    Agentic systems need persistent state (often Redis or a vector DB), a task queue, and sandboxed execution environments — because agents that can run shell commands need isolation. A common pattern is a Compose stack with an orchestrator, a worker, and a state store:

    # docker-compose.yml
version: "3.9"
services:
  agent-worker:
    build: ./worker
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - REDIS_URL=redis://redis:6379
    depends_on:
      - redis
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
    networks:
      - agent-net

  redis:
    image: redis:7-alpine
    networks:
      - agent-net

networks:
  agent-net:
    driver: bridge

    Notice the resource limits and dedicated network — agentic workers that execute arbitrary tool calls should never share a network namespace with production databases or have unrestricted host access. Treat the container boundary as a real security boundary, not a formality. If you’re running this on a VPS, pair it with proper firewall rules and container network isolation so a compromised or misbehaving agent can’t reach anything it shouldn’t.

    docker compose up -d
docker compose logs -f agent-worker

    Monitor these workers aggressively. Because agents make autonomous decisions, you need observability into every tool call, not just uptime. A dead-simple healthcheck won’t tell you an agent looped infinitely calling a paid API 10,000 times overnight.

    Generative AI vs. Agentic AI: The Core Tradeoffs

    | Factor | Generative AI | Agentic AI |
    |—|—|—|
    | State | Stateless per request | Persistent across steps |
    | Action | Produces content only | Calls tools, executes actions |
    | Ops complexity | Low — standard API service | High — needs orchestration, sandboxing |
    | Failure mode | Bad output | Bad output and real-world side effects |
    | Monitoring needs | Latency, cost per call | Full audit trail, action logging, kill switch |

    When to Choose Generative AI

    Use generative AI when the task ends with producing an artifact a human reviews before anything happens — copywriting, code suggestions, image generation, summarization. It’s cheaper to run, easier to secure, and far simpler to debug when something goes wrong.

    When to Choose Agentic AI

    Use agentic AI when you need multi-step task completion without a human in the loop for every step — automated incident triage, infrastructure remediation, or research pipelines that chain multiple tool calls. But budget real engineering time for sandboxing, rate limiting, and rollback mechanisms. An agent with docker exec access and no guardrails is a production incident waiting to happen.

    Hybrid Architectures Are the Practical Default

    Most production systems in 2026 aren’t purely one or the other. A common pattern is: agentic orchestration for planning and tool selection, generative calls for the actual content generation at each step. For example, an agent might decide which runbook to execute, then use a generative call to draft the incident summary for Slack. Keep the destructive actions (terminating instances, deleting data, pushing to prod) gated behind explicit approval steps regardless of how autonomous the rest of the pipeline is.

    For teams monitoring these hybrid pipelines, uptime and log aggregation tooling like BetterStack makes it far easier to catch an agent looping or a generative call silently failing before it becomes a cost overrun or an outage. If you’re hosting these workloads yourself, providers like DigitalOcean and Hetzner both offer straightforward VPS plans that handle containerized agent workers without the overhead of a full Kubernetes cluster — which is often overkill until you’re running dozens of concurrent agents.

    Security Considerations Specific to Agentic Systems

    Agentic AI introduces attack surface that generative-only systems don’t have:

  • Prompt injection leading to tool misuse: if an agent reads untrusted content (a webpage, an email) and that content contains instructions, the agent may follow them.
  • Excessive tool permissions: an agent with a single API key that can both read metrics and delete resources is a single point of failure.
  • Runaway loops: agents that retry indefinitely on failure can rack up API costs or hammer downstream services.
  • Unlogged autonomous actions: without a full audit trail, you can’t reconstruct why an agent did what it did after an incident.

    • Scope API keys per-tool, use short-lived credentials where possible, and log every tool invocation with its input and output. Treat agent permissions the same way you’d treat a CI/CD service account — least privilege, rotated regularly, and never shared across environments.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Is agentic AI just generative AI with extra steps?
    No. Agentic AI uses generative models as one component, but adds planning, memory, and autonomous tool execution on top. The generative model handles language understanding and output; the agent framework handles decision-making and action.

    Which is more expensive to run, generative AI or agentic AI?
    Agentic AI is almost always more expensive per task because a single goal can trigger many model calls plus tool invocations, compared to one call for a generative request. Budget for this with cost alerts and per-agent spending caps.

    Can I run agentic AI safely on a shared VPS?
    Yes, but isolate it. Run agent workers in their own Docker containers with strict resource limits, no shared network with production databases, and scoped API credentials. Never give an autonomous agent root or unrestricted host access.

    Do I need Kubernetes to deploy agentic AI workloads?
    Not necessarily. Docker Compose on a single VPS handles small-to-medium agent deployments fine. Move to Kubernetes when you need multi-node scaling, auto-recovery across hosts, or dozens of concurrent agent instances.

    What’s the biggest operational risk with agentic AI?
    Unbounded autonomous action — an agent that can execute destructive commands without a human checkpoint. Always gate irreversible actions (deletions, financial transactions, production deploys) behind explicit approval.

    Should generative AI and agentic AI be monitored differently?
    Yes. Generative AI monitoring focuses on latency, cost per call, and output quality. Agentic AI monitoring needs a full audit trail of every tool call, state transition, and decision point, because failures can cascade into real infrastructure changes.

    Wrapping Up

    The generative ai vs. agentic ai decision isn’t about picking the trendier term — it’s about matching the architecture to how much autonomy your task actually needs. Generative AI is the right call for content-producing features you can deploy as a simple stateless container. Agentic AI is the right call when you need multi-step task completion, but it demands real investment in sandboxing, permission scoping, and monitoring before it touches production systems. Start narrow, log everything, and expand agent autonomy only as your guardrails prove themselves.

    Related articles:

  • How to Create an AI Agent with Docker and Python

    How to Create an AI Agent: A Developer’s Step-by-Step Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Every framework, tutorial, and vendor pitch these days claims to help you “create an AI agent” in five minutes. Most of them skip the part that actually matters to developers and sysadmins: how do you build something reliable, containerize it, and run it in production without it falling over or leaking your API keys? This guide walks through the full lifecycle — architecture, code, Docker packaging, and deployment — using tools you already know.

    We’ll build a small but functional AI agent in Python, wrap it in Docker, and deploy it to a VPS with proper logging and monitoring. No hand-wavy diagrams, just working code you can copy and run.

    What Is an AI Agent?

    An AI agent is a program that uses a language model to decide what to do next, not just what to say. Unlike a chatbot that returns text, an agent can call tools, read files, hit APIs, and loop until a goal is satisfied. The distinction matters because it changes how you architect and deploy the thing.

    Agents vs. Chatbots vs. Scripts

  • A script runs a fixed sequence of steps you defined in advance.
  • A chatbot takes input and returns text — no autonomous action.
  • An agent observes state, reasons about a goal, chooses a tool, executes it, and repeats until done.

    • That loop — observe, reason, act, repeat — is the entire mental model. Everything else (frameworks like LangChain, vector databases, memory stores) is scaffolding around that loop.

    Core Components Every AI Agent Needs

    Before writing code, you need four pieces in place:

  • A model or API to do the reasoning (OpenAI, Anthropic, or a self-hosted model via Ollama)
  • A tool interface so the agent can take real actions (shell commands, HTTP calls, file I/O)
  • A loop controller that manages iterations, retries, and stop conditions
  • Logging and guardrails so the agent doesn’t run unbounded API calls or destructive commands

    • Skipping the guardrails is the single most common mistake we see in production incidents — an agent stuck in a retry loop can burn through an API budget or hammer a downstream service in minutes.

    Setting Up Your Development Environment

    You’ll need Python 3.11+, pip, and an API key from your model provider of choice. If you’re running this on a fresh Linux box, our Docker Compose guide for beginners covers getting a clean environment up quickly.

    Installing Python and Dependencies

    # Create an isolated environment
python3 -m venv agent-env
source agent-env/bin/activate

# Install core dependencies
pip install openai python-dotenv requests

    Store your API key in a .env file rather than hardcoding it:

    echo "OPENAI_API_KEY=sk-your-key-here" > .env
echo ".env" >> .gitignore

    Never commit API keys to version control. If you’re new to secrets management, the OpenAI API documentation has a good primer on key rotation and scoping.

    Building Your First AI Agent in Python

    Here’s a minimal but real agent loop. It takes a goal, decides whether to call a tool or respond directly, executes the tool, and feeds the result back until the model signals it’s done.

    # agent.py
import os
import json
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def get_weather(city: str) -> str:
    # Placeholder for a real API call
    return f"It is 72F and clear in {city}."

TOOLS = {
    "get_weather": get_weather,
}

TOOL_SCHEMA = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]

def run_agent(goal: str, max_iterations: int = 5) -> str:
    messages = [{"role": "user", "content": goal}]

    for i in range(max_iterations):
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
            tools=TOOL_SCHEMA,
        )
        choice = response.choices[0]
        messages.append(choice.message)

        if not choice.message.tool_calls:
            return choice.message.content

        for call in choice.message.tool_calls:
            fn_name = call.function.name
            args = json.loads(call.function.arguments)
            result = TOOLS[fn_name](**args)
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": result,
            })

    return "Max iterations reached without a final answer."

if __name__ == "__main__":
    print(run_agent("What's the weather like in Austin?"))

    That max_iterations cap is the guardrail mentioned earlier. Without it, a malformed tool response can send the agent into an infinite loop that quietly drains your API quota.

    Giving Your Agent Tools

    Real agents need more than a weather stub. Common tools include shell execution, file reads, and HTTP requests. Wrap each one defensively:

    import subprocess

def run_shell(command: str) -> str:
    # Restrict to an allowlist in production — never pass raw LLM output to shell=True
    allowed = {"ls", "df", "uptime"}
    cmd = command.split()[0]
    if cmd not in allowed:
        return f"Command '{cmd}' is not permitted."
    result = subprocess.run(command.split(), capture_output=True, text=True, timeout=10)
    return result.stdout or result.stderr

    The allowlist isn’t optional. An agent with unrestricted shell access is a remote code execution vulnerability with extra steps.

    Containerizing Your AI Agent with Docker

    Once the agent runs locally, package it so it behaves identically everywhere — your laptop, CI, and production VPS.

    Writing the Dockerfile

    FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY agent.py .

# Run as a non-root user
RUN useradd -m agentuser
USER agentuser

CMD ["python", "agent.py"]

    Running as a non-root user inside the container limits the blast radius if the agent’s shell tool is ever tricked into executing something unexpected.

    docker-compose for Local Testing

    version: "3.9"
services:
  ai-agent:
    build: .
    env_file: .env
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: "0.5"
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

    Build and run it:

    docker compose up --build

    The memory and CPU limits matter more for agents than typical web services — a runaway reasoning loop can spike CPU usage in a way a stateless API rarely does.

    Deploying Your AI Agent to a VPS

    For anything beyond local testing, you need a real server. A $6–12/month VPS is plenty for most single-agent workloads; you don’t need Kubernetes for this.

    Setting Up a Production-Ready VPS

    A solid starting spec looks like:

  • 2 vCPUs, 2–4GB RAM (agents are memory-light unless you’re running a local model)
  • Ubuntu 22.04 LTS or Debian 12
  • Docker and Docker Compose installed
  • A non-root deploy user with SSH key auth only
  • UFW or a cloud firewall restricting inbound ports to SSH and whatever port your agent exposes

    • Both DigitalOcean and Hetzner offer droplets/instances in this price range with one-click Docker images, which cuts your setup time down to a few minutes. If you’re comparing providers, our best VPS providers for self-hosting breakdown covers pricing and network performance in more depth.

    Once your server is ready:

    # On the VPS
git clone https://github.com/yourname/ai-agent.git
cd ai-agent
cp .env.example .env   # fill in real keys
docker compose up -d --build

    Check it’s running:

    docker compose logs -f ai-agent

    Monitoring, Logging, and Securing Your Agent

    An agent that fails silently is worse than one that fails loudly. At minimum, ship logs somewhere durable and set up uptime alerts.

  • Send container logs to a centralized log drain instead of relying on docker logs on a box you might reboot
  • Set up uptime and error-rate monitoring with a service like BetterStack so you get paged before a customer notices
  • If your agent exposes an HTTP endpoint, put it behind Cloudflare for DDoS protection, rate limiting, and TLS termination rather than exposing it raw
  • Rotate API keys regularly and scope them to the minimum permissions the agent actually needs
  • Cap token usage and dollar spend per run — most model providers let you set hard usage limits per API key

    • These aren’t nice-to-haves. An unmonitored agent with an unbounded loop and a live API key is one bad prompt away from a surprise bill.

    Handling Failures Gracefully

    Wrap your agent’s main loop in retry logic with exponential backoff for transient API failures, and make sure a crash restarts the container instead of leaving it dead:

        restart: unless-stopped

    That one line in your compose file has saved more on-call pages than any amount of clever error handling.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Do I need a framework like LangChain to create an AI agent?
    No. For a single-purpose agent, plain Python with the model provider’s SDK (as shown above) is easier to debug and has fewer dependencies. Frameworks earn their keep once you’re managing multiple agents, complex memory, or many tool integrations.

    Can I run an AI agent without paying for a cloud model API?
    Yes — tools like Ollama let you run open models like Llama or Mistral locally or on your own VPS, trading API cost for compute cost and slightly lower reasoning quality.

    How do I stop an agent from running away with API costs?
    Set a hard max_iterations cap in your loop, enforce per-key spend limits with your model provider, and monitor usage daily until you trust the agent’s behavior in production.

    Is Docker actually necessary for a simple agent script?
    Not strictly, but it guarantees the same Python version, dependencies, and OS libraries wherever you deploy, which eliminates an entire class of “works on my machine” bugs.

    How much does it cost to run an AI agent in production?
    Expect $6–15/month for the VPS, plus variable API costs that depend entirely on call volume and model choice — a lightweight agent making a few dozen calls a day can cost under $5/month in API fees.

    What’s the biggest security risk with AI agents?
    Giving the agent tools with more access than it needs — especially unrestricted shell or file system access. Always allowlist commands and scope credentials tightly.

    Creating an AI agent isn’t magic — it’s a loop, a couple of tools, and the same deployment discipline you’d apply to any other service. Start small, cap your iterations, containerize early, and monitor from day one.

    Related articles:

  • Ai For Real Estate Agents

    AI for Real Estate Agents: A Self-Hosted Deployment Guide for DevOps Teams

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    AI for real estate agents is quickly becoming less of a novelty and more of a standard operational layer, and for engineering teams tasked with building or supporting these systems, the real work isn’t the AI model itself — it’s the infrastructure around it. Lead scoring, listing description generation, appointment scheduling, and client follow-up all need reliable pipelines, not just a clever prompt. This guide walks through how to architect, deploy, and operate AI for real estate agents in a way that a working engineering team can actually maintain.

    Real estate brokerages and individual agents increasingly ask for “an AI assistant,” but what they actually need is a set of connected services: a language model for drafting and summarizing, a workflow engine to move data between CRM, MLS feeds, and messaging channels, and a database to persist leads and conversation history. This article focuses on the DevOps side of that stack — how to self-host it, secure it, and keep it running without surprises.

    Why Self-Host AI for Real Estate Agents

    Most vendors selling “AI for real estate agents” as a SaaS product charge per-seat or per-lead fees that scale poorly once a brokerage grows past a handful of agents. Self-hosting the orchestration layer — while still calling out to a managed LLM API when needed — gives you control over data retention, integration flexibility, and cost predictability.

    There are three practical reasons teams choose to self-host this kind of system instead of buying a turnkey SaaS tool:

  • Data ownership. Client conversations, financial pre-qualification details, and property preferences are sensitive. Keeping the orchestration and storage layer on infrastructure you control simplifies compliance conversations with brokers and legal teams.
  • Integration depth. Real estate workflows touch MLS feeds, DocuSign, CRM systems like Follow Up Boss or kvCORE, and SMS/email providers. A self-hosted workflow engine can integrate with all of these without waiting on a vendor’s roadmap.
  • Cost control. Per-agent SaaS pricing multiplies quickly across a brokerage. A self-hosted stack running on a modest VPS, paired with pay-per-token LLM calls, is often cheaper at scale.

    • Core Components of the Stack

    A typical self-hosted deployment for AI for real estate agents involves four moving pieces:

    1. A workflow automation engine (n8n is a common choice for teams already comfortable with node-based automation).
    2. A relational database for lead and conversation state (PostgreSQL is the standard choice).
    3. A message broker or webhook layer connecting SMS/email/chat channels to the workflow engine.
    4. An LLM API client for drafting responses, summarizing calls, or scoring lead intent.

    If you’re new to building this kind of orchestration layer, the general patterns in How to Create an AI Agent: A Developer’s Guide and How to Build Agentic AI: A Developer’s Guide apply directly here — real estate is simply a specific domain wrapped around the same agent architecture.

    Architecture Patterns for AI for Real Estate Agents

    Before writing any code, it helps to separate the system into three layers: ingestion, reasoning, and action. Ingestion pulls in new leads (form submissions, MLS updates, missed calls). Reasoning is where the LLM classifies intent, drafts a response, or scores lead quality. Action is where the system sends a message, updates the CRM, or schedules a follow-up task.

    Keeping these layers decoupled matters because each has different failure modes. A reasoning-layer outage (LLM API rate limit, timeout) shouldn’t block ingestion — you want new leads queued, not dropped. Similarly, action-layer failures (a CRM webhook returning a 500) shouldn’t corrupt the reasoning state.

    Workflow Orchestration with n8n

    n8n is a reasonable default for this because it has native nodes for HTTP requests, webhooks, and database access, and it’s straightforward to self-host on a small VPS. If you’re comparing it against alternatives, n8n vs Make: Workflow Automation Comparison Guide 2026 covers the tradeoffs in more depth. For teams starting from scratch, n8n Self Hosted: Full Docker Installation Guide 2026 walks through the base installation.

    A minimal docker-compose.yml for the orchestration layer looks like this:

    services:
  n8n:
    image: n8nio/n8n:latest
    restart: unless-stopped
    ports:
      - "5678:5678"
    environment:
      - N8N_HOST=0.0.0.0
      - N8N_PORT=5678
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
    volumes:
      - n8n_data:/home/node/.n8n
    depends_on:
      - postgres

  postgres:
    image: postgres:16
    restart: unless-stopped
    environment:
      - POSTGRES_DB=n8n
      - POSTGRES_USER=n8n
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    volumes:
      - pg_data:/var/lib/postgresql/data

volumes:
  n8n_data:
  pg_data:

    For database-specific tuning and volume management, Postgres Docker Compose: Full Setup Guide for 2026 is worth reviewing before you go to production.

    Connecting the LLM Layer

    The reasoning layer is typically an HTTP call out to an LLM provider’s API. Whether you use OpenAI, Anthropic, or a self-hosted model, the important DevOps concern is retry logic and rate-limit handling — real estate lead flow is bursty (open house weekends, new listing drops), and a naive integration will hit rate limits at exactly the wrong moment. If you’re evaluating API costs for this layer, OpenAI API Pricing: A Developer’s Cost Guide 2026 is a useful reference point.

    Deploying and Securing AI for Real Estate Agents on a VPS

    A single mid-tier VPS (2-4 vCPUs, 4-8 GB RAM) is usually sufficient to run n8n, PostgreSQL, and a lightweight reverse proxy for a single-brokerage deployment. Larger multi-office deployments benefit from separating the database onto its own instance.

    Basic checklist before exposing any part of this stack to the internet:

  • Put the workflow engine’s web UI behind a reverse proxy with TLS, never expose it directly on an unencrypted port.
  • Store all API keys (LLM provider, CRM, SMS gateway) in environment variables or a secrets manager, never in workflow JSON committed to version control.
  • Restrict database access to the internal Docker network only — do not publish the Postgres port externally.
  • Enable automated backups of the Postgres volume; lead and conversation history is not something you want to lose to a bad docker compose down -v.

    • If you’re new to managing environment variables and secrets in this kind of stack, Docker Compose Env: Manage Variables the Right Way and Docker Compose Secrets: Secure Config Management Guide cover the patterns in detail. For provisioning the underlying server, a straightforward option is a DigitalOcean droplet or a Hetzner cloud instance — either gives you enough headroom for this stack without over-provisioning.

    Debugging the Pipeline

    When a workflow silently stops responding to leads, the first place to look is the container logs, not the workflow UI. docker compose logs will surface API timeouts, malformed webhook payloads, or database connection errors long before the workflow engine’s own execution history does.

    docker compose logs -f n8n --tail=200

    For a deeper debugging workflow, including filtering by service and time range, Docker Compose Logs: The Complete Debugging Guide is directly applicable here.

    Rebuilding After Configuration Changes

    Any time you change environment variables or update the base image, rebuild rather than just restarting, so the container actually picks up the new configuration:

    docker compose down
docker compose pull
docker compose up -d --build

    Docker Compose Rebuild: Complete Guide & Best Tips covers the difference between a restart, a rebuild, and a full recreate — a distinction that matters when you’re troubleshooting why a config change didn’t take effect.

    Integrating AI for Real Estate Agents with CRM and Communication Tools

    The value of AI for real estate agents comes almost entirely from how well it’s wired into existing tools, not from the language model itself. A lead-scoring model that can’t write back to the CRM is just a dashboard. The integration layer typically needs to handle:

  • Inbound webhooks from lead-generation forms and MLS syndication feeds.
  • Outbound API calls to CRM systems to update lead status and add notes.
  • SMS/email dispatch for automated follow-up, gated behind human review for anything beyond a scheduling confirmation.
  • Calendar integration for showing appointments.

    • Each of these is a discrete workflow node or microservice, and each needs its own error handling and retry policy — a failed SMS send shouldn’t retry silently five times and spam a client, and a failed CRM write shouldn’t silently drop a lead update.

    Handling Rate Limits and Retries Gracefully

    Real estate lead spikes (a new listing going live, an open house weekend) can trigger bursts of concurrent LLM calls. Build exponential backoff into any HTTP request node calling an external API, and cap concurrency so a burst doesn’t exhaust your LLM provider’s rate limit or your own VPS’s connection pool. This is a general agentic-workflow concern, not specific to real estate, and the same patterns described in AI Agentic Workflow: Build Automated DevOps Pipelines apply directly.

    Monitoring and Maintaining the System Long-Term

    Once AI for real estate agents is live and handling real leads, monitoring shifts from “is the server up” to “is the pipeline producing correct output.” A workflow can be technically running while quietly sending malformed messages or misclassifying every lead as low priority.

    Practical monitoring targets:

  • Track the ratio of workflow executions that error out versus complete successfully, and alert on any sustained increase.
  • Log every LLM output before it’s sent to a client, even if only for a short retention window, so misbehavior can be audited.
  • Periodically spot-check automated lead classifications against manual agent review to catch model drift.
  • Monitor database growth — conversation history accumulates quickly across an active brokerage and can outgrow initial storage estimates.

    • For the underlying container orchestration reference, the Docker documentation and Kubernetes documentation are both worth bookmarking if this deployment eventually needs to scale beyond a single VPS into a multi-node setup.


    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Does AI for real estate agents replace human agents?
    No. The systems described here handle repetitive tasks — initial lead response, scheduling, drafting listing descriptions — while leaving negotiation, client relationships, and final decisions to the human agent. Framing it as augmentation rather than replacement also tends to get better buy-in from brokerages.

    What’s the minimum infrastructure needed to run AI for real estate agents in production?
    A single VPS with 2-4 vCPUs and 4-8 GB RAM is usually enough for a single-brokerage deployment running n8n, PostgreSQL, and a reverse proxy. Scale up the database and add load balancing only once you have evidence of real bottlenecks.

    How do I keep client data secure in a self-hosted AI for real estate agents setup?
    Restrict database access to the internal network, use TLS on any public-facing endpoint, store secrets outside of version control, and set explicit retention limits on conversation logs rather than keeping everything indefinitely.

    Can I use an open-source LLM instead of a commercial API for this?
    Yes, though it shifts the operational burden — you’re now responsible for hosting and scaling inference infrastructure yourself. For most brokerage-scale deployments, a pay-per-token commercial API is simpler to operate and cheaper than running dedicated GPU infrastructure.

    Conclusion

    AI for real estate agents is fundamentally a systems integration problem: connecting a language model to a workflow engine, a database, and a handful of external APIs (CRM, SMS, calendar, MLS). None of the individual pieces are exotic, but getting the retry logic, secrets management, and monitoring right is what separates a demo from something a brokerage can actually depend on. Start with a single VPS, a self-hosted workflow engine, and a managed LLM API, and expand the architecture only as real usage demands it.

    Related articles:

  • AI Agents Logo: Design, Generate & Deploy It with Docker

    AI Agents Logo: A Developer’s Guide to Designing and Deploying Branding for Your AI Agent Projects

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’re building, self-hosting, or shipping an AI agent — a Slack bot, a DevOps automation assistant, a RAG-powered support tool, or a full multi-agent orchestration platform — at some point you need a visual identity. An AI agents logo isn’t just decoration for a landing page. It shows up in your README badge, your dashboard favicon, your Docker Hub listing, your Slack app manifest, and every screenshot someone posts when they talk about your project. Get it wrong (or skip it entirely) and your tool looks unfinished, even if the underlying code is solid.

    This guide is written for developers and sysadmins, not designers. We’ll cover how to actually produce an AI agents logo, generate the full asset set you need (favicons, social cards, dashboard icons), and — the part most branding guides skip — how to serve those assets efficiently from a real Docker-based deployment behind Nginx and a CDN.

    Why Your AI Agent Needs a Logo (Even a Simple One)

    Visual Identity Builds Trust in Automated Systems

    Users are already wary of autonomous agents making decisions, sending messages, or touching infrastructure on their behalf. A consistent, professional-looking logo is a small but real trust signal. It tells a user this is a maintained product, not a weekend script, even when — behind the scenes — it might genuinely be a weekend script that grew up. Open-source maintainers consistently report that projects with a clear visual identity get more GitHub stars and faster community adoption than functionally identical but unbranded repos.

    Where the Logo Actually Gets Used

    Before you design anything, map out every surface the logo needs to appear on. For a typical self-hosted AI agent, that list looks like:

  • Browser tab favicon (16×16, 32×32, and Apple touch icon sizes)
  • Dashboard header / sidebar icon
  • README badge and social preview card (Open Graph image)
  • Docker Hub / GHCR repository icon
  • Slack, Discord, or Teams app manifest icon (if the agent integrates with chat platforms)
  • Terminal/CLI banner (ASCII or ANSI art version, if you ship a CLI)
  • Systemd or Docker container labels for observability dashboards like Grafana

    • If you design a single square PNG and call it done, you’ll be re-exporting assets every time a new surface comes up. Plan for the full set upfront.

    Designing the AI Agents Logo: Tools and Approaches

    AI-Generated Logo Options

    The fastest path is using an AI image generator to produce initial concepts, then cleaning the result up into a proper vector. Tools like Midjourney, DALL-E, or Adobe Firefly can produce strong concept art in minutes, but they output raster images (PNG/JPEG), not the scalable vector format you actually need for a production logo. Treat AI-generated output as a mood board, not a final asset.

    A practical workflow:

    1. Generate 10-20 concepts with prompts describing your agent’s function (e.g., “minimal geometric mascot for an AI DevOps automation agent, dark background, single accent color, flat vector style”).
    2. Pick the strongest 2-3 concepts.
    3. Trace or rebuild the winner as a proper SVG using a tool like Inkscape or Figma.
    4. Simplify aggressively — a logo that needs to render at 16x16px in a browser tab cannot have fine detail.

    Hand-Built SVG Icons for Full Control

    If you’re comfortable with basic vector shapes, building the mark directly in SVG gives you a small, dependency-free asset you can version-control alongside your codebase. A minimal geometric mark for an AI agent — a hexagon, a circuit-node motif, or a stylized eye — is easy to reproduce cleanly at any size and easy to recolor for dark/light theme dashboards.

    <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64">
  <polygon points="32,4 58,18 58,46 32,60 6,46 6,18"
           fill="#0f172a" stroke="#38bdf8" stroke-width="2"/>
  <circle cx="32" cy="32" r="10" fill="#38bdf8"/>
</svg>

    This kind of asset is tiny (a few hundred bytes), scales perfectly, and can be themed with CSS custom properties if you inline it in your dashboard’s HTML.

    Turning a Logo File into a Full Asset Set

    Once you have a master SVG or high-resolution PNG, generate every derivative size you mapped out earlier. A quick script using ImageMagick covers most of it:

    #!/usr/bin/env bash
set -euo pipefail

SRC="logo-master.svg"
OUT="dist/branding"
mkdir -p "$OUT"

for size in 16 32 48 64 128 192 512; do
  convert -background none -density 300 "$SRC" 
    -resize "${size}x${size}" "$OUT/logo-${size}.png"
done

convert "$OUT/logo-16.png" "$OUT/logo-32.png" "$OUT/logo-48.png" "$OUT/favicon.ico"

convert -background none -density 300 "$SRC" 
  -resize 1200x630 -gravity center -extent 1200x630 
  "$OUT/og-image.png"

echo "Generated asset set in $OUT"

    At minimum, your generated set should include:

  • favicon.ico (multi-resolution, for legacy browser support)
  • logo-192.png and logo-512.png (PWA manifest icons)
  • apple-touch-icon.png (180×180)
  • og-image.png (1200×630, for Slack/Twitter/Discord link previews)
  • A monochrome/inverted variant for dark-mode dashboards

    • If you’re integrating with a real favicon generator to double-check cross-browser rendering, RealFaviconGenerator is worth running your master file through once before shipping.

    Serving Your Logo Assets with Docker and Nginx

    Once assets are generated, they need to be served efficiently. If your AI agent dashboard already runs in Docker — see our guide on self-hosting AI agents with Docker — the cleanest approach is baking the static assets into the image and serving them with Nginx as a reverse proxy in front of your agent’s API.

    Directory layout:

    agent-dashboard/
├── Dockerfile
├── nginx.conf
├── docker-compose.yml
└── dist/
    ├── index.html
    └── branding/
        ├── favicon.ico
        ├── logo-192.png
        ├── logo-512.png
        └── og-image.png

    Dockerfile:

    FROM nginx:1.27-alpine

COPY dist/ /usr/share/nginx/html/
COPY nginx.conf /etc/nginx/conf.d/default.conf

EXPOSE 80

    nginx.conf, with long-lived caching for the branding assets specifically:

    server {
    listen 80;
    server_name _;

    root /usr/share/nginx/html;

    location /branding/ {
        add_header Cache-Control "public, max-age=31536000, immutable";
    }

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /api/ {
        proxy_pass http://agent-api:8000/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

    docker-compose.yml tying the dashboard and the agent API together — pair this with our Nginx reverse proxy guide for Docker containers if you need TLS termination as well:

    version: "3.9"
services:
  dashboard:
    build: .
    ports:
      - "80:80"
    depends_on:
      - agent-api

  agent-api:
    image: your-org/ai-agent-api:latest
    environment:
      - AGENT_MODE=production
    restart: unless-stopped

    Because the branding assets get Cache-Control: immutable, browsers and any CDN in front of the box will cache the logo aggressively — which matters once you have real traffic hitting the dashboard.

    Automating Asset Regeneration in CI

    Manually running the ImageMagick script before every release is easy to forget. Wire it into CI so the asset set regenerates automatically any time the master SVG changes, and fails the build if someone commits a stale derivative alongside an updated source file.

    name: regenerate-branding
on:
  push:
    paths:
      - "logo-master.svg"
jobs:
  build-assets:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install ImageMagick
        run: sudo apt-get update && sudo apt-get install -y imagemagick
      - name: Generate branding assets
        run: ./scripts/generate-branding.sh
      - name: Commit regenerated assets
        run: |
          git config user.name "branding-bot"
          git config user.email "[email protected]"
          git add dist/branding
          git commit -m "chore: regenerate branding assets" || true
          git push

    This keeps the favicon, Open Graph image, and dashboard icon permanently in sync with the source SVG, and it means a designer can update the master file without ever touching the Dockerfile or Nginx config.

    Optimizing Logo Images for Fast Load Times

    Compression and Format Choices

    A bloated logo is a surprisingly common source of unnecessary page weight. Before shipping:

  • Run PNGs through a compressor like Squoosh or pngquant — a well-optimized 512px logo should be under 20KB.
  • Prefer SVG wherever the target supports it; it’s smaller than PNG for flat geometric marks and scales without a blurry edge.
  • Serve WebP or AVIF variants for the Open Graph preview image where the platform supports it, falling back to PNG.
  • Never serve a 4000px source PNG scaled down in CSS — always export the exact pixel dimensions you need.

    • For background on the broader performance implications, web.dev’s image optimization guide is a solid reference, and it applies directly to dashboard assets, not just marketing pages. If your dashboard is already slow, check our image optimization walkthrough for more before-and-after benchmarks.

    For SVG specifically, run the exported file through SVGO to strip editor metadata, unnecessary precision, and comments left behind by design tools — a hand-exported SVG from Figma or Illustrator is often 3-5x larger than it needs to be before optimization.

    Deploying the Branded Dashboard to Production

    Once the container builds cleanly and assets are cached correctly, the actual hosting decision matters more than people expect. A small self-hosted AI agent dashboard doesn’t need a Kubernetes cluster — a single droplet is plenty, and DigitalOcean makes it straightforward to spin up a droplet, attach a floating IP, and run the docker-compose.yml above with almost no extra configuration. If you also want HTTPS out of the box, pair it with our Let’s Encrypt and Nginx SSL guide.

    For the logo and static assets specifically, sitting a CDN in front of the origin server pays off fast once the dashboard has real users. Cloudflare‘s free tier alone will cache the /branding/ path at edge locations globally, cut load time on the favicon and Open Graph image to near-zero, and absorb traffic spikes if a post about your agent goes viral on Hacker News or Reddit.

    A Note on Consistency Across Surfaces

    One mistake worth calling out: teams often redesign the logo mid-project and forget to regenerate every derivative asset. If you update the master SVG, rerun the generation script above and rebuild the Docker image — don’t hand-edit individual PNGs. Keeping a single source of truth (the master SVG, committed to the repo) and a single generation script avoids drift between the favicon, the README badge, and the Slack manifest icon, which otherwise slowly fall out of sync.

    Treat the AI agents logo the same way you’d treat any other production asset: version it, automate its build, cache it aggressively, and keep a single source of truth. It’s a small piece of the overall project, but it’s often the first thing a new user sees, and a polished one costs very little once the pipeline above is in place.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Do I need a professional designer for an AI agents logo, or can I use an AI generator?
    For an internal tool or early-stage open-source project, an AI generator plus manual SVG cleanup is enough. If the agent is a commercial product, budget for a few hours with a designer to get a mark that holds up at small sizes and across light/dark themes.

    What file format should the master logo file be?
    SVG. It’s resolution-independent, tiny, easy to version-control as a text diff, and every other format (PNG, ICO, WebP) can be generated from it programmatically.

    How many logo sizes do I actually need to generate?
    At minimum: 16, 32, 48 (favicon set), 180 (Apple touch icon), 192 and 512 (PWA manifest), and a 1200×630 Open Graph image. That covers browsers, mobile home screens, and chat link previews.

    Can I serve the logo directly from GitHub instead of my own server?
    You can for a README badge, but not for a production dashboard — GitHub’s raw content CDN isn’t meant for high-traffic asset serving and has no SLA. Bake assets into your Docker image or serve them via your own CDN instead.

    Does the logo need to change between light mode and dark mode?
    Ideally yes. Ship a monochrome or inverted variant and swap it with a CSS media query (prefers-color-scheme: dark) so the mark stays legible against both backgrounds.

    How do I keep the favicon from being cached with a stale version after a redesign?
    Version the filename (e.g., favicon-v2.ico) or append a query string cache-buster when you update the master logo, since the immutable cache header above means browsers won’t re-check the old file otherwise.

    Related articles:

  • Ai Agent Frameworks

    AI Agent Frameworks: A DevOps Guide to Choosing and Deploying One

    Choosing among the growing set of ai agent frameworks is now a routine part of shipping automation, not a niche research exercise. Teams building anything from support bots to internal ops tooling need a framework that fits their infrastructure, not just their prototype notebook. This guide walks through how ai agent frameworks are structured, how to evaluate them, and how to deploy one on your own infrastructure with Docker.

    What Ai Agent Frameworks Actually Provide

    An agent framework sits between a large language model and the rest of your stack. On its own, an LLM takes text in and produces text out. A framework adds the scaffolding that turns that into something useful in production:

  • A loop that lets the model call tools, observe results, and decide on a next step
  • Memory management, so context from earlier in a conversation or task persists
  • Structured output parsing, so the model’s response can drive real code paths
  • Integration points for external APIs, databases, and queues
  • Guardrails around retries, timeouts, and error handling when a tool call fails

    • Without this layer, every team ends up hand-rolling the same retry logic, prompt templating, and tool-calling parser. That duplicated effort is exactly what ai agent frameworks are meant to remove.

    Core Components Shared Across Frameworks

    Most frameworks, regardless of language or vendor, converge on a similar internal architecture:

  • Planner/executor loop — decides what action to take next based on the current state
  • Tool registry — a typed interface describing what functions the agent can call and their expected arguments
  • Memory store — short-term (conversation buffer) and sometimes long-term (vector store or database-backed) memory
  • Observability hooks — logging of each step so you can debug why an agent made a particular decision

    • Understanding these components matters more than picking a specific brand name, because it lets you evaluate a new framework on its merits rather than its marketing.

    Where Frameworks Diverge

    Where frameworks differ is in opinionation. Some enforce a strict graph-based execution model where every transition is explicit. Others are closer to a thin SDK that gives you primitives and lets you assemble your own loop. Neither approach is inherently better — a strict graph model gives you more predictable behavior and easier debugging in production, while a lightweight SDK gives you more flexibility for unusual workflows. This is a genuine architectural decision, similar in spirit to choosing between a Dockerfile and Docker Compose for a build: one is more explicit and constrained, the other more flexible and closer to raw primitives.

    Evaluating Ai Agent Frameworks for Production Use

    Before adopting any of the popular ai agent frameworks, run through a short evaluation checklist rather than defaulting to whatever has the most GitHub stars this month.

    Deployment and Runtime Requirements

    Check what the framework actually needs to run in production:

  • Does it require a persistent process, or can it run as a short-lived function/task?
  • What are its dependencies — does it pull in a large ML stack, or is it a thin HTTP client around a hosted model API?
  • Can it run inside a container without special GPU or system-level dependencies (unless you’re self-hosting the model too)?
  • Does it support horizontal scaling if you need to run many agent instances concurrently?

    • If you’re already running workflow automation on a VPS, you likely have infrastructure patterns you can reuse. Teams who’ve set up n8n on a self-hosted VPS or built agent-style automations directly in n8n’s own agent nodes already understand the tradeoffs of running orchestration logic close to their own data instead of a fully managed SaaS platform.

    Observability and Debugging

    Agent behavior is inherently less deterministic than typical application code, which makes observability non-negotiable. Look for:

  • Structured logs of every tool call, including inputs and outputs
  • The ability to replay a specific agent run against a fixed transcript for debugging
  • Integration with your existing logging stack rather than a proprietary dashboard you have to check separately

    • If a framework’s only debugging tool is a hosted web console with no exportable logs, that’s a real constraint worth weighing against your existing observability tooling.

    Popular Categories of Ai Agent Frameworks

    Rather than listing specific products (which change quickly), it’s more durable to think in categories:

    Orchestration-First Frameworks

    These frameworks model the agent’s behavior as an explicit graph or state machine. Each node represents a step, and transitions between nodes are defined up front. This style tends to be easier to reason about and test, because you can inspect the graph without running the agent at all. It’s a good fit for workflows with well-understood branching logic — approval flows, multi-step data pipelines, or anything where a human might need to audit the decision path afterward.

    Autonomous-Loop Frameworks

    These frameworks give the model more latitude to decide its own next step at each iteration, typically bounded by a maximum number of steps or a budget. This style is well suited to open-ended research or exploration tasks where the exact sequence of tool calls can’t be known ahead of time. The tradeoff is less predictability and a greater need for strict timeouts and cost caps.

    SDK-Style / Minimal Frameworks

    Some frameworks are closer to a library than a framework: they expose primitives for tool calling, memory, and prompt construction, but leave the control flow entirely to your own code. This is a reasonable choice for teams who already have strong opinions about how they want their agent loop structured and don’t want to fight a framework’s assumptions.

    Deploying an Ai Agent Framework with Docker

    Whatever framework you choose, containerizing it is the same exercise as containerizing any other service: pin your dependencies, keep secrets out of the image, and give it a clean way to talk to the rest of your stack.

    A minimal example for a Python-based agent framework might look like this:

    FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

ENV PYTHONUNBUFFERED=1

CMD ["python", "agent_worker.py"]

    And a corresponding Compose file that wires in environment-based configuration and a queue dependency:

    version: "3.9"
services:
  agent:
    build: .
    restart: unless-stopped
    env_file:
      - .env
    depends_on:
      - redis
    networks:
      - agent_net

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    networks:
      - agent_net

networks:
  agent_net:
    driver: bridge

    Keep your model API keys in .env, not baked into the image, and treat the container the same way you’d treat any other stateless worker — if your agent framework needs durable memory, back it with an external store like Redis or Postgres rather than the container’s local filesystem. If you need to debug a misbehaving agent container in production, the same log inspection habits used for debugging Docker Compose logs apply directly — tail the worker’s stdout and correlate timestamps against your framework’s own step logs.

    Managing Secrets and Configuration

    Agent frameworks typically need at least one API key (for the LLM provider) plus credentials for any tools the agent calls. Treat these the same way you’d treat any other sensitive configuration — injected via environment variables or a secrets manager, never committed to source control. If you’re already following a Compose-based secrets pattern elsewhere in your stack, extend it here rather than inventing a separate mechanism just for the agent service.

    Scaling Considerations

    If you expect to run many agent instances concurrently (for example, one per incoming support ticket), design for horizontal scaling from the start: keep the agent process itself stateless, push memory and task state into an external store, and use a queue to distribute work rather than relying on in-process concurrency. This mirrors how you’d scale any other worker-style service, and most ai agent frameworks are compatible with this pattern as long as you avoid keeping state only in local process memory.

    Common Pitfalls When Adopting Ai Agent Frameworks

    A few mistakes show up repeatedly across teams adopting agent frameworks for the first time:

  • Treating the framework as a black box. Understanding the actual loop — what triggers a tool call, what happens on failure — is necessary to debug production issues.
  • Skipping timeouts and step limits. Autonomous-loop style frameworks in particular can run far longer (and cost far more) than expected without a hard cap.
  • No cost visibility. Every tool call and every LLM round-trip has a cost. Log token usage per agent run so you can catch runaway loops before they show up on a bill.
  • Ignoring idempotency. If an agent’s tool call fails partway through and retries, make sure the retried action doesn’t duplicate side effects (e.g., sending the same email twice).
  • Under-investing in evaluation. Without a repeatable way to test agent behavior against known inputs, regressions in prompt or framework version upgrades go unnoticed until a user reports them.

    • Building this discipline early avoids the same category of duplicate-execution problems that show up in any production pipeline lacking proper ownership and idempotency guarantees — the same principle behind locking mechanisms used in automated SEO pipelines or content publishing systems that must guarantee a step only runs once per item.

    FAQ

    Do I need an agent framework, or can I just call an LLM API directly?
    For a single, simple tool call or a one-shot completion, a direct API call is often enough. Frameworks earn their keep once you need multi-step reasoning, several tools, persistent memory, or a repeatable structure across many different agents.

    Are ai agent frameworks tied to a specific LLM provider?
    Most modern frameworks are designed to be model-agnostic, supporting multiple providers behind a common interface. Check a framework’s documentation for which providers are officially supported before committing, since support quality varies.

    Can ai agent frameworks run entirely self-hosted, without calling an external API?
    Yes, if you pair the framework with a self-hosted model server. The framework itself (the loop, tool registry, memory) is typically independent of where the model runs — you can point it at a hosted API or a local inference server.

    How do I choose between an orchestration-first framework and a lightweight SDK?
    Start with how well-defined your workflow is. If you can draw the decision tree today, an orchestration-first framework will make that tree explicit and testable. If the task genuinely requires open-ended exploration, a lighter SDK that doesn’t fight the model’s own judgment will likely serve you better.

    Conclusion

    Ai agent frameworks are converging on a shared set of concepts — planning loops, tool registries, memory, and observability — even as individual products differ in how opinionated they are about control flow. The right choice depends less on which framework is trending and more on your actual workflow: how deterministic it needs to be, how it fits your existing deployment model, and how well you can observe and debug it once it’s running. Treat deploying an agent framework the same way you’d treat any other production service — containerized, stateless where possible, with real logging and cost controls — and you’ll avoid most of the operational surprises that come with adopting this category of tooling. For further reading on the underlying model APIs these frameworks build on, see the Docker documentation for container runtime specifics and the Kubernetes documentation if you’re planning to scale agent workers beyond a single host.

    Related articles:

  • Voice AI Agents: A DevOps Guide to Self-Hosting

    Voice AI Agents: A DevOps Guide to Self-Hosting and Deploying Them at Scale

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Voice AI agents are moving out of the demo phase and into production infrastructure. If you’re a developer or sysadmin who’s been asked to deploy a voice agent for customer support, IVR replacement, or an internal automation tool, the API-only route (hosted realtime speech APIs) gets expensive fast at scale, and it hands your audio data to a third party. This guide walks through the architecture of self-hosted voice AI agents, how to containerize the stack with Docker, and the infrastructure decisions that determine whether your deployment survives real production traffic.

    What Are Voice AI Agents, Exactly?

    A voice AI agent is a pipeline, not a single model. It listens to audio, transcribes it, reasons about what was said, generates a response, and speaks that response back — usually in under a second if it’s going to feel natural. Under the hood, that means chaining together at least three distinct systems: speech-to-text (STT), a language model or dialogue manager, and text-to-speech (TTS), plus an orchestration layer that manages turn-taking, interruptions, and session state.

    Most teams start with a hosted API because it’s the fastest way to get a proof of concept working. But once you’re running thousands of concurrent calls, or you have compliance requirements around where audio data is processed and stored, self-hosting becomes the more defensible option — both financially and operationally.

    Core Components of a Voice AI Agent Pipeline

    Every voice agent stack, whether hosted or self-hosted, breaks down into the same functional blocks:

  • Speech-to-text (STT): Converts incoming audio into text. Open-source options like Whisper and its faster C++ port whisper.cpp are the de facto standard for self-hosted deployments.
  • Dialogue/reasoning layer: An LLM or rules-based state machine that decides what the agent should say next. This is where you’d plug in a local model via Ollama or call out to a hosted LLM API.
  • Text-to-speech (TTS): Converts the response text back into audio. Piper and Coqui TTS are common self-hosted choices; both run comfortably on CPU for low-latency use cases.
  • Orchestration/session layer: Handles WebRTC or SIP audio streaming, voice activity detection (VAD), interruption handling, and conversation state. Frameworks like LiveKit Agents and Vocode exist specifically to solve this plumbing problem.
  • Telephony bridge (if applicable): Connects the pipeline to a phone network via SIP trunking, typically through something like Twilio or a self-hosted Asterisk/FreeSWITCH instance.

    • Understanding this breakdown matters because each component has different resource requirements. STT and TTS are GPU-friendly but can run on CPU with acceptable latency for smaller models; the dialogue layer is where most of your compute cost lives if you’re self-hosting an LLM.

    API-Based vs Self-Hosted: The Real Tradeoff

    The decision isn’t binary — most production voice agents end up as a hybrid. A common pattern is self-hosting STT and TTS (which are commoditized and cheap to run) while calling out to a hosted LLM API for reasoning (where model quality still matters most). Here’s how the tradeoffs actually shake out:

  • Cost at scale: Hosted STT/TTS APIs typically bill per minute of audio. At high call volumes, self-hosted whisper.cpp or Piper on a modest VPS pays for itself within weeks.
  • Latency: Self-hosting removes a network hop to a third-party API, which can shave 100-300ms off round-trip response time — often the difference between a conversation that feels natural and one that feels laggy.
  • Data residency and compliance: If you’re handling healthcare, financial, or EU customer data, keeping audio processing in your own infrastructure avoids a lot of contractual and legal complexity.
  • Operational burden: Self-hosting means you own uptime, scaling, and model updates. This is the real cost that gets underestimated.

    • If you’re already comfortable running containerized services in production, the operational burden is manageable. If you’re not, start with hosted APIs and revisit self-hosting once volume justifies it.

    Deploying a Voice AI Agent Stack with Docker

    Docker is the natural fit here because a voice agent pipeline is inherently multi-service, and each service (STT, TTS, orchestration, telephony bridge) has different dependencies that are painful to manage on bare metal. Below is a minimal but functional docker-compose.yml that stands up a self-hosted voice agent stack using whisper.cpp for STT, Piper for TTS, Redis for session state, and an Nginx reverse proxy in front of the orchestration service.

    # docker-compose.yml
version: "3.9"

services:
  stt:
    image: ghcr.io/ggerganov/whisper.cpp:main
    container_name: voice-stt
    command: ["./server", "-m", "models/ggml-base.en.bin", "--host", "0.0.0.0", "--port", "8081"]
    ports:
      - "8081:8081"
    volumes:
      - whisper_models:/app/models
    restart: unless-stopped

  tts:
    image: rhasspy/piper:latest
    container_name: voice-tts
    command: ["--model", "en_US-lessac-medium", "--port", "8082"]
    ports:
      - "8082:8082"
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    container_name: voice-session-store
    volumes:
      - redis_data:/data
    restart: unless-stopped

  orchestrator:
    build: ./orchestrator
    container_name: voice-orchestrator
    environment:
      - STT_URL=http://stt:8081
      - TTS_URL=http://tts:8082
      - REDIS_URL=redis://redis:6379
      - LLM_API_KEY=${LLM_API_KEY}
    depends_on:
      - stt
      - tts
      - redis
    restart: unless-stopped

  nginx:
    image: nginx:alpine
    container_name: voice-proxy
    ports:
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./certs:/etc/nginx/certs:ro
    depends_on:
      - orchestrator
    restart: unless-stopped

volumes:
  whisper_models:
  redis_data:

    The orchestrator is your own service — it’s the glue that receives audio from the client (browser or SIP trunk), streams it to the STT service, sends the transcript to your LLM of choice, and pipes the response through TTS back to the caller. It’s also where voice activity detection and interruption handling live.

    Building the Docker Compose Stack

    A minimal orchestrator can be built in Python using aiohttp for the WebSocket audio stream and redis for session persistence. Here’s a stripped-down example of the turn-handling logic that ties the pipeline together:

    # orchestrator/agent.py
import asyncio
import aiohttp
import redis.asyncio as redis

STT_URL = "http://stt:8081/inference"
TTS_URL = "http://tts:8082/synthesize"

async def handle_turn(audio_chunk: bytes, session_id: str, r: redis.Redis):
    async with aiohttp.ClientSession() as session:
        # 1. Transcribe incoming audio
        async with session.post(STT_URL, data=audio_chunk) as resp:
            transcript = (await resp.json())["text"]

        # 2. Persist conversation turn to Redis
        await r.rpush(f"session:{session_id}", transcript)

        # 3. Call the LLM for a response (pseudo-call, swap for your provider)
        reply_text = await generate_reply(session_id, transcript, r)

        # 4. Synthesize the reply back to audio
        async with session.post(TTS_URL, json={"text": reply_text}) as resp:
            audio_out = await resp.read()

    return audio_out

    This is intentionally minimal — in production you’d add streaming partial transcripts, barge-in handling (letting the caller interrupt the agent mid-sentence), and retry logic around the LLM call. But the shape of the pipeline doesn’t change: audio in, text out, text in, audio out, with Redis tracking state across turns.

    Bring the stack up with:

    docker compose up -d --build
docker compose logs -f orchestrator

    For a deeper dive into structuring multi-service Compose files for production, see our guide on running Docker Compose in production.

    Scaling and Infrastructure Considerations

    Voice agents are latency-sensitive in a way that most web services aren’t. A 500ms delay on a page load is annoying; a 500ms delay in a phone conversation makes the agent feel broken. That changes your infrastructure priorities.

    Choosing Infrastructure for Production Voice Workloads

    A few things matter more here than in typical web app hosting:

  • CPU/GPU balance: whisper.cpp’s smaller models (base, small) run well on CPU; if you need the large model for accuracy, budget for a GPU instance.
  • Network proximity to callers: For SIP/telephony workloads, deploy in a region close to your call volume to minimize jitter. Hetzner and DigitalOcean both offer solid price-to-performance for CPU-bound STT/TTS workloads across multiple regions.
  • Horizontal scaling of the orchestrator: The orchestrator should be stateless beyond what’s stored in Redis, so you can scale it horizontally behind a load balancer as concurrent call volume grows.
  • Autoscaling STT/TTS pools: Run multiple replicas of the STT and TTS containers behind a simple round-robin proxy once you exceed what a single instance can handle concurrently.

    • If you’re evaluating providers for this kind of CPU-heavy, latency-sensitive workload, our comparison of VPS providers for Docker workloads covers the tradeoffs between Hetzner’s price-performance and DigitalOcean’s managed tooling in more depth. Both are solid starting points — spin up a DigitalOcean droplet if you want managed load balancers and easy horizontal scaling, or go with Hetzner if raw CPU-per-dollar is your priority for running multiple whisper.cpp replicas.

    For DNS and edge protection in front of your orchestration API, putting Cloudflare in front of your public endpoints gives you DDoS mitigation and TLS termination without extra config on your Nginx layer.

    Monitoring and Reliability

    A voice agent that silently drops calls is worse than one that’s slow — silence looks like a hang-up to the caller, and you won’t know it happened unless you’re watching for it. At minimum, instrument:

  • STT/TTS response latency (p50, p95, p99)
  • Call setup success rate and average call duration
  • Redis connection health and session TTL expirations
  • Container restart counts and memory usage per service

    • Running docker stats gives you a quick local read on resource usage, but for anything customer-facing you want uptime and latency alerting that pages you before customers complain:

    docker stats --no-stream voice-stt voice-tts voice-orchestrator

    For production alerting, an external uptime and status-page service like BetterStack can monitor your orchestrator’s health endpoint and page you the moment latency spikes or the service goes down — which matters a lot more for a live voice pipeline than it does for a static website.

    Security Considerations for Voice Agents

    Voice pipelines handle sensitive audio and, often, transcripts containing PII. Treat the stack accordingly:

  • Encrypt audio in transit: Terminate TLS/DTLS at the edge (Nginx or a WebRTC-aware proxy) — never pass raw RTP or WebSocket audio over plaintext.
  • Don’t log full transcripts by default: Redact or hash session identifiers in logs, and set a short TTL on Redis session data unless you have an explicit retention requirement.
  • Isolate the LLM API key: Keep provider credentials in environment variables or a secrets manager, never baked into the orchestrator image.
  • Rate-limit inbound connections: A voice agent endpoint exposed to the public internet is a target for abuse; put rate limiting at the Nginx or Cloudflare layer.
  • Patch base images regularly: whisper.cpp, Piper, and Redis images should be rebuilt on a schedule, not left pinned to a stale tag indefinitely.

    • If you’re new to hardening containerized services generally, our Docker container monitoring and security guide covers the baseline practices — resource limits, read-only filesystems, and non-root users — that apply just as much to a voice pipeline as any other multi-container app.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: Do I need a GPU to self-host voice AI agents?
    A: Not necessarily. Smaller Whisper models (base, small) and Piper TTS run acceptably on modern CPUs for low-to-moderate concurrency. A GPU becomes worthwhile once you need the larger, more accurate STT models or you’re running dozens of concurrent sessions per node.

    Q: What’s the biggest source of latency in a self-hosted voice agent?
    A: Usually the LLM call in the dialogue layer, not STT or TTS. Streaming partial transcripts to the LLM and streaming its response into TTS as it’s generated (rather than waiting for the full response) is the single biggest latency win available.

    Q: Can I run this stack on a single small VPS?
    A: For testing and low-volume use, yes — a 4-8 vCPU instance can handle a handful of concurrent sessions comfortably. For production call volume, plan to scale the STT/TTS containers horizontally across multiple nodes.

    Q: How is a voice AI agent different from a chatbot?
    A: A chatbot operates on text with generous response-time tolerance. A voice agent has to handle real-time audio, voice activity detection, and interruptions, and users expect sub-second response times — the engineering constraints are much tighter.

    Q: Is self-hosting actually cheaper than hosted APIs?
    A: It depends on volume. Below a few thousand minutes per month, hosted APIs are usually cheaper once you account for engineering time. Above that, self-hosted STT/TTS on a well-sized VPS typically wins on cost, especially if you’re already running Docker infrastructure.

    Q: Do I need SIP/telephony integration, or can voice agents run in the browser?
    A: Both are common. WebRTC-based agents run entirely in-browser via frameworks like LiveKit and don’t need a telephony bridge at all. You only need SIP/Asterisk integration if the agent needs to answer or place actual phone calls.

    Wrapping Up

    Self-hosting voice AI agents is a solved infrastructure problem if you’re already comfortable with Docker and container orchestration — the hard part is tuning latency, not standing up the services. Start with a minimal Compose stack like the one above, measure your actual latency bottlenecks before optimizing, and scale the STT/TTS layer horizontally once real traffic demands it.

    Related articles:

  • Docker Compose Build: Complete Guide to Building Images

    Docker Compose Build: A Practical Guide to Building Images Correctly

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve ever run docker compose up and wondered why your code changes weren’t reflected in the container, the answer almost always traces back to how docker compose build works. This command is the backbone of any local development workflow that uses custom images instead of pulling pre-built ones from a registry, and understanding its mechanics will save you hours of confused debugging.

    This guide covers everything from the basic syntax to advanced caching strategies, build arguments, multi-stage builds, and the most common pitfalls developers run into when building images through Compose.

    What docker compose build Actually Does

    When you define a build key in your docker-compose.yml, Compose doesn’t just run a generic build — it reads the context, Dockerfile, and any build-time configuration you’ve specified, then hands the job off to the Docker build engine (BuildKit, by default on modern installs). The resulting image is tagged according to your service name and project, unless you override it with an explicit image key.

    Here’s a minimal example:

    services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "3000:3000"

    Running the build is straightforward:

    docker compose build

    This builds every service in the file that has a build section. If you only want to rebuild one service, target it by name:

    docker compose build web

    Difference Between build and up

    A common point of confusion is the relationship between docker compose build and docker compose up --build. Running docker compose up alone will use existing images if they exist and won’t rebuild automatically, even if your Dockerfile changed. Adding --build forces a rebuild before starting containers:

    docker compose up --build

    In practice, most developers alias this command or bake it into a Makefile because forgetting --build after editing a Dockerfile is one of the most common sources of “why isn’t my change showing up” bug reports. If you’re running a multi-service stack, this is also worth combining with a solid Docker Compose networking setup so services can reach each other reliably after rebuilds.

    Build Caching and Why It Matters

    Docker layers each instruction in your Dockerfile, and it caches those layers so subsequent builds are faster — as long as nothing upstream of a given instruction has changed. docker compose build respects this same caching behavior. The order of instructions in your Dockerfile directly affects build speed:

    FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
CMD ["node", "server.js"]

    By copying package.json before the rest of the source code, npm ci only reruns when dependencies actually change, not on every source edit. This pattern alone can cut build times from minutes to seconds during active development.

    If you need to bypass the cache entirely — for example, after a base image security patch — use:

    docker compose build --no-cache

    For a deeper dive into layer caching and image size reduction, the official Docker build cache documentation is worth reading in full, especially the section on cache invalidation triggers.

    Using Build Arguments

    Build arguments let you pass values into the build process without hardcoding them into the Dockerfile. This is especially useful for things like version pinning or environment-specific configuration that shouldn’t be baked into a production image.

    services:
  api:
    build:
      context: .
      args:
        NODE_ENV: production
        APP_VERSION: 1.4.2

    Your Dockerfile then references these with ARG:

    FROM node:20-alpine
ARG NODE_ENV
ARG APP_VERSION
ENV NODE_ENV=${NODE_ENV}
LABEL version=${APP_VERSION}
WORKDIR /app
COPY . .
RUN npm ci --omit=dev
CMD ["node", "server.js"]

    You can also override args at build time from the CLI:

    docker compose build --build-arg APP_VERSION=1.5.0 api

    Keep in mind that build args are visible in the image history unless you use secrets mounts, so never pass credentials or API keys this way — that’s a common security misstep. If you’re handling secrets in your build pipeline, pair this with proper Docker security hardening practices to avoid leaking sensitive values into image layers.

    Multi-Stage Builds With Compose

    Multi-stage builds are one of the most effective ways to keep production images small while still having full build tooling available during compilation. Compose handles multi-stage Dockerfiles natively — you just need to tell it which target to build.

    # Build stage
FROM node:20 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Production stage
FROM node:20-alpine AS production
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
CMD ["node", "dist/server.js"]

    In your docker-compose.yml, specify the target stage:

    services:
  app:
    build:
      context: .
      target: production

    This approach routinely shrinks final image sizes by 60-80% compared to single-stage builds that ship build tools and dev dependencies into production.

    Parallel Builds and Performance

    By default, docker compose build builds services sequentially. If you have multiple independent services, you can speed things up significantly by building in parallel:

    docker compose build --parallel

    This only helps when services don’t depend on each other’s build output. If one service’s Dockerfile copies artifacts from another service’s build context, parallel builds won’t help and could even cause race conditions — structure your Dockerfiles to avoid cross-service file dependencies where possible.

    Common Build Errors and How to Fix Them

    A few errors show up constantly in docker compose build logs:

  • “failed to compute cache key” — usually means a file referenced in COPY or ADD doesn’t exist in the build context. Double-check your .dockerignore isn’t excluding something you need.
  • “context deadline exceeded” — often a network timeout pulling a base image or dependency. Retry, or check if you’re behind a corporate proxy that needs Docker daemon configuration.
  • “no space left on device” — your local Docker storage is full of dangling images and build cache. Run docker system prune to reclaim space (use -a to also remove unused images, but be careful on shared machines).
  • Stale dependency errors after editing package.json — usually means the layer cache wasn’t invalidated correctly. Force a clean rebuild with --no-cache for that one service.
  • Build succeeds locally but fails in CI — almost always an environment difference, like a missing .env file or a platform architecture mismatch (Apple Silicon vs. x86 CI runners).

    • For the platform mismatch issue specifically, you can pin the target platform explicitly:

    services:
  app:
    build:
      context: .
      platforms:
        - linux/amd64

    Debugging a Failing Build Interactively

    When a build fails partway through and the error message isn’t clear, it helps to build up to the failing layer and inspect the intermediate state:

    docker build --target production -t debug-image .
docker run -it debug-image sh

    This drops you into a shell inside the partially built image so you can manually run the commands that failed and see the actual output, rather than guessing from truncated CI logs.

    Optimizing Your Compose Build Workflow

    A few practical habits make a real difference for teams running Compose day to day:

  • Pin base image versions (node:20.11-alpine instead of node:latest) to avoid surprise breakage when upstream images update.
  • Use .dockerignore aggressively to keep build contexts small — excluding node_modules, .git, and test fixtures can cut context upload time dramatically on large repos.
  • Separate docker-compose.yml and docker-compose.override.yml so local dev overrides (bind mounts, debug ports) don’t leak into production configs.
  • Tag images explicitly with image: alongside build: so you can push the same image you tested locally without an unnecessary rebuild.
  • Run builds in CI with --no-cache periodically (e.g., weekly) to catch cache-related drift that wouldn’t otherwise surface.

    • If your team is deploying these built images to a VPS or cloud instance rather than just running locally, it’s worth comparing providers before committing to one. DigitalOcean offers straightforward Droplets that work well for small-to-medium Compose-based deployments, and their managed container registry integrates cleanly with docker compose push workflows if you want to skip building images repeatedly on the target server.

    For teams running builds and deployments at higher frequency, keeping an eye on server resource usage during build spikes matters too — CPU and memory can spike hard during compilation-heavy multi-stage builds, so monitoring your host is worth setting up before it becomes a problem. Once your images are built and running, pairing them with proper container health checks and monitoring will help you catch build-related regressions before they hit production.

    Choosing Infrastructure for Your Build Pipeline

    If you’re self-hosting your CI runners or build servers, the underlying hardware matters more than people expect — slow disk I/O alone can double build times on image-heavy Dockerfiles. Hetzner is a solid budget option for dedicated build servers with fast NVMe storage, which noticeably speeds up layer extraction and cache writes compared to standard cloud block storage.

    Frequently Asked Questions

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: What’s the difference between docker build and docker compose build?
    A: docker build builds a single image from a Dockerfile and context you specify manually on the command line. docker compose build reads your docker-compose.yml and builds every service with a build: section, using the context, args, and target defined in that file — it’s essentially a batch wrapper that keeps your build configuration version-controlled alongside your service definitions.

    Q: Why doesn’t docker compose up rebuild my image automatically?
    A: Compose intentionally avoids rebuilding on every up to save time — it assumes the existing image is still valid unless told otherwise. Use docker compose up --build to force a rebuild, or run docker compose build separately before starting containers.

    Q: How do I rebuild only one service instead of the whole stack?
    A: Pass the service name directly: docker compose build <service-name>. This only rebuilds that service’s image, leaving others untouched and speeding up iteration in multi-service projects.

    Q: Can I use build secrets with docker compose build without exposing them in the image?
    A: Yes. Use BuildKit secret mounts in your Dockerfile with RUN --mount=type=secret,id=mysecret, and reference the secret in your Compose file’s build.secrets section. This keeps sensitive values out of the final image layers and build history entirely.

    Q: Why is my build cache not being used even though nothing changed?
    A: Check whether your build context includes files that change on every run, like log files or generated artifacts not excluded in .dockerignore. Even a timestamp file in the context can invalidate cache for every subsequent COPY . . instruction.

    Q: Does docker compose build push images to a registry?
    A: No, build only builds and tags images locally. To push, tag the service with an image: field pointing to your registry path, then run docker compose push separately after a successful build.

    —

    Getting comfortable with docker compose build — its caching behavior, build args, and multi-stage targets — pays off quickly once you’re managing more than a couple of services. Start by auditing your existing Dockerfiles for cache-friendly instruction ordering, since that’s usually the fastest win available with zero risk.

    Related articles:

  • Agentic AI Courses: Best Picks for DevOps Engineers

    Agentic AI Courses: A Practical Guide for DevOps and Cloud Engineers

    Agentic AI systems — software that can plan, call tools, and execute multi-step tasks with minimal human intervention — are moving from research demos into production infrastructure. If you run Docker hosts, manage Kubernetes clusters, or maintain a streaming or self-hosted media stack, you’ve probably already seen an agent framework mentioned in a changelog or a vendor pitch deck. The problem is that most agentic ai courses on the market are built for product managers and prompt writers, not for the people who actually deploy, monitor, and secure these systems on real servers.

    This guide breaks down what agentic AI actually means for infrastructure teams, what separates a useful course from a marketing funnel, and how to build a sandboxed lab on a cheap VPS so you can test agent frameworks before anything touches production.

    Why Agentic AI Matters for Infrastructure Teams

    Traditional automation — cron jobs, Ansible playbooks, CI/CD pipelines — executes a fixed sequence of steps. Agentic AI is different: an agent is given a goal, a set of tools (shell access, an API client, a database connector), and a loop that lets it observe results and decide the next action. In practice, this looks like an agent that reads a failing health check, queries logs, identifies a memory leak in a container, and restarts the service with a corrected resource limit — without a human writing the exact remediation script in advance.

    For DevOps and platform teams, this changes the shape of the job. Instead of writing every automation path by hand, you’re increasingly responsible for scoping what an agent is allowed to touch, auditing what it did, and rolling back when it gets something wrong. That’s a genuinely different skill set than traditional scripting, and it’s why picking the right agentic ai courses matters more than skimming a blog post or two.

    If you’re already running a self-hosted stack, this is directly relevant to how you’ll manage infrastructure going forward. Teams that maintain their own monitoring stack for Docker hosts are natural candidates for agent-assisted alert triage, since the agent can read the same Prometheus and Loki data a human on-call engineer would.

    What Makes a Good Agentic AI Course

    Most of the “agentic AI” course catalog right now is prompt-engineering content wearing a new label. Before you pay for anything, filter for courses that actually cover infrastructure-relevant skills.

    Hands-On Labs with Real Infrastructure

    A course that only shows you a Jupyter notebook calling a hosted API isn’t teaching you anything about running agents in production. Look for courses that require you to spin up your own compute — a VPS, a local Docker host, or a Kubernetes namespace — and deploy an agent that talks to real services: a database, a message queue, a set of internal APIs. If the course’s final project can be completed entirely inside a free-tier notebook with no infrastructure of your own, it’s not going to prepare you for on-call reality.

    Framework Coverage: LangChain, AutoGen, and CrewAI

    The agent framework landscape moves fast, but three names have stayed relevant long enough to be worth learning properly: LangChain for tool-calling and retrieval pipelines, Microsoft’s AutoGen for multi-agent conversation patterns, and CrewAI for role-based agent orchestration. A good course won’t just teach one framework’s API syntax — it should explain the underlying patterns (planning loops, tool schemas, memory management) so the knowledge transfers when the next framework replaces the current favorite.

    Production Deployment and Observability

    This is the section most courses skip entirely, and it’s the one that matters most for this audience. You need to understand how to containerize an agent process, set token and cost budgets, log every tool call the agent makes for audit purposes, and set up alerting for runaway loops (an agent stuck retrying a failing API call can burn through an API budget in minutes). If a course doesn’t mention rate limiting, sandboxing, or logging at all, treat that as a red flag.

    Top Agentic AI Courses Worth Evaluating

    A few programs consistently come up when infrastructure engineers compare notes on what was actually worth the time:

  • DeepLearning.AI’s agent-focused short courses — free, framework-specific, and taught by the people building the tools; good for a fast primer before committing to a paid program. See their catalog at deeplearning.ai.
  • LangChain Academy — free, official documentation-adjacent course that walks through LangGraph state machines, which is the closest thing to a production pattern for agent control flow.
  • Vendor-neutral cloud provider workshops — several cloud providers now run hands-on labs pairing agent frameworks with managed compute; these are useful specifically because they force you to think about cost and infrastructure limits, not just API calls.
  • University extension programs on multi-agent systems — heavier on theory, useful if you want to understand the research behind planning and memory architectures rather than just tool usage.
  • Paid bootcamp-style programs — worth it only if they include a graded infrastructure project; otherwise you’re paying for content available for free elsewhere.

    • When comparing options, prioritize courses with a capstone project that deploys to real infrastructure over ones that end with a slide deck.

    Build a Sandboxed Agent Lab on a VPS

    The fastest way to actually learn this material is to run it yourself, isolated from anything that matters. A small VPS with Docker installed is enough. If you don’t already have a preferred provider, our guide to choosing a VPS for Docker workloads covers the tradeoffs between the usual options.

    Start with a docker-compose file that isolates the agent process, gives it a scoped API key, and logs everything:

    # docker-compose.yml
version: "3.9"
services:
  agent-runtime:
    build: ./agent
    container_name: agent-sandbox
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - AGENT_MAX_STEPS=8
      - AGENT_TOOL_ALLOWLIST=http_get,read_file
    volumes:
      - ./agent/logs:/app/logs
    networks:
      - agent-net
    restart: "no"
    mem_limit: 512m
    cpus: 0.5

networks:
  agent-net:
    driver: bridge
    internal: true

    Notice the internal: true network setting and the hard memory/CPU limits — those two lines do more to keep a misbehaving agent contained than anything in the agent’s own code. Full reference for the compose spec is in the Docker Compose documentation.

    Next, a minimal Python agent loop to run inside that container, so you can see exactly how the plan-act-observe cycle works before you adopt a heavier framework:

    # agent/main.py
import os
import json
import logging
from openai import OpenAI

logging.basicConfig(
    filename="logs/agent.log",
    level=logging.INFO,
    format="%(asctime)s %(message)s",
)

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
MAX_STEPS = int(os.environ.get("AGENT_MAX_STEPS", 5))

def http_get(url: str) -> str:
    import requests
    resp = requests.get(url, timeout=5)
    return resp.text[:2000]

TOOLS = {"http_get": http_get}

def run_agent(goal: str):
    messages = [{"role": "user", "content": goal}]
    for step in range(MAX_STEPS):
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
        )
        content = response.choices[0].message.content
        logging.info("step=%s output=%s", step, content)

        try:
            action = json.loads(content)
            tool = action.get("tool")
            if tool in TOOLS:
                result = TOOLS[tool](action.get("arg", ""))
                messages.append({"role": "assistant", "content": content})
                messages.append({"role": "user", "content": f"Result: {result}"})
                continue
        except (json.JSONDecodeError, AttributeError):
            pass

        print(content)
        return content

    logging.warning("agent hit max_steps without finishing")

if __name__ == "__main__":
    run_agent("Check if example.com is returning a 200 status.")

    This is deliberately simple — a real course will take you much further into structured tool schemas and retry logic — but running this on your own VPS teaches you the failure modes (infinite loops, malformed tool calls, runaway costs) that a hosted notebook demo will never show you.

    Monitoring and Securing Your Agent Workflows

    Once an agent has shell or API access, treat it like any other service account: least privilege, full audit logging, and alerting on anomalous behavior. A few practices worth adopting before you let an agent touch anything beyond a sandbox:

  • Scope API keys per agent, not per team, so you can revoke one agent’s access without breaking others.
  • Log every tool call with timestamp, arguments, and result — this is your incident response trail if something goes wrong.
  • Set hard step and token limits at the framework level, not just in your prompt instructions.
  • Run agents in network-isolated containers, similar to the compose example above, so a compromised or confused agent can’t reach production systems directly.
  • Alert on cost or call-volume spikes the same way you’d alert on CPU or memory spikes.

    • If you’re already running a hardened SSH configuration on your Linux servers, extend that same least-privilege mindset to agent service accounts — they’re a new kind of user on your systems, and they deserve the same scrutiny.

    Cost and Token Budgeting

    Agentic workflows can be expensive in ways traditional automation isn’t, because every planning step is a paid API call. A course that doesn’t cover cost estimation is leaving out a core operational skill. Before deploying anything beyond a sandbox, calculate a worst-case cost per run (max steps × average tokens per step × your model’s price), set a hard budget alert, and log actual spend against that estimate. This is the same discipline you’d apply to any cloud cost — the difference is that agent costs can spike far faster than a forgotten EC2 instance, because a stuck loop can make hundreds of calls in minutes.

    FAQ

    Do I need to know machine learning to take agentic AI courses?
    No. Most infrastructure-focused courses assume you can write Python and use APIs, not that you understand model training. Agentic AI courses aimed at engineers focus on orchestration, tool integration, and deployment — not model internals.

    Which agentic AI courses are actually free?
    DeepLearning.AI’s short courses and LangChain Academy are both free and cover real framework usage rather than generic prompt tips. They’re a reasonable starting point before paying for anything.

    How long does it take to become productive with agent frameworks?
    Most engineers with existing scripting and API experience can build a working sandboxed agent within a few days. Getting comfortable with production concerns — cost control, sandboxing, observability — usually takes a few weeks of hands-on practice.

    Can I run agent frameworks on a small VPS, or do I need a GPU?
    Most agent frameworks call a hosted model API rather than running inference locally, so a small VPS with a few gigabytes of RAM is enough for orchestration. You only need a GPU if you’re self-hosting the model itself.

    Is LangChain still worth learning given how fast the space changes?
    Yes, mainly because the underlying patterns — tool schemas, memory, state graphs — transfer to other frameworks even if LangChain’s specific API changes. It also remains the most widely documented option, which matters when you’re debugging.

    What’s the biggest mistake engineers make when deploying their first agent?
    Skipping hard limits. Engineers new to agentic systems often let an agent run with no step cap, no cost ceiling, and full network access, then are surprised when a bug causes runaway API calls or unintended side effects. Treat agent permissions the same way you’d treat a new service account: scoped, logged, and capped.

    Agentic AI courses are only as useful as the infrastructure discipline you bring to what they teach. Pick a course with a real deployment component, build the sandbox yourself, and apply the same monitoring and least-privilege habits you already use for every other service running on your servers.

    Related articles:

  • AI Agent Framework: A Developer’s Deployment Guide

    AI Agent Framework: A Practical Guide for Developers Running Production Workloads

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve spent time evaluating an ai agent framework for a real project, you’ve probably noticed the field moves faster than the documentation can keep up with. New releases from LangChain, AutoGen, CrewAI, and LangGraph land every few weeks, each promising better tool-calling, memory, and orchestration. For developers and sysadmins who actually have to deploy, monitor, and secure these systems, the marketing noise doesn’t matter — what matters is uptime, cost control, and not leaking API keys into a public repo.

    This guide skips the hype and focuses on what you need to know to pick, deploy, and run an AI agent framework on infrastructure you control, whether that’s a Docker host on your homelab or a fleet of VPS instances.

    What Is an AI Agent Framework?

    An AI agent framework is a software layer that sits between a large language model (LLM) and the tools, APIs, and data sources it needs to complete a task autonomously. Instead of a single prompt-response cycle, an agent framework gives the model:

  • A planning loop that breaks a goal into steps
  • Access to external tools (search, code execution, databases, shell commands)
  • Memory — short-term (conversation context) and long-term (vector stores, key-value caches)
  • Guardrails to stop it from looping forever or executing unsafe actions

    • Under the hood, most frameworks are orchestration engines. They don’t train models; they coordinate calls to models you already have API access to (OpenAI, Anthropic, local models via Ollama) and manage the state between calls.

    Core Components You’ll Configure

    Regardless of which framework you pick, you’ll end up configuring the same handful of building blocks:

  • Model provider — the LLM backend (hosted API or self-hosted via something like vLLM)
  • Tool registry — functions the agent is allowed to call, with strict input/output schemas
  • Memory backend — usually Redis, Postgres with pgvector, or a dedicated vector database
  • Orchestrator — the loop that decides what step runs next
  • Observability layer — logging, tracing, and cost tracking per run

    • If you’re already running a Docker-based stack, most of this maps cleanly onto services you can define in a single docker-compose.yml, which we’ll get to below.

    Popular Frameworks Compared

    Here’s a quick, opinionated breakdown of the frameworks developers actually reach for in 2026:

  • LangChain / LangGraph — the most widely adopted, huge ecosystem of integrations, but the API surface is large and changes often. LangGraph adds a proper state-machine model on top, which is a better fit for production than raw LangChain chains.
  • AutoGen (Microsoft) — strong for multi-agent conversations where several agents debate or collaborate on a task. Good defaults, heavier resource footprint.
  • CrewAI — opinionated “role-based” agent teams (researcher, writer, reviewer). Fast to prototype, less flexible for custom control flow.
  • Semantic Kernel — Microsoft’s alternative, popular in .NET shops, has solid plugin architecture.

    • None of these are drop-in replacements for each other — pick based on whether you need multi-agent collaboration (AutoGen, CrewAI) or a deterministic, debuggable state graph (LangGraph) for a single agent doing structured work.

    Choosing the Right AI Agent Framework for Your Stack

    Before committing to a framework, answer three questions:

    1. Does it need to run unattended? If yes, you need strong guardrails and a hard step limit — agents that loop indefinitely will burn through your API budget in hours.
    2. Does it call external tools that touch production systems? If yes, treat every tool call like an unauthenticated API request until proven otherwise.
    3. Where will state live? In-memory state disappears on container restart. If your agent needs to resume a task, you need a persistent backend from day one.

    Self-Hosted vs Managed

    You can run agent frameworks two ways:

  • Self-hosted, where you own the compute, the queue, and the logs. This is the right call if you’re processing sensitive data or need predictable costs.
  • Managed/serverless, where a platform runs the orchestration for you. Faster to start, but you lose visibility into retries and cost spikes.

    • For most teams already running Dockerized services, self-hosting on a VPS is the more economical option long-term. We cover the general setup in our Docker Compose production guide, which pairs well with the agent stack below.

    Deploying an AI Agent Framework with Docker

    Here’s a minimal but realistic docker-compose.yml for running a Python-based agent framework (LangGraph in this example) alongside Redis for short-term memory and Postgres with pgvector for long-term memory.

    version: "3.9"

services:
  agent:
    build: ./agent
    container_name: agent-runtime
    restart: unless-stopped
    env_file: .env
    depends_on:
      - redis
      - postgres
    ports:
      - "8080:8080"
    deploy:
      resources:
        limits:
          memory: 1g
          cpus: "1.0"

  redis:
    image: redis:7-alpine
    container_name: agent-redis
    restart: unless-stopped
    volumes:
      - redis_data:/data

  postgres:
    image: ankane/pgvector:latest
    container_name: agent-postgres
    restart: unless-stopped
    environment:
      POSTGRES_USER: agent
      POSTGRES_PASSWORD_FILE: /run/secrets/pg_password
      POSTGRES_DB: agent_memory
    secrets:
      - pg_password
    volumes:
      - pg_data:/var/lib/postgresql/data

volumes:
  redis_data:
  pg_data:

secrets:
  pg_password:
    file: ./secrets/pg_password.txt

    A minimal agent/Dockerfile:

    FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
CMD ["python", "main.py"]

    Notice the hard memory and CPU limits on the agent service. Agent loops that call an LLM in a retry chain can spike memory usage fast if a tool call hangs — cap it at the container level rather than trusting the framework’s internal timeouts.

    Environment Variables and Secrets

    Never hardcode API keys in your agent code. Use an .env file excluded from version control, or better, Docker secrets for anything touching a database:

    # .env
OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx
ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxxxxxxxxxx
REDIS_URL=redis://redis:6379/0
POSTGRES_URL=postgresql://agent:changeme@postgres:5432/agent_memory
MAX_AGENT_STEPS=15

    That MAX_AGENT_STEPS variable matters more than it looks — it’s your circuit breaker against runaway loops. Set it explicitly in your orchestrator’s config rather than relying on framework defaults, which are sometimes unbounded.

    A minimal Python entry point tying it together with LangGraph:

    import os
from langgraph.graph import StateGraph, END
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"])
MAX_STEPS = int(os.environ.get("MAX_AGENT_STEPS", 10))

def agent_step(state: dict) -> dict:
    state["steps"] = state.get("steps", 0) + 1
    if state["steps"] >= MAX_STEPS:
        state["done"] = True
        return state
    response = llm.invoke(state["messages"])
    state["messages"].append(response)
    state["done"] = "FINAL_ANSWER" in response.content
    return state

graph = StateGraph(dict)
graph.add_node("agent", agent_step)
graph.set_entry_point("agent")
graph.add_conditional_edges("agent", lambda s: END if s["done"] else "agent")
app = graph.compile()

    Run it with docker compose up -d, and check logs with docker compose logs -f agent.

    Monitoring and Scaling Your Agent Framework

    Once an agent is running unattended, you need visibility into three things: cost per run, failure rate, and latency. Most frameworks emit structured logs or OpenTelemetry traces — pipe those into a real monitoring stack rather than grepping container logs.

  • Track cost per agent run by logging token counts from each LLM response
  • Alert on step-count spikes, which usually indicate a broken tool call causing a retry loop
  • Set a hard timeout at the reverse proxy level, not just inside the agent

    • We’ve covered building a self-hosted logging pipeline in our self-hosted monitoring stack guide — the same Prometheus/Grafana pattern works well for agent metrics, and pairs nicely with an uptime checker like BetterStack if you want alerting without running your own Alertmanager.

    If your agent workload grows past a single VPS, horizontal scaling is straightforward since most frameworks are stateless between steps as long as memory lives in Redis/Postgres rather than in-process. Spin up additional agent containers behind a queue (Celery, RQ, or a simple Redis list) and let workers pull tasks independently.

    Picking Infrastructure for Agent Workloads

    Agent workloads are bursty — mostly idle, then a burst of CPU during tool execution and network calls during LLM requests. That profile fits well on cost-efficient VPS providers rather than committing to expensive dedicated GPU instances, since most of the heavy lifting (the LLM inference itself) happens on someone else’s API unless you’re self-hosting the model too.

  • Hetzner cloud instances are a solid, budget-friendly option for the orchestration layer
  • DigitalOcean droplets work well if you want managed Postgres/Redis add-ons alongside your agent containers
  • Both support Docker out of the box, and neither locks you into a proprietary orchestration layer

    • Security Considerations

    Agent frameworks introduce a new class of risk: prompt injection leading to unintended tool execution. If your agent has a “run shell command” or “call internal API” tool, an attacker who controls any text the model reads (a scraped web page, a user-submitted ticket) can potentially manipulate it into calling that tool maliciously.

    Sandboxing Tool Execution

  • Run any code-execution tool inside a disposable container, never on the host
  • Whitelist exact commands or API endpoints an agent can call — never pass raw shell strings to subprocess
  • Strip or sanitize any content pulled from external sources before it reaches the model’s context if it will influence tool selection
  • Log every tool call with its input and output for audit purposes

    • Treat your agent’s tool registry the same way you’d treat an API surface exposed to the public internet — because functionally, it often is one, just mediated by a language model instead of an HTTP router.

    Common Pitfalls When Running an AI Agent Framework in Production

    Teams that have shipped agent frameworks to production report the same handful of mistakes over and over. Watching for these early saves you from a painful debugging session at 2 a.m. when an agent has silently been retrying a failed API call for six hours.

  • No idempotency on tool calls. If a tool call fails partway through (say, a payment or a database write) and the agent retries, make sure the underlying operation is safe to run twice. Idempotency keys solve this cleanly.
  • Unbounded context growth. Long-running agents accumulate conversation history fast. Without a summarization or truncation strategy, you’ll blow past the model’s context window or pay for tokens you don’t need.
  • Trusting model output as structured data without validation. Even with function-calling APIs, always validate the JSON an agent returns against a schema before acting on it — models occasionally hallucinate a field name or type.
  • Skipping dry-run mode. Before letting an agent touch production data, run it against a staging environment with mocked tool responses to confirm the planning logic actually converges instead of looping.
  • No cost ceiling per user or per session. If your agent framework powers a customer-facing feature, cap spend per session so a single confused user (or bot) can’t run up thousands of API calls.

    • None of these require exotic tooling to fix — they’re mostly about applying the same discipline you’d already use for any external-facing production service.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    What’s the difference between an AI agent and a chatbot?
    A chatbot responds to messages. An agent plans multi-step tasks, calls tools or APIs, and keeps working toward a goal without a human prompting each step.

    Which AI agent framework is best for a solo developer project?
    CrewAI or a lightweight LangGraph setup are the fastest to get running solo — both have smaller learning curves than a full AutoGen multi-agent setup.

    Can I run an agent framework without sending data to OpenAI or Anthropic?
    Yes. Most frameworks support local models through Ollama or vLLM as a drop-in replacement for the hosted API, though you’ll trade some capability for privacy and cost control.

    How do I stop an agent from running up a huge API bill?
    Set a hard step limit and a per-run token budget in your orchestrator, and alert when either threshold is approached. Never deploy an agent loop without a circuit breaker.

    Do I need Kubernetes to run an AI agent framework in production?
    No. A single VPS with Docker Compose handles most workloads fine. Kubernetes only makes sense once you’re running many agent workers that need dynamic scaling.

    Is LangChain still worth learning in 2026?
    Yes, mostly through LangGraph, which is now the recommended way to build anything beyond a simple prompt chain in the LangChain ecosystem.

    Related articles: