AI Agents Software: A DevOps Guide to Self-Hosting in Production
Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.
AI agents software has moved from research demos to production workloads fast. Teams are wiring up autonomous agents to trigger deployments, monitor logs, answer support tickets, and orchestrate multi-step workflows without a human clicking through every step. The problem is that most tutorials assume you’ll just point at a hosted API and call it a day. That’s fine until you need predictable latency, data residency, cost control, or the ability to run agents against your own internal systems without shipping credentials to a third party.
What Counts as AI Agents Software
An AI agent, in the DevOps sense, is a process that combines a language model with tool-calling capability and a loop: observe state, decide an action, execute it, observe the result, repeat. Unlike a chatbot that answers one prompt and stops, an agent can call shell commands, query APIs, read and write files, or trigger CI/CD pipelines on its own.
Common frameworks you’ll run into:
All of these need a runtime, a place to store state (vector DB, Redis, Postgres), and network access to whatever tools the agent is allowed to call. That’s infrastructure work, not just prompt engineering.
Why Self-Host Instead of Using a SaaS Agent Platform
Hosted agent platforms are convenient, but they come with tradeoffs that matter once you’re running anything beyond a demo:
If you already run Docker workloads and manage your own VPS fleet, self-hosting agent software is a natural extension of your existing stack rather than a new discipline.
Building the Agent Runtime
Containerizing an AI Agent with Docker
Treat your agent the same way you’d treat any long-running service — package it as a container with a clear entrypoint, health check, and restart policy. Here’s a minimal example using a Python agent built on LangChain:
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
ENV PYTHONUNBUFFERED=1
HEALTHCHECK --interval=30s --timeout=5s
CMD python healthcheck.py || exit 1
CMD ["python", "agent_runner.py"]A basic requirements.txt:
langchain==0.3.7
langchain-openai==0.2.9
redis==5.2.0
fastapi==0.115.5
uvicorn==0.32.1And a stripped-down runner that exposes the agent behind an HTTP endpoint so it fits cleanly into a reverse-proxy setup:
# agent_runner.py
from fastapi import FastAPI, Request
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
import uvicorn
app = FastAPI()
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
@app.post("/run")
async def run_agent(request: Request):
payload = await request.json()
task = payload.get("task")
result = llm.invoke(task)
return {"output": result.content}
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8000)Wrap it in docker-compose.yml alongside Redis for state and a Postgres instance for persistent logs:
version: "3.9"
services:
agent:
build: .
restart: unless-stopped
ports:
- "8000:8000"
env_file: .env
depends_on:
- redis
- db
redis:
image: redis:7-alpine
restart: unless-stopped
db:
image: postgres:16-alpine
restart: unless-stopped
environment:
POSTGRES_USER: agent
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_DB: agent_logs
volumes:
- pgdata:/var/lib/postgresql/data
volumes:
pgdata:This pattern — one container per concern, restart policies set, secrets pulled from .env rather than hardcoded — is the same discipline covered in our Docker Compose production guide, and it applies directly to agent workloads.
Choosing Infrastructure: VPS vs Managed Kubernetes
For most teams running one to a handful of agents, a single well-sized VPS is enough — you don’t need Kubernetes to run a container with a health check and a restart policy. A 4 vCPU / 8GB droplet handles a moderate agent workload comfortably, especially since the heavy compute (the LLM inference itself) usually happens on a hosted model API rather than locally.
If you’re evaluating providers, DigitalOcean droplets are a solid starting point for agent workloads that need predictable pricing and fast provisioning, while Hetzner is worth comparing if you’re optimizing hard for cost-per-core on longer-running background agents. Both work fine with the Docker Compose setup above — no orchestration platform required until you’re running dozens of agent instances concurrently.
Once you outgrow a single box, look at our scaling Docker workloads across multiple hosts writeup before jumping straight to Kubernetes — a lot of agent workloads scale fine horizontally with a load balancer and a shared Redis/Postgres backend.
Securing and Monitoring Your AI Agent Stack
Agents that can execute shell commands or hit internal APIs are a bigger attack surface than a typical web app. Treat the agent’s tool-calling permissions the same way you’d treat a service account: narrow scope, explicit allowlists, no blanket sudo.
A few concrete steps:
/run endpoint so a runaway agent loop can’t hammer your own infrastructure or burn through API creditsFor uptime and alerting, hook the health check endpoint into BetterStack so you get paged if the agent process dies or starts returning errors — the same way you’d monitor any other production API.
# quick manual check before wiring up automated monitoring
curl -sf http://localhost:8000/run
-X POST -H "Content-Type: application/json"
-d '{"task": "ping"}' | jq .Cost and Scaling Considerations
Token costs dominate agent operating expense, not compute. A well-optimized agent loop that limits unnecessary reasoning steps and caches repeated tool calls in Redis can cut API spend by 30-50% compared to a naive implementation that re-queries the model on every iteration. Before scaling out infrastructure, profile where your token spend is actually going — it’s usually cheaper to fix a chatty prompt loop than to add more servers.
When you do need to scale horizontally, put a load balancer in front of multiple agent container replicas and share state through Redis rather than in-memory — this keeps any single container disposable, which matters when you’re doing rolling deploys of new agent logic.
Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.
FAQ
What’s the difference between an AI agent and a simple chatbot API call?
A chatbot responds to one prompt and stops. An agent runs a loop — it can call tools, evaluate the result, and decide on a follow-up action without a human in between each step. That loop is what needs the extra infrastructure (state storage, tool permissions, monitoring).
Do I need a GPU to self-host AI agents software?
Usually not. Most production agent setups call a hosted model API (OpenAI, Anthropic, etc.) for inference and only self-host the orchestration layer — the container that manages the loop, tool calls, and state. You only need local GPU compute if you’re also self-hosting the language model itself, which is a separate and much heavier undertaking.
Is Docker Compose enough, or do I need Kubernetes?
For a handful of agents, Docker Compose on a single VPS is enough and far simpler to operate. Move to Kubernetes only once you’re running many agent replicas that need automated scheduling, scaling, and failover — most teams overestimate how soon they’ll need it.
How do I stop an agent from running away and burning API credits?
Set hard iteration limits in your agent loop, add request timeouts, and rate-limit the endpoint that triggers agent runs. Also log token usage per run so you can spot runaway loops in monitoring before they show up on a bill.
Can I run open-source models instead of a hosted API?
Yes — tools like Ollama let you run open-weight models locally, but expect a real hit to reasoning quality on complex multi-step tasks compared to frontier hosted models. It’s a reasonable tradeoff for cost-sensitive or data-sensitive workloads, less so if you need reliable complex tool-use chains.
Where should agent logs and state live?
Use Postgres or a similar durable store for conversation/run history you need to audit later, and Redis for short-lived state like in-progress task context. Don’t rely on container-local disk — it disappears on redeploy.
Wrapping Up
Running AI agents software in production isn’t fundamentally different from running any other containerized service — it just adds a few new failure modes around tool permissions, runaway loops, and token cost. Start with a single Docker container behind a reverse proxy, wire up basic health checks and logging, and only add complexity (multi-host scaling, Kubernetes, custom model hosting) once you actually hit the limits of the simple setup. The teams that get burned are the ones that skip monitoring and rate limits because “it’s just a demo” — and then it isn’t.
Leave a Reply