How to Create AI Agents: A Practical Guide for Developers and Sysadmins

Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

If you’re a developer or sysadmin trying to figure out how to create AI agents that actually run in production — not just in a notebook demo — this guide is for you. We’ll skip the marketing fluff and walk through the real infrastructure decisions: containerization, orchestration, model serving, and deployment patterns that hold up under real traffic.

AI agents — autonomous or semi-autonomous programs that use large language models (LLMs) to reason, call tools, and complete multi-step tasks — have moved from research labs into everyday DevOps workflows. Teams use them for log triage, incident response, infrastructure provisioning, and customer support automation. But most tutorials stop at “pip install” and never address how to actually deploy and operate these agents reliably.

Why Create AI Agents Instead of Using a SaaS Tool?

Before you commit engineering time, it’s worth asking why you’d build a custom agent instead of subscribing to a hosted platform. Three reasons come up repeatedly:

Data residency and compliance. Self-hosting keeps sensitive logs, credentials, and customer data off third-party servers.

Cost control at scale. SaaS agent platforms often charge per-execution; a self-hosted agent on your own compute can be dramatically cheaper once volume grows.

Custom tool integration. Internal APIs, proprietary databases, and legacy systems rarely have first-class connectors in commercial agent platforms.

If none of those apply to your situation, a hosted product like OpenAI’s Assistants API might genuinely be the faster path. But if you need control over infrastructure, keep reading.

Core Components of an AI Agent Stack

Every production AI agent, regardless of framework, is built from the same basic layers:

LLM backend — the reasoning engine (hosted API like OpenAI/Anthropic, or a self-hosted model via Ollama)

Orchestration layer — the framework that manages prompts, memory, and tool calls (LangChain, LlamaIndex, CrewAI, or a custom loop)

Tool/function layer — the code the agent can actually execute (shell commands, API calls, database queries)

State/memory store — Redis, Postgres, or a vector database for conversation history and retrieval

Execution environment — usually a container, so the agent’s tool calls are sandboxed and reproducible

Getting the execution environment right is where most home-grown agent projects fall apart. An agent that can run arbitrary shell commands on your host is a security incident waiting to happen — which is exactly why containerization matters here, not just for portability but for isolation.

Setting Up the Environment

We’ll build a minimal but production-realistic agent using Python, the OpenAI API, and Docker for isolation. This pattern generalizes to Anthropic’s Claude API or a self-hosted model with minimal changes.

Step 1: Project Structure and Dependencies

Start with a clean project layout:

mkdir ai-agent-demo && cd ai-agent-demo
python3 -m venv venv
source venv/bin/activate
pip install openai python-dotenv requests

Create a requirements.txt so the environment is reproducible:

pip freeze > requirements.txt

Step 2: Write the Agent Loop

The core of an agent is a loop: the model reasons about the task, decides whether to call a tool, executes it, and feeds the result back in. Here’s a minimal but functional version:

# agent.py
import os
import json
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def get_disk_usage():
    import subprocess
    result = subprocess.run(["df", "-h", "/"], capture_output=True, text=True)
    return result.stdout

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_disk_usage",
            "description": "Return current disk usage for the root filesystem",
            "parameters": {"type": "object", "properties": {}},
        },
    }
]

def run_agent(user_prompt):
    messages = [{"role": "user", "content": user_prompt}]
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        tools=tools,
    )
    msg = response.choices[0].message

    if msg.tool_calls:
        for call in msg.tool_calls:
            if call.function.name == "get_disk_usage":
                result = get_disk_usage()
                messages.append(msg)
                messages.append({
                    "role": "tool",
                    "tool_call_id": call.id,
                    "content": result,
                })
        final = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
        return final.choices[0].message.content
    return msg.content

if __name__ == "__main__":
    print(run_agent("How much disk space is free on this server?"))

This is intentionally simple: one tool, one loop. Real agents chain multiple tool calls, but the pattern is the same — model decides, code executes, result feeds back.

Step 3: Containerize the Agent

Running tool calls directly on your host is risky. Wrap the agent in Docker so its filesystem access, network access, and process list are isolated:

FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY agent.py .

ENV OPENAI_API_KEY=""
CMD ["python", "agent.py"]

Build and run it with the API key injected at runtime, never baked into the image:

docker build -t ai-agent-demo .
docker run --rm -e OPENAI_API_KEY=$OPENAI_API_KEY ai-agent-demo

For agents that need to run shell commands as a genuine capability, add resource limits and a read-only root filesystem where possible:

docker run --rm 
  --memory=512m --cpus=1 
  --read-only --tmpfs /tmp 
  -e OPENAI_API_KEY=$OPENAI_API_KEY 
  ai-agent-demo

This single change — running the agent as a constrained container instead of a bare process — closes off most of the accidental-damage scenarios that make people nervous about giving an LLM shell access in the first place.

Choosing a Framework vs. Rolling Your Own

Once the basic loop works, you’ll want to decide whether to adopt a framework. If you’re new to this, our guide on setting up a self-hosted media server covers a similar build-vs-buy tradeoff for infrastructure decisions, and the same logic applies here.

LangChain — the most widely adopted, huge ecosystem of integrations, but can feel heavy for simple use cases.

CrewAI — built specifically for multi-agent collaboration (agents that delegate to other agents).

LlamaIndex — strongest when your agent’s primary job is retrieval over documents.

Custom loop — as shown above; best when you have one or two well-defined tools and want full control over latency and cost.

For teams running dozens of agents in production, the framework choice matters less than the operational discipline around it: logging every tool call, rate-limiting API usage, and having a kill switch.

Deploying Agents to a VPS

Once your container works locally, deploying it to a VPS is straightforward with Docker Compose. A reasonable production setup includes a reverse proxy, environment-based secrets, and log persistence:

# docker-compose.yml
version: "3.9"
services:
  ai-agent:
    build: .
    restart: unless-stopped
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    mem_limit: 512m
    cpus: 1.0
    volumes:
      - ./logs:/app/logs

Deploy with:

docker compose up -d
docker compose logs -f ai-agent

For the underlying compute, a small VPS is usually enough for a single agent handling moderate traffic — you don’t need GPU infrastructure unless you’re self-hosting the LLM itself. Providers like DigitalOcean offer droplets that are well suited to this kind of lightweight, containerized workload, with predictable pricing and a straightforward API for scaling up if your agent’s usage grows. If you want lower base costs for a always-on agent process, Hetzner is worth comparing for the same workload.

We cover general container orchestration patterns in more depth in our Docker Compose production guide, which is directly applicable once you’re running more than one agent.

Monitoring and Observability

Agents fail differently than normal applications — they can loop indefinitely, call tools with bad arguments, or silently degrade in output quality without ever throwing an exception. Treat observability as mandatory, not optional:

Log every prompt, tool call, and response with timestamps and token counts.

Set hard timeouts on the agent loop to prevent runaway tool-call chains.

Track API cost per agent run — a bug in your loop can turn into a large bill overnight.

Alert on error rates and unusual latency spikes.

A service like BetterStack can centralize your agent’s logs and uptime monitoring in one dashboard, which is far easier to reason about than grepping container logs during an incident. If your agent is exposed via a public API endpoint, putting it behind Cloudflare adds a layer of DDoS protection and rate limiting without much configuration overhead.

Security Considerations When You Create AI Agents

Security deserves its own section because it’s the part most tutorials skip entirely.

Never give an agent raw shell access without a sandbox. Use containers, restricted user accounts, and allowlisted commands.

Validate tool arguments before execution. An LLM can hallucinate a destructive command; your code, not the model, is the last line of defense.

Rotate and scope API keys. Use a key with the minimum permissions the agent actually needs.

Rate-limit tool calls that touch external APIs or billing-sensitive services.

Log everything. If an agent does something unexpected, you need an audit trail to reconstruct what happened.

These aren’t theoretical concerns — there have been real incidents of agents deleting production data or leaking credentials because a developer trusted model output without a validation layer in between.

Scaling Beyond a Single Agent

Once you’re comfortable with one agent, the natural next step is orchestrating several agents that specialize in different tasks — one for monitoring, one for deployment, one for customer-facing chat. At that point you’re effectively running a small distributed system, and the same DevOps fundamentals apply: health checks, graceful restarts, and centralized logging. Docker Compose can handle a handful of agents; beyond that, Kubernetes or Nomad becomes worth the added complexity.

A practical migration path looks like this:

1. Validate the agent loop locally with a single container.
2. Add resource limits and logging, deploy via Docker Compose on a VPS.
3. Introduce a message queue (Redis or RabbitMQ) if agents need to communicate.
4. Move to Kubernetes only once you have more than 4-5 distinct agent services.

Skipping straight to Kubernetes for a single agent is a common and avoidable mistake — it adds operational overhead without solving a problem you have yet.

Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

FAQ

Q: What’s the fastest way to create AI agents without writing custom code?
A: Frameworks like CrewAI or LangChain’s agent executors let you define tools and prompts declaratively, cutting setup time significantly compared to a fully custom loop. They trade some control for speed of development.

Q: Do I need a GPU to run an AI agent?
A: No, not if you’re calling a hosted LLM API (OpenAI, Anthropic, etc.). You only need GPU infrastructure if you’re self-hosting the language model itself, such as running Llama or Mistral locally via Ollama.

Q: How do I prevent an AI agent from running dangerous commands?
A: Sandbox execution in a container with resource limits, validate all tool arguments in code before running them, and use an allowlist of permitted commands rather than trusting free-form model output.

Q: What’s the difference between an AI agent and a chatbot?
A: A chatbot responds conversationally; an agent takes autonomous action by calling tools, executing code, or interacting with external systems to complete a task, often across multiple steps without human intervention at each one.

Q: How much does it cost to run a self-hosted AI agent?
A: Compute costs are usually minimal (a small VPS suffices for most workloads), but LLM API costs scale with usage. Budget for token costs per request and set hard caps to avoid runaway spending from looping agents.

Q: Can I run multiple AI agents on the same server?
A: Yes, using Docker Compose to isolate each agent as its own container with defined memory and CPU limits. For more than a handful of agents, consider moving to Kubernetes for better orchestration and scaling.

Wrapping Up

Learning to create AI agents that survive contact with production traffic comes down to treating them like any other service: containerize them, monitor them, limit their blast radius, and log everything. The LLM is just one component in a larger system — the DevOps fundamentals around it are what determine whether your agent is a reliable tool or a 3am incident.

Start small: one container, one tool, hard resource limits. Expand only when you have a concrete reason to.