That doesn’t mean frameworks are bad — for many teams they’re the right tradeoff. But if you’re a DevOps engineer who wants to understand exactly what’s happening between a user prompt and an API call, building the loop yourself is the fastest way to actually learn how these systems work, and it gives you a codebase you fully own.

The Core Agent Loop

At its simplest, an agent is a loop: observe the current state, ask a model what to do next, execute that action, and feed the result back in. When you build ai agents from scratch, this loop is the one piece of code you’ll spend the most time refining.

def agent_loop(goal, tools, model_client, max_steps=10):
    history = [{"role": "user", "content": goal}]
    for step in range(max_steps):
        response = model_client.chat(history, tools=tools)
        if response.tool_call is None:
            return response.content  # final answer
        result = execute_tool(response.tool_call, tools)
        history.append({"role": "assistant", "content": response.tool_call})
        history.append({"role": "tool", "content": result})
    return "Max steps reached without a final answer."

This is intentionally minimal. No retries, no memory compression, no parallel tool calls — just the skeleton. Everything else in this article builds on top of it.

Choosing a Model Provider

You don’t need to commit to one provider forever, but you do need a consistent interface. Most people who build ai agents from scratch start with the OpenAI API or Anthropic’s API because both have mature tool-calling support and clear documentation. Whichever you pick, isolate the provider-specific code behind a thin client class so you can swap models without rewriting your loop.

Designing the Tool Layer

Tools are what separate an agent from a chatbot. A tool is any function the model can request to be executed — searching the web, querying a database, hitting an internal API, or writing a file. When you build ai agents from scratch, the tool layer is where most of the real engineering effort goes, not the prompt.

Each tool needs three things: a machine-readable schema (so the model knows it exists and what arguments it takes), an execution function, and error handling that returns something useful back into the loop rather than crashing it.

TOOLS = [
    {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    }
]

def execute_tool(tool_call, tools):
    try:
        if tool_call.name == "get_weather":
            return fetch_weather(tool_call.arguments["city"])
        return f"Unknown tool: {tool_call.name}"
    except Exception as e:
        return f"Tool error: {str(e)}"

Guardrails on Tool Execution

Never let an agent call a tool with unbounded permissions. If a tool can write to a filesystem, delete records, or spend money, put an explicit allowlist and a confirmation step in front of it. A reasonable pattern is to classify each tool as read-only or mutating, and require a human-in-the-loop confirmation for anything mutating until you’ve built enough confidence in the agent’s behavior to relax that.

Rate Limiting and Timeouts

An agent stuck in a loop calling a slow external API can silently rack up cost and latency. Wrap every tool call with a timeout, and cap the number of tool calls per session. This is a small amount of code that prevents most of the runaway-cost incidents teams run into once an agent goes from a demo to something users actually invoke repeatedly.

Managing State and Memory

A single-session agent loop is easy. The harder problem when you build ai agents from scratch is state that persists across sessions — remembering what a user asked yesterday, or tracking the status of a long-running multi-step task.

For most use cases you don’t need a vector database on day one. A simple relational table tracking session_id, role, content, and timestamp is enough to reconstruct conversation history. Only reach for embeddings and semantic search once you have a concrete need to retrieve relevant history that a simple recency window can’t satisfy.

Trimming Context Without Losing Intent

As conversations grow, you’ll exceed the model’s context window or just waste tokens on irrelevant history. A common approach is to keep the last N full turns verbatim and summarize everything older into a single condensed block, re-injected at the top of the history. This keeps token usage predictable while preserving enough context for the agent to stay coherent across a long session.

Build AI Agents From Scratch: Deployment With Docker

Once the loop and tool layer work locally, the next step is packaging the agent so it runs reliably outside your laptop. This is the part of “build ai agents from scratch” that most tutorials skip entirely, and it’s where a DevOps background actually pays off.

A minimal agent service needs: the agent process itself, a way to receive triggers (webhook, queue, or scheduler), and persistent storage for state. Docker Compose is a reasonable starting point for a single-host deployment before you need anything like Kubernetes.

version: "3.9"
services:
  agent:
    build: .
    restart: unless-stopped
    environment:
      - MODEL_API_KEY=${MODEL_API_KEY}
      - DATABASE_URL=postgres://agent:agent@db:5432/agent
    depends_on:
      - db
    ports:
      - "8080:8080"
  db:
    image: postgres:16
    restart: unless-stopped
    environment:
      - POSTGRES_USER=agent
      - POSTGRES_PASSWORD=agent
      - POSTGRES_DB=agent
    volumes:
      - agent_pgdata:/var/lib/postgresql/data
volumes:
  agent_pgdata:

If you’re new to the Compose file format itself, the Postgres Docker Compose setup guide and the Docker Compose Env guide cover the surrounding details — secrets handling, environment variable precedence, and volume persistence — that this snippet assumes you already know.

Running the Agent as a Long-Lived Service

If your agent needs to poll a queue or run on a schedule rather than respond to a single webhook, wrap the loop in a service with a clear retry policy and restart behavior. Docker’s restart: unless-stopped covers crash recovery, but you still need your own logic for “what happens if the model API is down for five minutes” — usually an exponential backoff with a cap, rather than a tight retry loop that burns through your rate limit.

For teams that don’t want to write this orchestration layer themselves, it’s worth comparing against a visual workflow tool. The guide on how to build AI agents with n8n shows the same core loop implemented as a workflow instead of raw code — useful context even if you ultimately decide to build ai agents from scratch in your own codebase, since it clarifies which parts of the problem are genuinely hard versus just boilerplate.

Hosting Considerations

Wherever you run this, you want predictable CPU and a static IP for outbound webhook calls. A small VPS is enough for most single-agent workloads — you don’t need a Kubernetes cluster until you’re running many agents concurrently or need horizontal autoscaling. Providers like DigitalOcean offer straightforward droplet sizing if you want to start with a single box before deciding whether to scale out.

Observability and Debugging

An agent that fails silently is worse than one that crashes loudly. Every production agent needs structured logging of the full decision trace: the prompt sent, the tool selected, the arguments, the result, and the final output. Without this, debugging a bad decision means guessing.

Log each step as a structured event rather than free text, so you can query it later:

echo '{"session_id":"abc123","step":2,"tool":"get_weather","args":{"city":"Berlin"},"result":"12C, cloudy"}' 
  | tee -a agent_trace.jsonl

A JSONL trace file like this is a low-effort starting point. As the system matures, ship these events to a real log aggregator so you can search and alert on patterns like repeated tool failures or sessions that hit max_steps without resolving.

Testing the Agent Loop

Unit-test the tool functions in isolation — they’re regular functions, so this is straightforward. For the loop itself, write scenario tests that feed a scripted sequence of model responses (mocked, not live API calls) and assert the loop takes the expected path. This catches regressions in your control flow without burning API credits on every test run, and it’s the same testing discipline you’d apply to any other stateful service.

Common Pitfalls When You Build AI Agents From Scratch

A few mistakes show up repeatedly in early agent implementations:

Keeping these in mind from the start saves a rewrite later. Most of them are standard software engineering discipline — logging, validation, idempotency — applied to a system where one of the callers happens to be a language model instead of a human.

Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

Choosing a Memory Strategy

An agent without memory re-derives everything from scratch on every call, which is expensive and limits what it can do across a multi-turn conversation. There are two broad memory tiers worth separating:

Persisting State With Redis or Postgres

For most self-hosted setups, Redis is a good fit for short-term session state because it’s fast and simple to run alongside your agent process. If you’re already running a Docker Compose stack, our Redis Docker Compose setup guide covers the exact service definition you’ll need. For long-term memory that needs relational queries or joins against other application data, Postgres is a solid default — see the Postgres Docker Compose guide for a working configuration.

A minimal docker-compose.yml combining both, alongside the agent service itself, looks like this:

services:
  agent:
    build: .
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - REDIS_URL=redis://redis:6379
      - DATABASE_URL=postgresql://agent:agent@postgres:5432/agent
    depends_on:
      - redis
      - postgres

  redis:
    image: redis:7-alpine
    restart: unless-stopped

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      - POSTGRES_USER=agent
      - POSTGRES_PASSWORD=agent
      - POSTGRES_DB=agent
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

If you need to manage secrets like your OPENAI_API_KEY more carefully than a plain .env file allows, our Docker Compose secrets guide walks through the options.

Deployment and Infrastructure Choices

Once the agent logic works locally, you need somewhere reliable to run it. This is where a lot of “from scratch” projects stall — not because the agent code is wrong, but because the deployment story was never planned.

Picking a VPS and Sizing It Correctly

An LLM-calling agent is usually not CPU-bound (the model runs remotely via API), so you don’t need a large instance unless you’re also running local embeddings or a vector database with a heavy index. A small, unmanaged VPS is often enough for the agent process itself. DigitalOcean and Vultr both offer straightforward droplet/instance sizing if you want a starting point, and our unmanaged VPS hosting guide covers what “unmanaged” actually means for day-to-day maintenance.

If you expect to scale the agent horizontally later (multiple workers pulling from a shared task queue), it’s worth sizing for that from day one rather than re-architecting under load.

Logging, Debugging, and Observability

Agent behavior is nondeterministic by nature — the same input can produce a different tool-call sequence on different runs. That makes logging non-negotiable. At minimum, log every model call’s input, the chosen tool, the tool’s result, and the final response. If you’re running the agent inside Docker Compose, the Docker Compose logs debugging guide and the related Docker Compose logging guide cover how to keep those logs queryable without them growing unbounded.

FAQ

Do I need a framework to build AI agents from scratch?
No. A framework can save time on boilerplate, but the core loop — call model, execute tool, feed result back — is a few dozen lines of code. Many teams choose to build ai agents from scratch specifically to avoid framework lock-in and keep the reasoning loop fully inspectable.

How many tools should a single agent have?
Keep it focused. An agent with a large, unfocused tool list tends to make worse tool-selection decisions than one with a small, well-described set. Start with the minimum set needed for the task and add tools only when there’s a clear gap.

How do I prevent an agent from taking destructive actions?
Classify tools as read-only or mutating, and require explicit confirmation or a dry-run step before any mutating tool executes. Log every tool call so you can audit what happened after the fact.

Should I run the agent on a VPS or a serverless platform?
Either works. A VPS with Docker Compose is simpler to reason about and debug for a single agent or small fleet. Serverless makes more sense once you have bursty, infrequent invocations where paying for an always-on container doesn’t make sense.

Conclusion

Learning to build ai agents from scratch means understanding the reasoning loop, the tool execution layer, state management, and the deployment and observability work that turns a demo into a system you can trust. None of these pieces are individually complex, but skipping any of them — especially guardrails on tool execution and structured logging — is what turns an agent from a useful automation into a production incident. Start with the minimal loop shown here, add tools deliberately, and invest in tracing early; it’s far easier to build that discipline in from the start than to retrofit it once the agent is already handling real traffic. For deeper reference on the model side, the Anthropic API documentation and OpenAI API documentation are both good starting points once your own loop is solid enough to plug either provider into.