How to Build AI Agent From Scratch: A DevOps Guide

Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

If you want to build AI agent from scratch rather than wiring together a no-code platform, you need to understand the moving parts: an LLM as the reasoning engine, a tool-execution layer, a state/memory store, and a deployment target that keeps the whole thing running reliably. This guide walks through the architecture, the code you actually need, and the infrastructure decisions that separate a weekend prototype from something you can run in production.

Plenty of teams reach for a drag-and-drop builder first, and that’s a reasonable starting point for prototyping — see our comparison of how to build AI agents with n8n if you want that route. But once you need custom control flow, non-standard tool integrations, or tight latency budgets, you’ll want to build ai agent from scratch using plain code and a few well-chosen libraries. This article covers that path end-to-end.

Why Build AI Agent From Scratch Instead of Using a Framework

There’s no shortage of agent frameworks now, and many of them are genuinely useful. So why would you choose to build ai agent from scratch instead of adopting one wholesale?

Full control over the reasoning loop. Frameworks abstract away the exact prompt structure, retry logic, and tool-call parsing. When something misbehaves in production, debugging an abstraction you don’t own is harder than debugging your own code.

Smaller dependency surface. Fewer transitive dependencies means fewer surprise breaking changes and a smaller attack surface to audit.

Easier to reason about cost and latency. You know exactly how many tokens each step consumes because you wrote the loop.

No lock-in. Swapping model providers or hosting environments is simpler when your agent logic isn’t tightly coupled to a specific SDK’s internals.

None of this means frameworks are bad — for many teams, a framework is the right tradeoff. But if your use case is narrow, well-defined, or has strict latency/cost constraints, building from scratch is often the more maintainable long-term choice. Our related guide on building AI agents covers the same tradeoff from a slightly different angle if you want a second opinion before committing.

Core Components You’ll Need

Whatever framework decision you make, the underlying components are the same:

1. An LLM client (via an API or a self-hosted model server)
2. A tool/function registry with clear input/output contracts
3. A loop that alternates between “ask the model what to do” and “execute the chosen tool”
4. A memory or state store (short-term conversation history, long-term vector or relational storage)
5. A deployment and observability layer

Designing the Agent Loop

The heart of any agent is the loop: the model receives context, decides on an action, that action executes, and the result feeds back into the next model call. This is sometimes called the ReAct pattern (reason + act), though you don’t need to use that exact name in your code — the concept is simple enough to implement directly.

The Minimal Reasoning Loop

Here’s a minimal, dependency-light version in Python using an OpenAI-compatible chat completions API. It’s intentionally small so you can see every decision point:

import json
import openai

client = openai.OpenAI()

TOOLS = {
    "get_weather": lambda city: f"Weather data for {city} is unavailable in this demo.",
    "add_numbers": lambda a, b: a + b,
}

TOOL_SCHEMA = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    },
]

def run_agent(user_message, max_steps=5):
    messages = [{"role": "user", "content": user_message}]
    for _ in range(max_steps):
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
            tools=TOOL_SCHEMA,
        )
        choice = response.choices[0].message
        if not choice.tool_calls:
            return choice.content
        messages.append(choice)
        for call in choice.tool_calls:
            fn = TOOLS[call.function.name]
            args = json.loads(call.function.arguments)
            result = fn(**args)
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result),
            })
    return "Max steps reached without a final answer."

This loop is deliberately bare-bones: no retries, no streaming, no persistent memory. That’s the point — start with something you fully understand, then layer complexity on top only where you need it. For the exact reference on function/tool calling parameters, check the OpenAI API documentation.

Handling Tool Failures and Retries

Real tools fail — an API times out, a database connection drops, a rate limit kicks in. When you build ai agent from scratch, you own this failure handling directly instead of trusting a framework’s defaults:

def call_tool_safely(fn, args, retries=2):
    for attempt in range(retries + 1):
        try:
            return fn(**args)
        except Exception as exc:
            if attempt == retries:
                return f"Tool failed after {retries} retries: {exc}"

Feed tool failures back into the conversation as plain text rather than swallowing them — the model can often recover gracefully if it knows a step didn’t work, for example by trying a different tool or asking the user for clarification.

Choosing a Memory Strategy

An agent without memory re-derives everything from scratch on every call, which is expensive and limits what it can do across a multi-turn conversation. There are two broad memory tiers worth separating:

Short-term (conversation) memory — the running message history for the current session. Simplest option: keep it in a list in application memory or a Redis key, trimmed or summarized once it exceeds a token budget.

Long-term (knowledge) memory — facts, documents, or past interactions the agent should recall across sessions. Typically backed by a vector database or a relational store with full-text search.

Persisting State With Redis or Postgres

For most self-hosted setups, Redis is a good fit for short-term session state because it’s fast and simple to run alongside your agent process. If you’re already running a Docker Compose stack, our Redis Docker Compose setup guide covers the exact service definition you’ll need. For long-term memory that needs relational queries or joins against other application data, Postgres is a solid default — see the Postgres Docker Compose guide for a working configuration.

A minimal docker-compose.yml combining both, alongside the agent service itself, looks like this:

services:
  agent:
    build: .
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - REDIS_URL=redis://redis:6379
      - DATABASE_URL=postgresql://agent:agent@postgres:5432/agent
    depends_on:
      - redis
      - postgres

  redis:
    image: redis:7-alpine
    restart: unless-stopped

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      - POSTGRES_USER=agent
      - POSTGRES_PASSWORD=agent
      - POSTGRES_DB=agent
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

If you need to manage secrets like your OPENAI_API_KEY more carefully than a plain .env file allows, our Docker Compose secrets guide walks through the options.

Deployment and Infrastructure Choices

Once the agent logic works locally, you need somewhere reliable to run it. This is where a lot of “from scratch” projects stall — not because the agent code is wrong, but because the deployment story was never planned.

Picking a VPS and Sizing It Correctly

An LLM-calling agent is usually not CPU-bound (the model runs remotely via API), so you don’t need a large instance unless you’re also running local embeddings or a vector database with a heavy index. A small, unmanaged VPS is often enough for the agent process itself. DigitalOcean and Vultr both offer straightforward droplet/instance sizing if you want a starting point, and our unmanaged VPS hosting guide covers what “unmanaged” actually means for day-to-day maintenance.

If you expect to scale the agent horizontally later (multiple workers pulling from a shared task queue), it’s worth sizing for that from day one rather than re-architecting under load.

Logging, Debugging, and Observability

Agent behavior is nondeterministic by nature — the same input can produce a different tool-call sequence on different runs. That makes logging non-negotiable. At minimum, log every model call’s input, the chosen tool, the tool’s result, and the final response. If you’re running the agent inside Docker Compose, the Docker Compose logs debugging guide and the related Docker Compose logging guide cover how to keep those logs queryable without them growing unbounded.

Log structured JSON, not free text, so you can filter by session ID or tool name later

Set a hard step limit on the agent loop (as shown above) to prevent runaway token spend from an infinite tool-calling cycle

Capture latency per step so you can tell whether slowness comes from the model call or your own tool code

Testing and Iterating on Agent Behavior

Unlike traditional software, you can’t fully unit-test an LLM’s decisions — but you can and should test everything around it. Write deterministic unit tests for your tool functions (they’re just regular code), and use a small, fixed set of example prompts as a regression suite you re-run whenever you change the system prompt or tool schema. Track whether the agent still picks the right tool for each example after changes — a regression here is easy to miss without a repeatable check.

Kubernetes-based deployments add real operational overhead for something this small; unless you already run a cluster for other services, a single Compose stack is usually the simpler starting point. Our comparison of Kubernetes vs Docker Compose walks through when the added complexity actually pays off.

Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

FAQ

Do I need a framework at all to build AI agent from scratch?
No. The core loop — call the model, execute the returned tool call, feed the result back — is only a few dozen lines of code, as shown above. Frameworks add convenience (built-in retries, memory abstractions, multi-agent orchestration) but aren’t required to get a working agent running.

Which LLM provider should I use for a from-scratch agent?
Any provider with a stable function/tool-calling API works. Check current pricing and rate limits before committing — our OpenAI API pricing guide is a useful reference point if you’re comparing costs across providers.

How do I prevent an agent from calling tools in an infinite loop?
Enforce a hard maximum number of loop iterations (as in the max_steps parameter above), and consider adding a token or cost budget per session that terminates the loop early if exceeded.

Is it safe to let an agent execute arbitrary tools without review?
Not by default. Any tool that mutates state (writing to a database, sending an email, calling a paid API) should have its inputs validated before execution, and destructive actions should require an explicit confirmation step rather than running automatically on the model’s first suggestion.

Conclusion

To build ai agent from scratch, you don’t need a large stack — you need a clear reasoning loop, a tool registry with strict input/output contracts, a deliberate memory strategy, and infrastructure that’s sized for what the agent actually does (usually light, since the model runs remotely). Start with the minimal loop shown here, add logging before you add features, and only reach for a framework once you’ve hit a concrete limitation that justifies the extra dependency. For further reading on the official model APIs referenced throughout this guide, see the Python official documentation for the language-level tooling used in these examples.