How to Build AI Agents From Scratch: A DevOps Deployment Guide
Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.
If you want to build AI agents from scratch instead of relying on a closed-source framework, you need to understand three things: how an agent reasons about the next action, how it calls tools, and how you actually deploy and monitor that process in production. This guide walks through the architecture, the code, and the operational side of the problem — the parts most tutorials skip once the demo works on a laptop.
Most articles about agents stop at “call an LLM in a loop.” That’s a fine starting point, but it isn’t a system you can run reliably. Below we cover the core loop, tool execution, state management, deployment with Docker, and the observability you need once the agent is doing real work against real APIs.
Why Build AI Agents From Scratch Instead of Using a Framework
Frameworks like LangChain or CrewAI are useful when you want to move fast and don’t mind the abstraction overhead. But there are good reasons teams choose to build ai agents from scratch instead:
That doesn’t mean frameworks are bad — for many teams they’re the right tradeoff. But if you’re a DevOps engineer who wants to understand exactly what’s happening between a user prompt and an API call, building the loop yourself is the fastest way to actually learn how these systems work, and it gives you a codebase you fully own.
The Core Agent Loop
At its simplest, an agent is a loop: observe the current state, ask a model what to do next, execute that action, and feed the result back in. When you build ai agents from scratch, this loop is the one piece of code you’ll spend the most time refining.
def agent_loop(goal, tools, model_client, max_steps=10):
history = [{"role": "user", "content": goal}]
for step in range(max_steps):
response = model_client.chat(history, tools=tools)
if response.tool_call is None:
return response.content # final answer
result = execute_tool(response.tool_call, tools)
history.append({"role": "assistant", "content": response.tool_call})
history.append({"role": "tool", "content": result})
return "Max steps reached without a final answer."This is intentionally minimal. No retries, no memory compression, no parallel tool calls — just the skeleton. Everything else in this article builds on top of it.
Choosing a Model Provider
You don’t need to commit to one provider forever, but you do need a consistent interface. Most people who build ai agents from scratch start with the OpenAI API or Anthropic’s API because both have mature tool-calling support and clear documentation. Whichever you pick, isolate the provider-specific code behind a thin client class so you can swap models without rewriting your loop.
Designing the Tool Layer
Tools are what separate an agent from a chatbot. A tool is any function the model can request to be executed — searching the web, querying a database, hitting an internal API, or writing a file. When you build ai agents from scratch, the tool layer is where most of the real engineering effort goes, not the prompt.
Each tool needs three things: a machine-readable schema (so the model knows it exists and what arguments it takes), an execution function, and error handling that returns something useful back into the loop rather than crashing it.
TOOLS = [
{
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
}
]
def execute_tool(tool_call, tools):
try:
if tool_call.name == "get_weather":
return fetch_weather(tool_call.arguments["city"])
return f"Unknown tool: {tool_call.name}"
except Exception as e:
return f"Tool error: {str(e)}"Guardrails on Tool Execution
Never let an agent call a tool with unbounded permissions. If a tool can write to a filesystem, delete records, or spend money, put an explicit allowlist and a confirmation step in front of it. A reasonable pattern is to classify each tool as read-only or mutating, and require a human-in-the-loop confirmation for anything mutating until you’ve built enough confidence in the agent’s behavior to relax that.
Rate Limiting and Timeouts
An agent stuck in a loop calling a slow external API can silently rack up cost and latency. Wrap every tool call with a timeout, and cap the number of tool calls per session. This is a small amount of code that prevents most of the runaway-cost incidents teams run into once an agent goes from a demo to something users actually invoke repeatedly.
Managing State and Memory
A single-session agent loop is easy. The harder problem when you build ai agents from scratch is state that persists across sessions — remembering what a user asked yesterday, or tracking the status of a long-running multi-step task.
For most use cases you don’t need a vector database on day one. A simple relational table tracking session_id, role, content, and timestamp is enough to reconstruct conversation history. Only reach for embeddings and semantic search once you have a concrete need to retrieve relevant history that a simple recency window can’t satisfy.
Trimming Context Without Losing Intent
As conversations grow, you’ll exceed the model’s context window or just waste tokens on irrelevant history. A common approach is to keep the last N full turns verbatim and summarize everything older into a single condensed block, re-injected at the top of the history. This keeps token usage predictable while preserving enough context for the agent to stay coherent across a long session.
Build AI Agents From Scratch: Deployment With Docker
Once the loop and tool layer work locally, the next step is packaging the agent so it runs reliably outside your laptop. This is the part of “build ai agents from scratch” that most tutorials skip entirely, and it’s where a DevOps background actually pays off.
A minimal agent service needs: the agent process itself, a way to receive triggers (webhook, queue, or scheduler), and persistent storage for state. Docker Compose is a reasonable starting point for a single-host deployment before you need anything like Kubernetes.
version: "3.9"
services:
agent:
build: .
restart: unless-stopped
environment:
- MODEL_API_KEY=${MODEL_API_KEY}
- DATABASE_URL=postgres://agent:agent@db:5432/agent
depends_on:
- db
ports:
- "8080:8080"
db:
image: postgres:16
restart: unless-stopped
environment:
- POSTGRES_USER=agent
- POSTGRES_PASSWORD=agent
- POSTGRES_DB=agent
volumes:
- agent_pgdata:/var/lib/postgresql/data
volumes:
agent_pgdata:If you’re new to the Compose file format itself, the Postgres Docker Compose setup guide and the Docker Compose Env guide cover the surrounding details — secrets handling, environment variable precedence, and volume persistence — that this snippet assumes you already know.
Running the Agent as a Long-Lived Service
If your agent needs to poll a queue or run on a schedule rather than respond to a single webhook, wrap the loop in a service with a clear retry policy and restart behavior. Docker’s restart: unless-stopped covers crash recovery, but you still need your own logic for “what happens if the model API is down for five minutes” — usually an exponential backoff with a cap, rather than a tight retry loop that burns through your rate limit.
For teams that don’t want to write this orchestration layer themselves, it’s worth comparing against a visual workflow tool. The guide on how to build AI agents with n8n shows the same core loop implemented as a workflow instead of raw code — useful context even if you ultimately decide to build ai agents from scratch in your own codebase, since it clarifies which parts of the problem are genuinely hard versus just boilerplate.
Hosting Considerations
Wherever you run this, you want predictable CPU and a static IP for outbound webhook calls. A small VPS is enough for most single-agent workloads — you don’t need a Kubernetes cluster until you’re running many agents concurrently or need horizontal autoscaling. Providers like DigitalOcean offer straightforward droplet sizing if you want to start with a single box before deciding whether to scale out.
Observability and Debugging
An agent that fails silently is worse than one that crashes loudly. Every production agent needs structured logging of the full decision trace: the prompt sent, the tool selected, the arguments, the result, and the final output. Without this, debugging a bad decision means guessing.
Log each step as a structured event rather than free text, so you can query it later:
echo '{"session_id":"abc123","step":2,"tool":"get_weather","args":{"city":"Berlin"},"result":"12C, cloudy"}' \
| tee -a agent_trace.jsonlA JSONL trace file like this is a low-effort starting point. As the system matures, ship these events to a real log aggregator so you can search and alert on patterns like repeated tool failures or sessions that hit max_steps without resolving.
Testing the Agent Loop
Unit-test the tool functions in isolation — they’re regular functions, so this is straightforward. For the loop itself, write scenario tests that feed a scripted sequence of model responses (mocked, not live API calls) and assert the loop takes the expected path. This catches regressions in your control flow without burning API credits on every test run, and it’s the same testing discipline you’d apply to any other stateful service.
Common Pitfalls When You Build AI Agents From Scratch
A few mistakes show up repeatedly in early agent implementations:
Keeping these in mind from the start saves a rewrite later. Most of them are standard software engineering discipline — logging, validation, idempotency — applied to a system where one of the callers happens to be a language model instead of a human.
Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.
FAQ
Do I need a framework to build AI agents from scratch?
No. A framework can save time on boilerplate, but the core loop — call model, execute tool, feed result back — is a few dozen lines of code. Many teams choose to build ai agents from scratch specifically to avoid framework lock-in and keep the reasoning loop fully inspectable.
How many tools should a single agent have?
Keep it focused. An agent with a large, unfocused tool list tends to make worse tool-selection decisions than one with a small, well-described set. Start with the minimum set needed for the task and add tools only when there’s a clear gap.
How do I prevent an agent from taking destructive actions?
Classify tools as read-only or mutating, and require explicit confirmation or a dry-run step before any mutating tool executes. Log every tool call so you can audit what happened after the fact.
Should I run the agent on a VPS or a serverless platform?
Either works. A VPS with Docker Compose is simpler to reason about and debug for a single agent or small fleet. Serverless makes more sense once you have bursty, infrequent invocations where paying for an always-on container doesn’t make sense.
Conclusion
Learning to build ai agents from scratch means understanding the reasoning loop, the tool execution layer, state management, and the deployment and observability work that turns a demo into a system you can trust. None of these pieces are individually complex, but skipping any of them — especially guardrails on tool execution and structured logging — is what turns an agent from a useful automation into a production incident. Start with the minimal loop shown here, add tools deliberately, and invest in tracing early; it’s far easier to build that discipline in from the start than to retrofit it once the agent is already handling real traffic. For deeper reference on the model side, the Anthropic API documentation and OpenAI API documentation are both good starting points once your own loop is solid enough to plug either provider into.
Leave a Reply