How to Build AI Agents From Scratch
Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.
Learning how to build AI agents from scratch is one of the most practical skills a backend or DevOps engineer can pick up right now. Rather than relying entirely on a no-code platform, understanding the core loop — perception, reasoning, tool use, memory — lets you debug, extend, and self-host agents with full control over cost and data. This guide walks through the architecture, the code, and the deployment steps you need to ship a working agent.
Why Learn How To Build AI Agents From Scratch
Visual builders like n8n are excellent for wiring together triggers and API calls quickly, and they remain a good choice for many production workflows — see our guide on how to build AI agents with n8n if that’s your starting point. But there’s real value in also knowing how to build AI agents from scratch in plain code, because it removes a layer of abstraction between you and the model’s behavior.
When you build an agent yourself, you control:
This matters most once an agent moves from a demo into something users or internal systems depend on daily. A from-scratch implementation is also just a better teaching tool: once you’ve built one agent loop by hand, every framework (LangChain, LlamaIndex, the OpenAI Agents SDK) becomes easier to read because you recognize the same primitives underneath.
When a Framework Makes More Sense
Not every project should start from zero. If you need broad tool-ecosystem support out of the box, or you’re prototyping quickly for a non-technical stakeholder, a framework or a visual tool will get you there faster. Our comparison of n8n vs Make is a good reference if you’re deciding between two popular automation platforms for that use case. The right call depends on whether you’re optimizing for speed of delivery or depth of control — both are legitimate engineering tradeoffs.
The Core Architecture of an AI Agent
At its simplest, an agent is a loop: the model receives context, decides on an action, that action executes, and the result feeds back into the next iteration. Understanding this loop is the real answer to how to build AI agents from scratch — everything else (memory stores, tool registries, orchestration frameworks) is scaffolding around this one idea.
The Perception-Reasoning-Action Loop
Every agent architecture, no matter how elaborate, reduces to three stages repeated until a task is complete:
1. Perception — the agent receives input: a user message, a webhook payload, or the output of its previous tool call.
2. Reasoning — the underlying LLM decides what to do next, given the current context and the list of tools it has access to.
3. Action — the agent executes a tool call (an API request, a database query, a shell command) and captures the result.
The loop terminates when the model decides it has enough information to produce a final answer, or when you hit a hard iteration limit — which you should always set, since an unbounded loop against a paid API is a real cost risk.
Tool Definition and Function Calling
Modern LLM APIs support structured function calling, where you describe available tools with a JSON schema and the model returns a structured call rather than free text you’d have to parse yourself. This is the mechanism that makes agents reliable enough for production use — before structured tool calling existed, developers had to regex-parse model output, which was fragile.
tools = [
{
"name": "get_weather",
"description": "Get current weather for a city",
"input_schema": {
"type": "object",
"properties": {
"city": {"type": "string"}
},
"required": ["city"]
}
}
]Setting Up Your Development Environment
Before writing agent logic, get a clean, reproducible environment. Python is the most common choice for agent development because of its mature ecosystem around HTTP clients, async I/O, and LLM SDKs, but the same architecture applies in Node.js or Go.
Project Structure and Dependencies
A minimal from-scratch agent project needs surprisingly little:
mkdir my-agent && cd my-agent
python3 -m venv venv
source venv/bin/activate
pip install anthropic python-dotenv requestsKeep your API key out of source control from day one — load it via environment variables, not a hardcoded string:
echo "ANTHROPIC_API_KEY=your_key_here" > .env
echo ".env" >> .gitignoreContainerizing the Agent for Deployment
Once the agent runs locally, package it for deployment the same way you would any backend service. A Dockerfile keeps the runtime consistent between your laptop and production:
version: "3.9"
services:
agent:
build: .
env_file: .env
restart: unless-stopped
volumes:
- ./data:/app/dataIf you’re new to the difference between a single Dockerfile and a multi-service Compose setup, our Dockerfile vs Docker Compose guide covers when to reach for each. For managing secrets like your API key across environments, see Docker Compose secrets.
Writing the Agent Loop in Code
This is the part that actually answers how to build AI agents from scratch: a working loop, not a diagram. Below is a minimal but complete implementation using the Anthropic Python SDK. It’s deliberately small — enough to understand every line, not enough for production hardening (that comes later).
import os
import json
from anthropic import Anthropic
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
def get_weather(city: str) -> str:
return f"It's 18C and cloudy in {city}."
TOOLS = [{
"name": "get_weather",
"description": "Get current weather for a city",
"input_schema": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}]
def run_agent(user_message: str, max_turns: int = 5):
messages = [{"role": "user", "content": user_message}]
for _ in range(max_turns):
response = client.messages.create(
model="claude-sonnet-5",
max_tokens=1024,
tools=TOOLS,
messages=messages
)
if response.stop_reason != "tool_use":
return response.content[0].text
messages.append({"role": "assistant", "content": response.content})
tool_results = []
for block in response.content:
if block.type == "tool_use" and block.name == "get_weather":
result = get_weather(**block.input)
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": result
})
messages.append({"role": "user", "content": tool_results})
return "Max turns reached without a final answer."This same loop pattern scales to dozens of tools — a database lookup, a REST API call, a file read — as long as each tool has a clear schema and a predictable, serializable return value.
Adding Memory and State
A stateless agent forgets everything between requests, which is fine for a single-turn tool but unacceptable for anything conversational. The simplest durable memory is a database row per conversation, keyed by a session ID, storing the running message list as JSON. For agents deployed alongside a Postgres instance, our Postgres Docker Compose setup guide is a solid starting point if you don’t already have one running.
For longer-running or multi-session memory, teams often add a vector store for semantic recall, but don’t reach for one until you actually need similarity search over past interactions — a plain relational table with a timestamp index solves most real use cases first.
Deploying and Operating Your Agent in Production
Building the loop is maybe a third of the work. The rest is making it observable and reliable once real traffic hits it.
Logging, Debugging, and Observability
Log every tool call, its input, its output, and the model’s stop reason — this is the single highest-leverage debugging investment you can make, because agent failures are usually a bad tool call or a malformed schema, not a model problem. If your agent runs inside Docker Compose, you already have a debugging path available: see Docker Compose logs for the commands to tail and filter service output efficiently, and Docker Compose logging for configuring log drivers and rotation so a chatty agent doesn’t fill your disk.
Where to Host Your Agent
A from-scratch agent with a handful of tool calls per request runs comfortably on a small VPS — you don’t need a GPU or a large instance unless you’re also self-hosting the model weights rather than calling a hosted API. If you’re setting up infrastructure for the first time, DigitalOcean is a reasonable option for a straightforward Docker-based deployment, and our unmanaged VPS hosting guide walks through the tradeoffs of running your own box versus a managed platform.
Rate Limits, Retries, and Cost Control
Every LLM API enforces rate limits, and every agent should implement exponential backoff on 429 responses rather than failing the whole request. Equally important is a hard cap on iterations per run — without one, a model stuck in a reasoning loop can burn through your API budget quickly. Track token usage per request and alert on unusual spikes; this is standard operational hygiene, the same way you’d monitor request volume for any backend service.
Comparing From-Scratch Agents to Framework-Based Agents
It’s worth being explicit about the tradeoff once you’ve built a working agent loop yourself. A from-scratch implementation gives you full visibility into every prompt and tool call, at the cost of having to build your own retry logic, memory layer, and tool registry. A framework or platform like n8n gives you those pieces pre-built, at the cost of an abstraction layer you don’t fully control. Neither is universally better — see our breakdown on building agentic AI for a broader look at where agentic patterns fit versus simpler automation. Many teams end up running both: hand-coded agents for core product logic, and a visual tool for lower-stakes internal automation.
Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.
FAQ
Do I need a framework to build an AI agent?
No. A framework like LangChain can speed up development, but the core agent loop — call the model, execute a tool, feed the result back — is straightforward to implement directly against any LLM API’s function-calling interface, as shown above.
Which programming language is best for building AI agents from scratch?
Python is the most common choice due to its LLM SDK support and general ecosystem maturity, but Node.js and Go both have mature HTTP and JSON tooling that work equally well for the same architecture.
How do I stop an agent from looping indefinitely?
Always set a maximum number of turns (or a token budget) per run, and treat hitting that limit as a normal failure mode to handle gracefully, not an exception to catch as an afterthought.
Is it expensive to run a self-built AI agent in production?
Cost is driven mostly by the number of tokens sent per turn and how many tool-call round trips a task requires, not by which language or hosting setup you use. Logging token usage per request, as described above, is the most direct way to keep costs predictable.
Conclusion
Knowing how to build AI agents from scratch gives you a durable mental model that transfers to every framework and platform you’ll encounter afterward. Start with the simple perception-reason-act loop, add one tool at a time, log everything, and deploy behind normal backend practices — rate limiting, retries, and observability. From there, deciding whether to stay code-first or adopt a visual orchestration tool like n8n becomes a much easier, better-informed decision. For the official API references used throughout this guide, see the Anthropic API documentation and Docker’s official documentation.
Leave a Reply