How to Build a Self-Hosted AI Real Estate Agent with Docker

Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

Every SaaS vendor is now shipping an “AI real estate agent” bolt-on for lead qualification, listing summaries, and chat-based property search. Most of them are thin wrappers around GPT-4 with a markup attached. If you run infrastructure for a real estate brokerage, property management company, or proptech startup, you can build the same thing yourself — self-hosted, cheaper at scale, and fully under your control.

This guide walks through the actual architecture: containerized LLM inference, a retrieval layer for property data, an agent framework to tie it together, and a production deployment on a cloud VPS. This is written for developers and sysadmins, not marketers — expect Docker Compose files and Python, not buzzwords.

What Is an AI Real Estate Agent, Technically?

Strip away the marketing and an AI real estate agent is a retrieval-augmented generation (RAG) pipeline with domain-specific tools bolted on. It typically does four things:

Answers natural-language questions about listings (“show me 3-bed homes under $450k near downtown with a garage”)

Summarizes property details, disclosures, and comps into digestible text

Qualifies leads by asking follow-up questions and scoring intent

Schedules showings or hands off to a human agent when confidence is low

None of this requires a proprietary platform. It requires an LLM, a vector store for your listing data, an orchestration layer, and a place to run it reliably. We’ve covered similar patterns before in our guide to self-hosting LLMs with Docker, and the same core pattern applies here — you’re just swapping the domain data.

Core Components of the Stack

A minimal production-grade AI real estate agent stack looks like this:

LLM inference server — Ollama or vLLM running an open-weight model (Llama 3, Mistral, or Qwen)

Vector database — Qdrant or pgvector for storing embedded listing descriptions

Orchestration layer — LangChain or LlamaIndex to handle retrieval and tool-calling

API layer — FastAPI to expose the agent to your website or CRM

Reverse proxy + TLS — Caddy or Nginx in front of everything

All five run comfortably as containers on a single mid-tier VPS for low-to-moderate traffic. At scale, you split inference onto a GPU node and keep the rest on CPU instances.

Architecture: From LLM to Deployment

The key architectural decision is whether to call a hosted API (OpenAI, Anthropic) or self-host the model. For a real estate agent handling sensitive client data — names, financials, property addresses — self-hosting avoids sending that data to a third party and gives you predictable costs at volume. We’ll build the self-hosted version.

Choosing and Running Your LLM Backend

Ollama is the fastest path to a running local LLM in a container. Here’s a docker-compose.yml that stands up the full stack:

version: "3.9"
services:
  ollama:
    image: ollama/ollama:latest
    container_name: agent-ollama
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  qdrant:
    image: qdrant/qdrant:latest
    container_name: agent-vectordb
    volumes:
      - qdrant_data:/qdrant/storage
    ports:
      - "6333:6333"

  agent-api:
    build: ./agent-api
    container_name: agent-api
    depends_on:
      - ollama
      - qdrant
    environment:
      - OLLAMA_HOST=http://ollama:11434
      - QDRANT_HOST=http://qdrant:6333
    ports:
      - "8000:8000"

volumes:
  ollama_data:
  qdrant_data:

Pull a model once the containers are up:

docker exec -it agent-ollama ollama pull llama3.1:8b

An 8B parameter model runs acceptably on CPU for low traffic and comfortably on a single consumer GPU for anything higher volume.

Building the Agent Logic with LangChain

The agent-api service is where the real estate logic lives. It embeds listing data into Qdrant, retrieves relevant listings per query, and lets the LLM reason over them with tool access.

# agent-api/main.py
from fastapi import FastAPI
from pydantic import BaseModel
from langchain_community.llms import Ollama
from langchain_community.vectorstores import Qdrant
from langchain_community.embeddings import OllamaEmbeddings
from langchain.chains import RetrievalQA

app = FastAPI()

llm = Ollama(base_url="http://ollama:11434", model="llama3.1:8b")
embeddings = OllamaEmbeddings(base_url="http://ollama:11434", model="llama3.1:8b")

vectorstore = Qdrant(
    client=None,
    collection_name="listings",
    embeddings=embeddings,
    url="http://qdrant:6333",
)

qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 5}),
)

class Query(BaseModel):
    question: str

@app.post("/ask")
def ask(query: Query):
    result = qa_chain.invoke(query.question)
    return {"answer": result["result"]}

This is intentionally minimal — production systems add conversation memory, lead-scoring logic, and a handoff tool that pings a human agent via Slack or CRM webhook when the model’s confidence drops below a threshold. LangChain’s tool-calling API is the cleanest way to wire that handoff in without writing a custom state machine.

To populate the vector store, run listing data through the same embeddings model on ingestion — typically a nightly cron job pulling from your MLS feed or CRM export, chunking property descriptions, and upserting them into Qdrant.

Deploying to Production

A local Docker Compose setup is fine for development, but production needs TLS, monitoring, and a host that won’t fall over under a traffic spike from a listing going viral on social media. A few notes from running similar RAG stacks in production:

Put a 4-8 vCPU / 16GB+ RAM VPS behind the agent-api and Qdrant containers if you’re running CPU inference — Hetzner instances are the best price-per-core we’ve found for this workload, and DigitalOcean droplets are a solid alternative if you want a more polished managed-network experience.

Front the API with Cloudflare for DDoS protection and caching of static assets, and to hide your origin IP from scrapers targeting your listing data.

Set container resource limits (mem_limit, cpus) in Compose so a runaway inference request doesn’t starve the vector DB.

Back up the Qdrant volume on a schedule — losing embedded listing data means a full re-ingestion.

Our Docker Compose guide for beginners covers the fundamentals of multi-container orchestration if you’re newer to Compose specifically.

For TLS termination, Caddy is the lowest-friction option — it handles Let’s Encrypt automatically:

agent.yourdomain.com {
    reverse_proxy agent-api:8000
}

Monitoring and Scaling the Agent

Once the agent is live, you need visibility into latency, error rates, and inference cost per query — LLM calls are far more expensive per-request than typical API endpoints, so a silent bug that triggers repeated calls can blow through compute budget fast.

Track p95 latency on the /ask endpoint — anything over 3-4 seconds will feel broken to a user typing into a chat widget

Alert on GPU/CPU saturation on the inference host before it causes timeouts

Log every query and response for auditing — real estate has fair-housing compliance implications, and you need a paper trail of what the AI told a prospective buyer

BetterStack is a solid option for uptime and log monitoring across the whole stack without standing up your own Prometheus/Grafana pair — worth it if you don’t already have observability tooling running. If you do run your own stack, our container monitoring guide with Prometheus covers the metrics worth scraping from each service.

At higher traffic, split the architecture: move Ollama to a dedicated GPU instance, keep the FastAPI layer on cheap CPU instances behind a load balancer, and scale the API horizontally since it’s stateless. Qdrant scales vertically well past what most single-brokerage deployments will ever need.

Handling Fair Housing and Compliance

This part matters more than the tech stack. Any AI system answering questions about housing in the US is subject to the Fair Housing Act — the model cannot discriminate or imply preference based on protected classes, even indirectly through phrasing. Bake explicit system-prompt constraints against discriminatory language into the LLM’s instructions, log every conversation, and have a human review process for edge cases. This isn’t optional legal boilerplate — it’s the single biggest risk in shipping this kind of system for a real brokerage.

Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

FAQ

Do I need a GPU to run an AI real estate agent locally?
No, but it helps. An 8B parameter model like Llama 3.1 runs on CPU at a few tokens per second, which is usable for low-traffic sites. For anything with real concurrent traffic, a single consumer GPU (RTX 4090 or similar) or a cloud GPU instance dramatically improves response time.

Is self-hosting actually cheaper than using OpenAI’s API?
At low volume, no — API calls are cheaper than running a dedicated server. Past a few thousand queries a day, self-hosting on a fixed-cost VPS typically wins, and you avoid sending client PII to a third party.

Can this replace a human real estate agent?
No, and it shouldn’t try to. It’s best used for lead qualification, initial questions, and listing search — freeing up human agents for negotiation, showings, and closing, where trust and local expertise matter most.

What LLM should I use for real estate use cases?
Llama 3.1 8B or Mistral 7B are good defaults — fast, cheap to run, and good enough at retrieval-augmented Q&A. Use a larger model (Llama 3.1 70B) if you need more nuanced reasoning over comps or disclosures and have the GPU budget for it.

How do I keep the agent from giving legally risky answers?
Constrain the system prompt explicitly against discriminatory language, log all conversations for audit, and route anything touching pricing negotiation or contract terms to a human. Treat the AI as a triage layer, not a decision-maker.

Does this integrate with existing CRMs like Follow Up Boss or kvCORE?
Yes, via webhook — have the FastAPI layer POST qualified leads and conversation summaries to your CRM’s API after each session, the same way you’d wire up any other lead source.

Self-hosting an AI real estate agent isn’t harder than using a SaaS platform — it’s a Docker Compose file, an open-weight model, and a VPS. The tradeoff is you own the maintenance, but you also own the data, the cost curve, and the ability to actually understand why the thing answered the way it did.