Category: Ai Agents

  • Zendesk AI Agents: The Complete Developer Setup Guide

    Zendesk AI Agents: A Developer’s Guide to Automating Support Workflows

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Support teams are drowning in repetitive tickets, and Zendesk AI agents are one of the more credible attempts to fix that without replacing the whole helpdesk stack. If you’re the engineer tasked with wiring this into your existing infrastructure — APIs, webhooks, monitoring, and all — this guide skips the marketing fluff and gets into the implementation details.

    We’ll cover what Zendesk AI agents actually do under the hood, how to authenticate and automate against the Zendesk API, how to deploy a custom integration service around them, and how to keep that service reliable once it’s carrying real customer traffic.

    What Are Zendesk AI Agents?

    Zendesk AI agents are conversational automation layers built on top of the standard Zendesk ticketing system. Unlike the old rule-based “macros” and canned responses, they use large language models to read incoming tickets, classify intent, pull context from your knowledge base, and either resolve the issue directly or hand off to a human agent with a pre-filled summary.

    From a developer’s perspective, the important part isn’t the chat UI — it’s that every action the AI agent takes is exposed through the same Zendesk REST API that powers everything else in the platform. That means you can observe, override, and extend agent behavior with regular HTTP calls and webhooks, the same way you’d instrument any other microservice.

    How Zendesk AI Agents Differ from Traditional Bots

    Traditional Zendesk bots relied on decision trees: match a keyword, return a canned macro. Zendesk AI agents instead run an intent-classification and retrieval pipeline against your help center content, so they can answer novel phrasing without you maintaining hundreds of trigger rules. The tradeoff is that behavior is less deterministic, which is exactly why you need proper logging, monitoring, and fallback logic around them — more on that later.

    Where AI Agents Fit in Your Support Architecture

    In most setups, Zendesk AI agents sit between your customer-facing channels (email, chat widget, help center) and your human agent queue. They triage first, resolve what they can, and escalate the rest. If you’re running a self-hosted knowledge base or documentation site alongside Zendesk, you’ll want to compare hosting tradeoffs — our self-hosted vs cloud comparison covers the same decision points that apply here.

    Setting Up Zendesk AI Agents via API

    Before you touch the AI agent configuration in the Zendesk admin panel, get your API access sorted. Everything downstream — webhooks, custom triggers, monitoring — depends on a working API token.

    Authenticating with the Zendesk API

    Zendesk supports API token auth, OAuth, and basic auth (deprecated for new apps). For server-to-server automation, an API token tied to a dedicated service account is the cleanest approach:

    # Store credentials as environment variables, never hardcode them
    export ZENDESK_SUBDOMAIN="yourcompany"
    export ZENDESK_EMAIL="[email protected]/token"
    export ZENDESK_API_TOKEN="your_api_token_here"
    
    # Verify authentication works
    curl -s -u "${ZENDESK_EMAIL}:${ZENDESK_API_TOKEN}" 
      "https://${ZENDESK_SUBDOMAIN}.zendesk.com/api/v2/users/me.json" | jq '.user.name'

    If that returns your service account’s name, you’re authenticated. Rotate this token regularly and store it in a secrets manager rather than a .env file committed to git.

    Automating Ticket Triage with Webhooks

    Zendesk AI agents publish events (ticket resolved, escalated, tagged) that you can consume via webhooks to trigger downstream automation — for example, pushing escalated tickets into PagerDuty or Slack. Here’s a minimal webhook receiver in Python using Flask:

    from flask import Flask, request, jsonify
    import hmac
    import hashlib
    import os
    
    app = Flask(__name__)
    WEBHOOK_SECRET = os.environ["ZENDESK_WEBHOOK_SECRET"]
    
    def verify_signature(payload, signature):
        expected = hmac.new(
            WEBHOOK_SECRET.encode(), payload, hashlib.sha256
        ).hexdigest()
        return hmac.compare_digest(expected, signature)
    
    @app.route("/webhooks/zendesk", methods=["POST"])
    def zendesk_webhook():
        signature = request.headers.get("X-Zendesk-Webhook-Signature", "")
        if not verify_signature(request.data, signature):
            return jsonify({"error": "invalid signature"}), 401
    
        event = request.json
        if event.get("type") == "ticket.escalated":
            # push to your incident/notification pipeline
            print(f"Escalation: ticket {event['ticket_id']}")
    
        return jsonify({"status": "received"}), 200
    
    if __name__ == "__main__":
        app.run(host="0.0.0.0", port=5000)

    Always verify the webhook signature — an unauthenticated endpoint that can trigger internal automations is a real attack surface. If you need a refresher on securing inbound webhooks generally, see our webhook security best practices guide.

    Deploying a Custom Integration Service

    You generally don’t want your webhook receiver running as a one-off script on someone’s laptop. Containerize it and deploy it behind a reverse proxy with TLS termination.

    # docker-compose.yml
    version: "3.9"
    services:
      zendesk-webhook:
        build: .
        restart: unless-stopped
        environment:
          - ZENDESK_WEBHOOK_SECRET=${ZENDESK_WEBHOOK_SECRET}
          - ZENDESK_API_TOKEN=${ZENDESK_API_TOKEN}
        ports:
          - "5000:5000"
        healthcheck:
          test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
          interval: 30s
          timeout: 5s
          retries: 3

    If you haven’t set up a Compose stack like this before, walk through our Docker Compose guide first — it covers the health check and restart-policy patterns used above in more depth.

    For hosting, a small VPS is more than enough for a webhook relay handling typical ticket volume — you don’t need a Kubernetes cluster for this. A DigitalOcean droplet in the 2GB/2vCPU tier comfortably handles thousands of webhook events per day. Put the endpoint behind Cloudflare so you get DDoS protection and can lock the origin down to Cloudflare’s IP ranges only, which meaningfully reduces the exposure of a public-facing automation endpoint.

    Testing the Integration End-to-End

    Before trusting this in production, exercise the full path — create a test ticket, let the AI agent process it, and confirm your webhook receiver logs the event correctly. Tools like Postman make it easy to replay Zendesk’s sample webhook payloads against your local receiver before you ever touch production credentials.

    Best Practices for Production Zendesk AI Agents

    A handful of things consistently separate a stable Zendesk AI agent deployment from a flaky one:

  • Pin your knowledge base sources. AI agents pull answers from your help center — stale or contradictory articles produce stale or contradictory answers. Audit content quarterly.
  • Set explicit escalation thresholds. Don’t let the agent guess indefinitely; define confidence thresholds below which it must hand off to a human.
  • Log every automated resolution. You need an audit trail for compliance and for spotting when the agent is confidently wrong.
  • Rate-limit your API calls. Zendesk enforces per-plan rate limits; back off with exponential retry rather than hammering 429 responses.
  • Rotate API tokens and webhook secrets on a fixed schedule, and store them in a secrets manager, not plaintext config.
  • Version your automation logic separately from Zendesk’s admin-configured triggers, so changes are reviewable in a pull request rather than buried in a UI history.
  • Monitoring and Reliability

    Once the integration is live, treat it like any other production service. Uptime monitoring on the webhook endpoint and alerting on error rates is non-negotiable — a silent failure here means tickets get stuck in limbo without anyone noticing. BetterStack is a solid option for this: it can monitor the /health endpoint on your webhook service, alert on-call via Slack or PagerDuty when it goes down, and centralize logs from the container so you’re not SSHing into the box to docker logs every time something looks off.

    For teams already tracking broader infrastructure health, it’s worth folding this into your existing observability stack rather than standing up a separate one-off dashboard — our DevOps monitoring tools roundup covers how to consolidate alerts like this across services.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Do Zendesk AI agents require a separate API integration to function?
    No — Zendesk AI agents work out of the box through the standard admin panel. A custom API/webhook integration is only needed if you want to trigger external automation (Slack alerts, PagerDuty escalations, custom analytics) based on agent activity.

    Can Zendesk AI agents access data outside the Zendesk knowledge base?
    By default, no. They’re scoped to your help center content and ticket history. Extending them to query external systems requires building a custom integration via Zendesk’s API and webhooks, as described above.

    What happens if the AI agent gives an incorrect answer?
    The agent should hand off to a human when confidence is low, but it can still be confidently wrong. This is why logging every automated resolution and periodically auditing transcripts is a hard requirement, not a nice-to-have.

    How do I secure the webhook endpoint that receives Zendesk AI agent events?
    Verify the HMAC signature on every incoming request, run the endpoint over HTTPS only, and restrict inbound traffic to your CDN or proxy’s IP ranges if you’re using one like Cloudflare.

    Will Zendesk AI agents work with a self-hosted help center?
    Zendesk AI agents are tied to Zendesk’s own Guide/help center product. If your documentation lives elsewhere, you’ll need to sync or mirror that content into Zendesk’s knowledge base for the agent to use it as a source.

    Is there a rate limit I need to plan for when integrating custom automation?
    Yes. Zendesk enforces per-plan API rate limits (varying by tier), and webhook delivery has its own retry/backoff behavior on Zendesk’s side. Design your receiver to be idempotent so retried webhook deliveries don’t cause duplicate actions.

    Wrapping Up

    Zendesk AI agents are genuinely useful for cutting down repetitive ticket volume, but the value for an engineering team comes from what you build around them — reliable webhook handling, proper authentication, and monitoring that catches failures before your support team does. Treat the integration like any other production service: containerize it, secure it, and put real observability on top before you trust it with customer-facing traffic.

  • Manus Ai Agent

    Manus AI Agent: A Practical Guide for DevOps Teams

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    The manus ai agent is an autonomous AI system designed to execute multi-step tasks with minimal human supervision, from web research to code generation and file manipulation. For DevOps and infrastructure teams evaluating agentic AI tools, understanding how a manus ai agent fits into an automation stack matters more than chasing hype. This guide covers what the manus ai agent does, how it compares to other agent frameworks, and how to integrate it safely into a production workflow.

    What Is the Manus AI Agent

    The manus ai agent belongs to a category of “general-purpose” autonomous agents that combine a large language model with tool-use capabilities: browsing, code execution, file system access, and API calls. Unlike a narrow chatbot that only answers questions in a single turn, a manus ai agent plans a sequence of actions, executes them, observes the results, and adjusts its plan accordingly.

    This loop — plan, act, observe, revise — is the defining characteristic of agentic AI systems in general, not something unique to any one vendor. What differentiates the manus ai agent from a simple scripted automation is that the planning step is handled by a language model rather than a fixed decision tree, which means it can handle tasks whose exact steps weren’t known in advance.

    Core Capabilities

    A typical manus ai agent deployment exposes the following capabilities to the underlying model:

  • Web browsing and information retrieval
  • Shell command execution in a sandboxed environment
  • File creation, editing, and reading
  • Code generation and execution across multiple languages
  • Task decomposition and multi-step planning
  • These capabilities are gated by permission layers, and in any serious deployment, engineers should treat each one as a potential attack surface rather than a convenience feature.

    How It Differs From Simple Chatbots

    A standard chatbot returns text. A manus ai agent returns actions — it can open a terminal, write a file, run a script, and report back on what happened. This makes it functionally closer to a junior engineer following a runbook than to a search assistant. That distinction is important operationally: an agent that can execute shell commands needs the same access controls, logging, and review discipline as any other automation that touches production systems.

    Manus AI Agent Architecture

    Most agent frameworks, including manus-style systems, follow a similar architectural pattern regardless of the specific vendor implementation. Understanding this pattern helps when deciding where to host and how to constrain the agent.

    Planning and Execution Loop

    The core loop typically looks like this:

    1. The agent receives a goal from the user.
    2. The model decomposes the goal into a sequence of candidate steps.
    3. Each step is executed via a tool call (shell, browser, API, file I/O).
    4. The result of that tool call is fed back into the model’s context.
    5. The model decides whether to continue, retry, or stop.

    This loop repeats until the goal is met or a stopping condition (timeout, error budget, explicit user intervention) is hit. Because each step depends on the outcome of the previous one, a manus ai agent’s behavior is inherently non-deterministic across runs, even with the same initial prompt — a property worth remembering when writing tests or automated verification around agent output.

    Sandboxing and Isolation

    Because a manus ai agent can execute arbitrary shell commands, sandboxing is not optional. Production-grade deployments isolate the execution environment using containers or VMs, restrict outbound network access to an allowlist, and cap resource usage (CPU, memory, disk, and wall-clock time) per task. If you’re already running containerized workloads, the same discipline you’d apply to an untrusted build step applies here — see this guide on managing secrets safely in Compose stacks for patterns that translate directly to isolating agent credentials.

    Deploying a Manus AI Agent on Your Own Infrastructure

    Some teams run a hosted manus ai agent product directly; others prefer to self-host an open agent framework with similar capabilities to retain control over data, logging, and cost. Self-hosting an agent stack on a VPS follows much the same pattern as self-hosting any other automation platform.

    Minimum Infrastructure Requirements

    A self-hosted agent runtime generally needs:

  • A container runtime (Docker or an OCI-compatible equivalent)
  • Persistent storage for agent memory/state and task logs
  • Network egress rules limiting which external services the agent can reach
  • A reverse proxy or gateway if the agent exposes a web UI or webhook endpoint
  • If you’re new to running this kind of stack on a VPS from scratch, this unmanaged VPS hosting guide walks through the baseline server setup you’d want before adding any agent workload on top.

    Example: Containerized Agent Runtime

    A minimal Compose definition for a self-hosted agent worker, isolated in its own network with no direct access to other stacks, looks like this:

    version: "3.9"
    services:
      agent-runtime:
        image: your-agent-runtime:latest
        restart: unless-stopped
        environment:
          - AGENT_MODE=sandboxed
          - MAX_TASK_TIMEOUT=300
        networks:
          - agent-net
        volumes:
          - agent-data:/data
        deploy:
          resources:
            limits:
              cpus: "1.0"
              memory: 1g
    
    networks:
      agent-net:
        driver: bridge
    
    volumes:
      agent-data:

    Resource limits here aren’t decorative — an agent that goes into a retry loop against a slow API can otherwise consume unbounded CPU and memory on a shared host.

    Verifying the Agent Container Is Healthy

    Once deployed, check that the container started cleanly and review its logs before pointing any real workload at it:

    docker compose up -d agent-runtime
    docker compose ps
    docker compose logs -f agent-runtime

    If you need to debug why a task silently failed or hung, the same log-reading discipline used for any Compose service applies — this Docker Compose logs guide covers filtering and following logs across multi-container stacks, which is useful once your manus ai agent setup grows beyond a single container.

    Comparing the Manus AI Agent to Other Agent Frameworks

    Before committing to any single agent framework, it’s worth understanding the landscape it competes in.

    Manus AI Agent vs. Workflow Automation Tools

    Workflow automation tools like n8n or Make execute predefined graphs of steps — reliable and auditable, but limited to what was explicitly configured. A manus ai agent instead generates its own step sequence at runtime based on the goal it’s given. The tradeoff is predictability versus flexibility: a workflow tool will do exactly the same thing every time; an agent might solve a novel problem but could also take an unexpected or inefficient path. Many teams end up combining both — using a workflow engine for the deterministic, auditable parts of a pipeline and an agent for the parts that genuinely require judgment. If you’re comparing these automation approaches directly, this n8n vs Make comparison is a useful reference point for the deterministic side of that tradeoff.

    Manus AI Agent vs. Custom-Built Agents

    Teams that need tighter control over tool access, logging format, or model choice often build a custom agent rather than adopting a packaged product. This guide on how to create an AI agent covers the fundamentals of that build-it-yourself path, and this deeper walkthrough on building agentic AI systems covers the planning-loop architecture in more detail. A custom agent takes more engineering time upfront but avoids vendor lock-in and gives you full visibility into every tool call the agent makes — which matters a great deal when the agent has shell or file-system access.

    Security Considerations for Running a Manus AI Agent

    Any agent with shell execution or file access should be treated as a privileged process, not a chat feature.

    Least-Privilege Execution

    Run the agent’s execution environment under a dedicated, unprivileged user or service account, never as root and never with the same credentials used by your CI/CD pipeline or production deployment user. Scope any API keys the agent has access to as narrowly as possible — if it only needs to read from one service, don’t hand it a token with write access to everything.

    Auditing Agent Actions

    Every action a manus ai agent takes — every shell command, file write, and outbound request — should be logged in a way that survives the agent’s own session, not just held in its internal memory. This is the same principle behind maintaining clean environment variable and secrets hygiene in any automated pipeline; see this guide on managing Compose environment variables for a pattern of keeping configuration explicit and auditable rather than buried inside the running process.

    Rate Limiting and Cost Control

    Because a manus ai agent can call an LLM API repeatedly within a single task (once per planning step), uncontrolled loops can generate unexpectedly high API costs. Set explicit step-count and timeout limits per task, and monitor token usage the same way you’d monitor any other metered cloud resource.

    Choosing Where to Host Your Manus AI Agent Stack

    If you’re self-hosting the runtime rather than using a fully managed product, the hosting decision matters for both cost and latency, especially if the agent makes frequent outbound calls to browsing or search tools. A general-purpose VPS provider such as DigitalOcean or Hetzner is sufficient for most agent workloads, since the heavy lifting (model inference) usually happens via an external API rather than on the box itself. Size the instance for the container orchestration and logging overhead, not for running the model locally, unless you’re specifically self-hosting an open-weight model as well.

    For official reference material on container orchestration and networking fundamentals that apply directly to isolating an agent runtime, see the Docker documentation and the Kubernetes documentation if you’re running the agent at larger scale across multiple nodes.


    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Is a manus ai agent the same as a chatbot?
    No. A chatbot returns text in response to a prompt. A manus ai agent plans and executes multi-step actions — running commands, browsing the web, editing files — and only reports back once those actions are complete or blocked.

    Can I self-host a manus ai agent instead of using a hosted product?
    Yes, provided you’re comfortable building or adopting an open agent framework and taking on the sandboxing, logging, and access-control work yourself. Self-hosting gives you more control over data handling and cost but requires more operational ownership than a managed product.

    What’s the biggest risk in running a manus ai agent in production?
    Uncontrolled tool access is the most common risk — an agent with unrestricted shell or network access can take actions well beyond what a task actually required. Least-privilege execution, sandboxing, and per-task resource limits are the standard mitigations.

    Does a manus ai agent replace workflow automation tools like n8n?
    Not typically. They solve different problems: workflow tools execute known, repeatable sequences reliably; agents handle tasks whose steps aren’t fully known in advance. Most production setups use both, with the agent handling the judgment-heavy parts and the workflow tool handling the deterministic parts.

    Conclusion

    The manus ai agent represents a broader shift toward AI systems that act rather than just respond, and that shift brings real operational responsibility along with it. Whether you adopt a managed manus ai agent product or self-host an equivalent open framework, the fundamentals stay the same: sandbox the execution environment, apply least-privilege access to every credential and API the agent can reach, log every action it takes, and cap runtime and cost per task. Treat it as you would any other automation with shell access to production infrastructure — because that’s exactly what it is.

  • AI Agentic Workflow: Build Automated DevOps Pipelines

    AI Agentic Workflow: What It Is and How to Build One

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve spent any time around DevOps or platform engineering lately, you’ve heard the phrase “agentic workflow” thrown around like it’s the new microservices. Strip away the hype and an ai agentic workflow is just a system where an LLM-driven agent plans, executes, and evaluates a sequence of tasks with minimal human babysitting — calling tools, reading their output, and deciding what to do next based on that output, rather than following a fixed script.

    This guide breaks down what actually makes a workflow “agentic” instead of just “scripted,” walks through a real implementation using Docker and a task queue, and covers the monitoring setup you need before you trust an agent anywhere near production infrastructure.

    What Is an AI Agentic Workflow?

    At its simplest, an agentic workflow is a loop: the agent receives a goal, decides on an action, executes that action through a tool (a shell command, an API call, a database query), observes the result, and decides whether to continue, retry, or stop. The key difference from a traditional CI/CD pipeline is that the decision points are made by a model at runtime instead of being hardcoded by a developer in advance.

    Agentic vs. Traditional Automation

    A traditional bash script or Ansible playbook executes a fixed sequence: step 1, then step 2, then step 3, with if/else branches a human wrote ahead of time. An agentic workflow instead gives the model:

  • A goal (“restart the failing service and confirm health checks pass”)
  • A set of tools it’s allowed to call (shell, HTTP client, database driver)
  • A feedback loop so it can see whether its last action worked
  • The model decides which tool to call and in what order, and can recover from unexpected states without a developer having anticipated every branch. That’s genuinely useful for operational tasks like log triage, incident diagnosis, or dependency upgrades where the failure modes are too varied to script exhaustively.

    The Core Loop: Plan, Act, Observe, Repeat

    Most production agent frameworks — LangGraph, CrewAI, AutoGen, or a hand-rolled loop — implement some version of the ReAct pattern (Reason + Act). You can read the original research behind this approach in the ReAct paper on arXiv. The loop looks like this:

    1. Plan — the model reasons about the current state and picks the next action.
    2. Act — the action is executed against a real tool or system.
    3. Observe — the tool’s output (stdout, HTTP response, exit code) is fed back to the model.
    4. Repeat or terminate — the model decides if the goal is met or another iteration is needed.

    The hard engineering problem isn’t the model call — it’s building a safe, observable execution environment around that loop.

    Core Components You Need

    Before you wire an LLM up to docker exec, you need three pieces of infrastructure in place.

    The Model and Tool-Calling Layer

    This is whatever LLM API you’re using (Anthropic, OpenAI, or a self-hosted model) combined with a structured tool-calling interface. The model doesn’t run code itself — it returns a structured request (“call restart_service with argument nginx“), and your code executes it.

    The Orchestrator

    Something has to own the loop state: which step the agent is on, what tools are available, and when to stop. This is typically a lightweight Python process or a task queue worker, not the LLM itself.

    State and Memory

    Agents need somewhere to persist context between steps — a Redis instance, a Postgres table, or even a JSON file for simple cases. Without persisted state, a crash mid-workflow means starting from zero.

    Building a Simple Agentic Pipeline with Docker

    Let’s build a minimal but real example: an agent that monitors a Docker container, and if it’s unhealthy, inspects logs, attempts a restart, and verifies recovery.

    Start with a docker-compose.yml that isolates the agent from the host, similar to the pattern we cover in our guide to Docker Compose for microservices:

    version: "3.9"
    services:
      agent:
        build: ./agent
        volumes:
          - /var/run/docker.sock:/var/run/docker.sock:ro
        environment:
          - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
        depends_on:
          - redis
      redis:
        image: redis:7-alpine
        ports:
          - "6379:6379"

    The agent container gets read-only access to the Docker socket so it can inspect container state without full host access — a small but important guardrail. Full Docker socket security guidance is available in Docker’s own documentation.

    Here’s the core agent loop in Python, using a tool-calling pattern:

    import docker
    import anthropic
    
    client = anthropic.Anthropic()
    docker_client = docker.from_env()
    
    tools = [
        {
            "name": "get_container_logs",
            "description": "Fetch the last 50 lines of logs for a container",
            "input_schema": {
                "type": "object",
                "properties": {"name": {"type": "string"}},
                "required": ["name"],
            },
        },
        {
            "name": "restart_container",
            "description": "Restart a named Docker container",
            "input_schema": {
                "type": "object",
                "properties": {"name": {"type": "string"}},
                "required": ["name"],
            },
        },
    ]
    
    def execute_tool(name, tool_input):
        if name == "get_container_logs":
            container = docker_client.containers.get(tool_input["name"])
            return container.logs(tail=50).decode()
        if name == "restart_container":
            container = docker_client.containers.get(tool_input["name"])
            container.restart()
            return f"Restarted {tool_input['name']}"
        return "Unknown tool"
    
    def run_agent(goal, container_name, max_steps=5):
        messages = [{"role": "user", "content": f"{goal} Container: {container_name}"}]
        for step in range(max_steps):
            response = client.messages.create(
                model="claude-sonnet-5",
                max_tokens=1024,
                tools=tools,
                messages=messages,
            )
            if response.stop_reason != "tool_use":
                return response.content
            tool_call = next(b for b in response.content if b.type == "tool_use")
            result = execute_tool(tool_call.name, tool_call.input)
            messages.append({"role": "assistant", "content": response.content})
            messages.append({
                "role": "user",
                "content": [{
                    "type": "tool_result",
                    "tool_use_id": tool_call.id,
                    "content": result,
                }],
            })
        return "Max steps reached without resolution"

    This is intentionally minimal — no retries, no timeout handling — but it demonstrates the full plan/act/observe loop with real tool execution against the Docker API.

    Orchestrating Multi-Agent Pipelines

    A single agent handling one container is fine for a demo. Real infrastructure needs a queue so multiple agents can run concurrently without stepping on each other. A common pattern uses Redis as both the state store and the queue, with Celery or a lightweight custom worker pulling jobs:

    from redis import Redis
    from rq import Queue
    
    redis_conn = Redis(host="redis", port=6379)
    queue = Queue("agent-tasks", connection=redis_conn)
    
    job = queue.enqueue(run_agent, "Diagnose and fix", "web-app-1")
    print(job.id)

    Each job carries its own conversation history and tool scope, so one misbehaving agent can’t affect another container it wasn’t authorized to touch. This is the same isolation principle we discuss in our piece on self-hosted monitoring stacks — least privilege, scoped credentials, and no shared blast radius between workers.

    Guardrails You Cannot Skip

    Before letting an agent touch anything resembling production, put these in place:

  • Explicit tool allowlists — never give the model a raw shell; expose only named, scoped functions.
  • Max-step limits — cap iterations so a confused agent can’t loop indefinitely and burn API spend.
  • Human approval gates for destructive actions (deletes, force-pushes, scaling down production).
  • Full audit logging of every tool call, input, and output the agent produced.
  • Rate limits on both API calls and infrastructure-affecting actions per time window.
  • Monitoring and Observability for Agentic Systems

    Agentic workflows fail silently if you’re not watching closely — a model can “succeed” at a task in a way that’s technically correct but operationally wrong (restarting the wrong container, for instance). You need three layers of visibility:

    1. Uptime and health checks on anything the agent manages. A service like BetterStack gives you incident alerting and status pages without building your own from scratch.
    2. Infrastructure metrics for the hosts running your agent workers — CPU, memory, and queue depth matter here as much as they do for any worker fleet, and providers like DigitalOcean make it straightforward to spin up isolated droplets per environment.
    3. Edge protection if any part of your agent’s tool surface is exposed via webhook or API — Cloudflare in front of that endpoint adds WAF rules and rate limiting cheaply.

    Log every single tool call the agent makes, with timestamps and full input/output. When something goes wrong at 3 a.m., that log is the only way to reconstruct what the agent actually did versus what you assumed it did.

    Common Pitfalls

    Teams adopting agentic workflows tend to hit the same set of problems:

  • Giving the agent unscoped shell access instead of named tools
  • No maximum step count, leading to runaway API costs
  • Treating agent output as ground truth without verification steps
  • Skipping structured logging until after the first incident
  • Running agents with the same credentials as human operators instead of scoped service accounts
  • None of these are exotic — they’re the same operational discipline you’d apply to any automated system with write access to production, just applied to a system whose decisions are less predictable than a static script.


    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    What’s the difference between an AI agentic workflow and a normal automation pipeline?
    A normal pipeline follows a fixed sequence written by a developer. An agentic workflow lets a model decide the sequence of actions at runtime based on the results of previous steps, so it can adapt to situations the developer didn’t explicitly script for.

    Do I need a specific framework like LangGraph or CrewAI to build one?
    No. Frameworks help with boilerplate (state management, tool schemas, retries) but the core loop — plan, act, observe, repeat — can be built with a plain API client and a few dozen lines of Python, as shown above.

    Is it safe to let an agent restart production containers on its own?
    Only with strict guardrails: scoped tool access, step limits, audit logging, and human approval for high-risk actions. Start agentic workflows in staging environments before granting any production write access.

    How do I control the cost of running agentic workflows?
    Cap the number of loop iterations per task, set a token budget per run, and cache repeated tool calls where possible. Runaway loops are the most common source of unexpected API spend.

    Can agentic workflows replace CI/CD pipelines entirely?
    No — they complement them. CI/CD is deterministic and auditable by design, which is exactly right for build and deploy steps. Agentic workflows are better suited to diagnosis, triage, and remediation tasks where the failure space is too broad to script exhaustively.

    What’s the best first project to try this on?
    A read-only diagnostic agent: something that inspects logs and metrics and produces a summary, without any write access to infrastructure. It lets you validate the model’s reasoning quality before you ever give it a tool that can change system state.

    AI agentic workflows aren’t magic — they’re a plan/act/observe loop wrapped around tools you already trust, with a model making the routing decisions. Get the guardrails right first, and the automation follows naturally.

  • Salesforce Agentic AI: A DevOps Deployment Guide

    Salesforce Agentic AI: A Practical DevOps Guide to Deployment, Monitoring, and Security

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Salesforce Agentic AI — branded as Agentforce — is moving fast from marketing slide to production workload. If you’re the DevOps or platform engineer tasked with actually running the integration layer that connects Agentforce to your internal systems, the vendor documentation stops well short of what you need: how to containerize the middleware, how to monitor autonomous agent actions, and how to keep API credentials from becoming your next incident.

    What Is Salesforce Agentic AI (Agentforce)?

    Salesforce Agentic AI refers to autonomous or semi-autonomous “agents” built on the Agentforce platform that can reason over CRM data, call external APIs, and take multi-step actions without a human triggering every step. Instead of a static workflow rule (“if case status = closed, send email”), an agent can decide which action to take based on context — routing a support ticket, updating a record, or calling a webhook into your own infrastructure.

    For DevOps teams, the practical implication is this: Agentforce doesn’t just read and write to Salesforce’s own database anymore. It reaches out — via REST APIs, webhooks, and Salesforce Connect — into systems you own. That means your Docker containers, your Kubernetes clusters, and your monitoring stack are now part of the blast radius.

    How Agentic AI Differs from Traditional Salesforce Automation

    Traditional Salesforce automation (Flow, Process Builder, Apex triggers) is deterministic. Given the same input, you get the same output every time, and the execution path is fully auditable in the Setup UI.

    Agentic AI is probabilistic. The agent uses a large language model to decide the next action, which means:

  • The same input can produce different outputs across runs
  • Actions are chosen dynamically from a defined “action library,” not hardcoded
  • Debugging requires reasoning traces, not just stack traces
  • Rate limits and cost controls matter more, since each agent decision may trigger an LLM inference call
  • This shift is exactly why treating agentic workflows like standard automation is a mistake. You need observability built for non-deterministic systems, which is closer to how you’d monitor a recommendation engine than a cron job.

    Architecting the Integration Layer

    Most real-world Agentforce deployments need a middleware layer — a small service that sits between Salesforce’s outbound webhooks/Platform Events and your internal systems (databases, ticketing tools, internal APIs). This is where DevOps ownership actually begins, since Salesforce itself is a black box you don’t control.

    A typical architecture looks like this:

  • Salesforce Agentforce triggers a Platform Event or calls an outbound webhook
  • A containerized listener service receives the event
  • The listener validates, logs, and forwards the payload to your internal API or queue
  • Your internal systems process the action and optionally call back into Salesforce via REST API
  • Running that listener service on a self-hosted VPS instead of relying on Salesforce’s native connectors gives you full control over logging, retries, and security — all things you’ll need once agents start making decisions autonomously.

    Containerizing Your Agentforce Middleware

    Here’s a minimal, production-ready pattern for a webhook listener that receives Agentforce events and forwards them to an internal queue. It’s written in Node.js, but the pattern applies to any language.

    # Dockerfile
    FROM node:20-alpine
    
    WORKDIR /app
    COPY package*.json ./
    RUN npm ci --omit=dev
    
    COPY src/ ./src/
    
    ENV NODE_ENV=production
    EXPOSE 8080
    
    HEALTHCHECK --interval=30s --timeout=3s 
      CMD wget -qO- http://localhost:8080/health || exit 1
    
    CMD ["node", "src/listener.js"]

    And the accompanying docker-compose.yml that pairs the listener with a Redis-backed queue and a log shipper:

    version: "3.9"
    services:
      agentforce-listener:
        build: .
        restart: unless-stopped
        ports:
          - "8080:8080"
        environment:
          - SALESFORCE_WEBHOOK_SECRET=${SALESFORCE_WEBHOOK_SECRET}
          - REDIS_URL=redis://queue:6379
        depends_on:
          - queue
        logging:
          driver: "json-file"
          options:
            max-size: "10m"
            max-file: "3"
    
      queue:
        image: redis:7-alpine
        restart: unless-stopped
        volumes:
          - redis-data:/data
    
    volumes:
      redis-data:

    A quick note on the webhook secret: Salesforce signs outbound messages, and your listener must verify that signature before processing anything. Skipping this step means anyone who guesses your endpoint URL can inject fake agent actions into your systems.

    // src/listener.js
    const express = require('express');
    const crypto = require('crypto');
    const app = express();
    
    app.use(express.json({ verify: (req, res, buf) => { req.rawBody = buf; } }));
    
    function verifySignature(req) {
      const signature = req.headers['x-sfdc-signature'];
      const expected = crypto
        .createHmac('sha256', process.env.SALESFORCE_WEBHOOK_SECRET)
        .update(req.rawBody)
        .digest('base64');
      return signature === expected;
    }
    
    app.post('/agentforce/webhook', (req, res) => {
      if (!verifySignature(req)) {
        return res.status(401).send('Invalid signature');
      }
      // forward to queue, log, ack
      console.log('Agent action received:', req.body.actionType);
      res.status(200).send('OK');
    });
    
    app.get('/health', (req, res) => res.status(200).send('ok'));
    
    app.listen(8080, () => console.log('Agentforce listener running on 8080'));

    If you’re new to running multi-container stacks like this, our Docker Compose guide walks through networking, volumes, and restart policies in more depth.

    Monitoring Agentic AI Workflows in Production

    Because agentic workflows are non-deterministic, standard uptime monitoring isn’t enough. You need to answer three questions your monitoring stack probably doesn’t cover today:

  • Did the agent take an action it shouldn’t have?
  • How many LLM inference calls happened, and what did they cost?
  • Are actions completing within acceptable latency, and are retries piling up?
  • Setting Up Alerts for Agent Actions

    At minimum, instrument your listener service to emit structured logs for every agent action it receives, then ship those logs to a monitoring platform that supports log-based alerting. A service like BetterStack works well here — you can set up an alert the moment an unexpected action type appears, rather than finding out from an angry Slack message.

    A basic structured log entry should include:

    {
      "timestamp": "2026-07-05T14:22:01Z",
      "agent_id": "support-triage-agent",
      "action_type": "update_case_status",
      "record_id": "500xx000003DHP",
      "decision_confidence": 0.87,
      "latency_ms": 412
    }

    Log that decision_confidence field if Agentforce exposes it in your org — a sudden drop in average confidence across agent decisions is often the earliest sign that something upstream (a prompt change, a data quality issue) has gone wrong.

    You should also track this at the infrastructure level, not just the application level. If you’re already using our guide to monitoring Dockerized services, extend those dashboards with a panel specifically for agent action volume and error rate.

    Securing API Credentials and Agent Permissions

    Agentforce needs credentials to call out to your systems, and your systems need credentials to call back into Salesforce. This is the part teams get wrong most often — treating agent credentials the same as a normal integration user, when an agent’s blast radius is fundamentally different because it acts autonomously.

    A few concrete rules:

  • Create a dedicated Salesforce integration user scoped to only the objects and fields the agent actually needs — never reuse an admin account
  • Store webhook secrets and API tokens in a secrets manager or environment-injected vault, never in a Docker image or committed .env file
  • Rotate the webhook signing secret on a schedule, and immediately after any suspected exposure
  • Put your listener endpoint behind a reverse proxy with rate limiting, and consider fronting it with Cloudflare to absorb abusive traffic before it hits your container
  • Log every agent-initiated write with enough context to reconstruct “why” the agent acted, not just “what” it did
  • For the underlying documentation on API scopes and Named Credentials, Salesforce’s own developer documentation is the authoritative source — read it before assuming your integration user’s permission set is scoped correctly.

    Scaling Your Integration Infrastructure

    As agent adoption grows inside an org, webhook volume grows with it — often faster than teams expect, since agents can trigger cascading actions (an agent updates a record, which triggers a flow, which fires another webhook). Plan your listener infrastructure for burst traffic, not just steady-state load.

    Practical scaling steps:

  • Run the listener behind a load balancer with at least two replicas, so a single container restart doesn’t drop events
  • Use a durable queue (Redis Streams, SQS, or similar) rather than processing webhooks synchronously, so spikes don’t cause timeouts back to Salesforce
  • Set conservative Salesforce API request limits in your org and alert well before you hit them — agentic workflows can burn through daily API call limits far faster than scheduled batch jobs
  • Host on infrastructure you can scale predictably; a provider like DigitalOcean makes it straightforward to add droplets or resize containers as agent traffic grows
  • If you’re hosting this stack yourself rather than relying on managed PaaS, our VPS deployment guide covers baseline hardening steps you’ll want in place before exposing any webhook endpoint to the public internet.


    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Does Salesforce Agentic AI require its own infrastructure, or does it run entirely inside Salesforce?
    Core agent reasoning runs on Salesforce’s infrastructure, but almost every real integration needs external middleware — a listener, queue, or API bridge — running on infrastructure you manage, which is the focus of this guide.

    Can I run the integration middleware on a single small server?
    Yes for pilots or low-volume use cases. A single 2-vCPU droplet running the Docker Compose stack above is enough to start. Plan to add replicas and a load balancer once agent-triggered traffic becomes unpredictable.

    How do I debug an agent action that shouldn’t have happened?
    You need structured logs correlating the agent’s decision, confidence score, and the exact payload it acted on. Without that trace, you’re debugging a black box. Log everything at the listener level, not just inside Salesforce.

    Is Agentforce’s outbound webhook traffic encrypted and authenticated by default?
    Traffic is encrypted via HTTPS, but authentication is your responsibility — you must verify the HMAC signature on every incoming webhook, as shown in the code example above.

    What’s the biggest cost risk with agentic AI integrations?
    Uncontrolled LLM inference calls and Salesforce API limit overages. Both scale with agent decision volume, which is harder to predict than a scheduled batch job, so budget alerts are essential from day one.

    Do I need Kubernetes to run this, or is Docker Compose enough?
    Docker Compose is sufficient for most single-region deployments. Move to Kubernetes only once you need multi-region failover or auto-scaling based on queue depth — most teams don’t need that on day one.

  • AI Call Agent Guide: Self-Host on Linux with Docker

    How to Self-Host an AI Call Agent on Linux with Docker

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve spent any time researching customer support automation, you’ve probably run into the term AI call agent — a voice-driven system that answers, routes, and even resolves phone calls without a human on the line. Most vendors want you to rent their hosted platform at a per-minute rate. If you run Linux boxes for a living, you can build the same functionality yourself, keep your call data on infrastructure you control, and avoid the recurring SaaS bill.

    This guide walks through a self-hosted AI call agent stack: SIP trunking with Asterisk, speech-to-text and text-to-speech pipelines, an LLM for reasoning, and Docker Compose to tie it all together. It assumes you’re comfortable with a Linux shell and basic networking.

    Why Self-Host an AI Call Agent

    Commercial AI call agent platforms are fast to set up but come with tradeoffs:

  • Per-minute or per-seat pricing that scales badly past a few hundred calls a month
  • Your call transcripts and customer data living on someone else’s servers
  • Limited ability to swap in a different LLM or fine-tune responses
  • Vendor lock-in on integrations and webhooks
  • A self-hosted stack costs you a VPS and some setup time, but you own the whole pipeline. If you already manage a Docker Compose stack for other services, adding a call agent is a natural extension.

    Core Components of the Stack

    An AI call agent isn’t one piece of software — it’s a pipeline. At minimum you need:

  • SIP/PBX layer — handles the actual phone call (we’ll use Asterisk)
  • Speech-to-text (STT) — converts caller audio to text in real time
  • LLM reasoning layer — decides what to say back, using tools/functions if needed
  • Text-to-speech (TTS) — converts the response back to audio
  • Telephony provider — a SIP trunk provider that connects to the public phone network (Twilio, Telnyx, or a local carrier)
  • Each of these runs as its own container, which keeps the system easy to update piece by piece without breaking the whole pipeline.

    Setting Up the Docker Environment

    Start with a clean VPS. A 4 vCPU / 8GB RAM instance is a reasonable starting point if you’re running a small local STT/TTS model alongside Asterisk. If you’re calling out to a hosted LLM API instead of running one locally, you can get away with less.

    # update and install docker
    sudo apt update && sudo apt install -y ca-certificates curl gnupg
    sudo install -m 0755 -d /etc/apt/keyrings
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
    echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
    sudo apt update && sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

    Verify the install:

    docker --version
    docker compose version

    Now lay out the project directory:

    mkdir -p ~/ai-call-agent/{asterisk,stt,tts,agent}
    cd ~/ai-call-agent

    Docker Compose Configuration

    Here’s a minimal docker-compose.yml that wires up the four core services. Each container is isolated on its own bridge network, with only the ports that need to be exposed actually exposed.

    version: "3.9"
    
    services:
      asterisk:
        image: andrius/asterisk:latest
        ports:
          - "5060:5060/udp"
          - "10000-10100:10000-10100/udp"
        volumes:
          - ./asterisk/conf:/etc/asterisk
        networks:
          - callagent
    
      stt:
        build: ./stt
        environment:
          - MODEL=whisper-small
        networks:
          - callagent
    
      tts:
        build: ./tts
        environment:
          - VOICE=en-us-standard
        networks:
          - callagent
    
      agent:
        build: ./agent
        depends_on:
          - stt
          - tts
        environment:
          - LLM_ENDPOINT=http://llm:8000/v1/chat/completions
        networks:
          - callagent
    
    networks:
      callagent:
        driver: bridge

    Bring the stack up:

    docker compose up -d
    docker compose logs -f agent

    The agent service is where the actual decision logic lives — it receives transcribed text from stt, sends it to an LLM endpoint, and passes the response text to tts for synthesis. Whether that LLM endpoint is a local model or a hosted API is entirely your call agent’s choice.

    Connecting a SIP Trunk

    Asterisk needs a SIP trunk to actually receive calls from the public phone network. Providers like Twilio Elastic SIP Trunking or Telnyx work well here — you register your Asterisk server’s public IP or domain with the provider and configure a pjsip.conf endpoint:

    [trunk]
    type=endpoint
    transport=transport-udp
    context=from-trunk
    disallow=all
    allow=ulaw
    allow=alaw
    outer_call = yes
    
    [trunk-auth]
    type=auth
    auth_type=userpass
    username=your_sip_username
    password=your_sip_password
    
    [trunk-registration]
    type=registration
    transport=transport-udp
    outer_call = yes
    server_uri=sip:your-provider.com
    client_uri=sip:[email protected]

    Open UDP port 5060 and the RTP range (10000-10100 in the compose file above) on your firewall. If you’re behind a cloud provider’s security groups, make sure those match your ufw or iptables rules too — a mismatch here is the most common reason calls connect but produce no audio.

    sudo ufw allow 5060/udp
    sudo ufw allow 10000:10100/udp

    Reasoning and Guardrails in the Agent Layer

    The LLM layer is where most of the actual “intelligence” of an AI call agent lives, but it’s also where things go wrong if you don’t constrain it. A phone call is real-time and irreversible — unlike a chat interface, the caller can’t scroll back and re-read a bad answer. A few practices that matter in production:

  • Keep responses short. Long LLM outputs sound unnatural and increase perceived latency once run through TTS.
  • Set a hard timeout (2-3 seconds) on the LLM call and fall back to a canned response if it’s exceeded.
  • Log every transcript and response pair for review — this is also how you catch hallucinated answers before a customer complains.
  • Use function calling to hand off to a real human or a scripted flow for anything involving payments, account changes, or legal commitments.
  • Rate-limit inbound calls per phone number to reduce abuse from robocall probing.
  • If you’re already running a monitoring stack for your Docker services, add the call agent containers to it. Call latency and STT/TTS failure rates are exactly the kind of metrics you want alerting on before a customer notices dropped audio.

    Hosting and Reliability Considerations

    A phone system has different uptime expectations than a blog or internal dashboard — people expect the phone to just work. A few infrastructure choices make a real difference:

  • Run the stack on a VPS with a dedicated public IP so SIP registration stays stable
  • Use a provider with good network peering to your telephony vendor to minimize jitter
  • Put a monitoring agent in front of your container health checks so a crashed STT container doesn’t silently kill inbound audio
  • Consider running Asterisk behind Cloudflare for DDoS protection on any web-facing management UI, though the SIP/RTP traffic itself typically bypasses Cloudflare’s proxy
  • For the actual compute, a provider like DigitalOcean or Hetzner gives you predictable pricing and enough headroom to run local STT/TTS models without per-minute billing surprises. If you want managed uptime monitoring on top of your call agent stack, BetterStack’s status pages and incident alerting are worth checking out too.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: Do I need my own LLM, or can I call a hosted API?
    A: Either works. A hosted API (OpenAI, Anthropic, etc.) is simpler to set up and requires less compute, but adds network latency and per-call cost. A locally hosted open-source model keeps everything on your own server but needs more RAM/GPU and more maintenance.

    Q: How much does a self-hosted AI call agent cost compared to a SaaS platform?
    A: Roughly the cost of a mid-size VPS ($40-80/month) plus your telephony provider’s per-minute rate, versus SaaS platforms that often charge $0.10-0.30 per minute on top of a monthly platform fee. Break-even usually happens somewhere around a few thousand minutes a month.

    Q: What’s the biggest source of latency in this pipeline?
    A: Usually the STT step, especially if you’re running a larger Whisper model on CPU only. Use a smaller model or GPU acceleration if you notice callers experiencing awkward pauses.

    Q: Can this handle outbound calls too?
    A: Yes — Asterisk supports originating calls through the same SIP trunk. You’d trigger an outbound call via the Asterisk AMI/ARI interface and route the audio through the same STT/TTS/agent pipeline.

    Q: Is this legal for handling customer calls?
    A: Generally yes, but call recording laws vary by jurisdiction (some US states require two-party consent). Make sure your agent’s opening message discloses that the call may be recorded and handled by an automated system.

    Q: How do I scale this past one server?
    A: Put Asterisk instances behind a SIP load balancer (like Kamailio) and run the STT/TTS/agent containers as a horizontally scaled pool behind an internal queue. Most teams don’t need this until they’re well past a few hundred concurrent calls.

    Wrapping Up

    Building your own AI call agent takes more upfront work than signing up for a SaaS platform, but if you’re already comfortable with Docker and Linux administration, it’s a very achievable weekend project — and one that pays for itself quickly at any real call volume. Start with the four-container stack above, get a single SIP trunk talking to it, and iterate on the agent’s prompt and guardrails from there.

    If you’re setting this up alongside other self-hosted infrastructure, check out our guide on running a reverse proxy for internal services to keep your management interfaces off the public internet.

  • AI Customer Support Agents: Self-Host with Docker

    Self-Hosting AI Customer Support Agents: A Docker-Based DevOps Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Every SaaS vendor wants to sell you a subscription to their hosted AI customer support agents platform. For teams with a DevOps background, that’s often the wrong trade — you’re paying per-seat or per-conversation fees for infrastructure you could run yourself on a $12/month VPS. This guide walks through containerizing, deploying, and operating AI customer support agents on infrastructure you control, using Docker, Nginx, and a proper monitoring stack.

    We’ll cover architecture, a working docker-compose.yml, reverse proxy configuration, scaling patterns, and the security hardening you actually need before putting an AI agent in front of real customers.

    Why Self-Host AI Customer Support Agents

    Hosted AI support platforms (Intercom Fin, Zendesk AI, Ada) are fine if you want zero ops overhead and don’t mind the recurring bill. But if you’re already running containers in production, self-hosting gives you three things vendors can’t:

  • Data control — customer conversations, PII, and support transcripts never leave your infrastructure
  • Cost predictability — a fixed monthly VPS bill instead of per-resolution or per-agent-seat pricing that scales with support volume
  • Model flexibility — swap the underlying LLM (self-hosted via Ollama or an API like OpenAI/Anthropic) without vendor lock-in
  • The trade-off is you own uptime, patching, and scaling. If your team already manages a Docker Compose stack for other services, this isn’t much additional burden.

    When Self-Hosting Makes Sense

    Self-hosting AI customer support agents is a good fit when you have:

  • An existing DevOps team comfortable with containers and reverse proxies
  • Compliance requirements (HIPAA, GDPR) that make third-party data processing risky
  • Support volume high enough that per-conversation SaaS pricing outweighs a VPS bill
  • A need to integrate the agent tightly with internal systems (CRM, ticketing, billing) via direct database or API access
  • If you’re a two-person startup handling 50 tickets a month, just use a hosted tool. This guide is for teams past that stage.

    Architecture Overview

    A self-hosted AI customer support agent stack typically has four components: a frontend widget, an API backend that orchestrates LLM calls and retrieval, a vector database for knowledge-base grounding, and a reverse proxy handling TLS and routing.

    [Customer Browser]
          |
          v
    [Nginx Reverse Proxy] --TLS--
          |
          v
    [Agent API (FastAPI/Node)] ---> [LLM Provider or Ollama]
          |
          v
    [Vector DB (Qdrant/pgvector)] <--- [Knowledge Base Ingestion]

    This maps cleanly onto Docker Compose, with each layer as its own service.

    Docker Compose Stack

    Here’s a working baseline stack. It uses an open-source agent framework, Qdrant for retrieval-augmented generation, and Nginx as the edge.

    # docker-compose.yml
    version: "3.9"
    
    services:
      agent-api:
        build: ./agent-api
        container_name: support-agent-api
        restart: unless-stopped
        environment:
          - LLM_API_KEY=${LLM_API_KEY}
          - VECTOR_DB_URL=http://qdrant:6333
          - LOG_LEVEL=info
        depends_on:
          - qdrant
        networks:
          - agent-net
    
      qdrant:
        image: qdrant/qdrant:latest
        container_name: support-agent-vectordb
        restart: unless-stopped
        volumes:
          - qdrant-data:/qdrant/storage
        networks:
          - agent-net
    
      nginx:
        image: nginx:1.27-alpine
        container_name: support-agent-proxy
        restart: unless-stopped
        ports:
          - "443:443"
          - "80:80"
        volumes:
          - ./nginx/conf.d:/etc/nginx/conf.d:ro
          - ./certs:/etc/nginx/certs:ro
        depends_on:
          - agent-api
        networks:
          - agent-net
    
    volumes:
      qdrant-data:
    
    networks:
      agent-net:
        driver: bridge

    The agent-api service is your own image — typically a FastAPI or Node app wrapping an agent framework like LangChain or a custom orchestration layer that handles retrieval, tool calls, and escalation logic to a human.

    Step-by-Step Deployment

    Assuming you’ve provisioned a VPS (see our VPS provider comparison if you haven’t picked one), here’s the deployment sequence.

    1. Provision and harden the box

    ssh root@your-server-ip
    apt update && apt upgrade -y
    ufw allow OpenSSH
    ufw allow 80,443/tcp
    ufw enable

    2. Install Docker

    curl -fsSL https://get.docker.com | sh
    usermod -aG docker $USER

    3. Clone your agent repo and configure secrets

    git clone https://github.com/your-org/support-agent-stack.git
    cd support-agent-stack
    cp .env.example .env
    # edit .env with your LLM_API_KEY and domain

    4. Bring up the stack

    docker compose up -d --build
    docker compose ps

    5. Verify the API is reachable internally before exposing it

    docker compose exec agent-api curl -s http://localhost:8000/health

    Expect a {"status":"ok"} response before moving to TLS and public exposure.

    Nginx Reverse Proxy Configuration

    Terminate TLS at Nginx and proxy to the agent API. If you already have a reverse proxy setup for other services, add a new server block rather than a separate stack.

    # nginx/conf.d/agent.conf
    server {
        listen 443 ssl http2;
        server_name support.yourdomain.com;
    
        ssl_certificate     /etc/nginx/certs/fullchain.pem;
        ssl_certificate_key /etc/nginx/certs/privkey.pem;
    
        location /api/ {
            proxy_pass http://agent-api:8000/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_read_timeout 60s;
        }
    
        location / {
            proxy_pass http://agent-api:8000/widget/;
        }
    }
    
    server {
        listen 80;
        server_name support.yourdomain.com;
        return 301 https://$host$request_uri;
    }

    Use Certbot or your DNS provider’s ACME integration to issue certificates before this config goes live.

    Scaling and Monitoring

    Once traffic grows past a single VPS, scale horizontally rather than vertically:

  • Run multiple agent-api replicas behind Nginx using least_conn load balancing
  • Move Qdrant to a dedicated node once your knowledge base exceeds a few million vectors
  • Cache frequent retrieval queries in Redis to cut LLM token spend
  • Set explicit CPU/memory limits per container so one runaway conversation loop doesn’t starve the host
  •   agent-api:
        deploy:
          resources:
            limits:
              cpus: "1.0"
              memory: 1G

    You also need visibility into latency, error rates, and LLM API failures — an agent that silently stops responding is worse than no agent at all. A dedicated uptime and log monitoring service catches this faster than checking dashboards manually. We cover a full setup in our self-hosted monitoring stack guide, but for a customer-facing agent, an external monitoring service that alerts on downtime independent of your own infrastructure is worth the cost.

    Security Considerations

    AI customer support agents handle sensitive data — account details, order history, sometimes payment info. Treat this stack like any other production service handling PII:

  • Never let the agent execute arbitrary shell commands or unsandboxed code from LLM output
  • Rate-limit the public API endpoint to prevent prompt-injection abuse and cost-draining loops
  • Log full conversation transcripts for audit purposes, but encrypt them at rest
  • Rotate your LLM provider API keys on a schedule and store them in a secrets manager, not plaintext .env files in production
  • Run a periodic review of what the agent is allowed to say — add explicit guardrails against discussing pricing exceptions, refund policy overrides, or anything requiring human sign-off
  • If you’re running this on a VPS you also manage for other services, isolate the agent stack in its own Docker network and avoid exposing internal management ports publicly.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Do I need a GPU to self-host AI customer support agents?
    No, not if you’re calling an external LLM API (OpenAI, Anthropic, etc.) for inference. A GPU is only needed if you’re running the model itself locally via something like Ollama, in which case a mid-range GPU handles small-to-mid-sized models fine.

    How much does it cost to self-host versus using a SaaS platform?
    A typical stack runs comfortably on a $20–40/month VPS plus LLM API usage costs, which scale with conversation volume. Compare that against SaaS platforms charging $50–150+ per agent seat monthly, and the break-even point is usually within the first month for teams handling meaningful support volume.

    Can I integrate this with my existing ticketing system?
    Yes. Most agent frameworks support outbound webhooks or direct API integration, so you can have the agent escalate unresolved conversations directly into Zendesk, Freshdesk, or a custom ticketing system.

    What happens if the LLM provider has an outage?
    Build a fallback path: if the primary LLM API fails health checks, route to a secondary provider or a self-hosted smaller model, and always have a hard fallback to “connect me to a human” messaging so customers aren’t stuck.

    Is Docker Compose enough, or do I need Kubernetes?
    For most teams, Docker Compose on one or two VPS instances is enough. Move to Kubernetes only when you need multi-region redundancy or your support volume genuinely requires auto-scaling across many nodes.

    How do I prevent prompt injection attacks through customer messages?
    Sanitize and constrain the system prompt so it can’t be overridden by user input, restrict the agent’s tool access to read-only operations by default, and log flagged conversations for manual review when the agent’s confidence score is low.

    Wrapping Up

    Self-hosting AI customer support agents isn’t for every team, but if you already run containerized infrastructure, it’s a straightforward extension of your existing DevOps practices rather than a new discipline to learn. Start with the Compose stack above, get TLS and monitoring solid before going live, and iterate on the agent’s knowledge base as real conversations come in.

    If you’re choosing where to host this stack, DigitalOcean and Hetzner both offer VPS plans well-suited to this workload — Hetzner tends to win on raw price-per-core, DigitalOcean on ease of use and managed add-ons. For monitoring the agent’s uptime and API latency, BetterStack gives you incident alerting without building your own stack from scratch.

  • AI Agent Development Services: A DevOps Deployment Guide

    AI Agent Development Services: A DevOps Deployment Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Choosing between building an AI agent in-house and working with ai agent development services usually comes down to infrastructure readiness, not model quality. Most teams can prototype an agent in an afternoon; the hard part is deploying it reliably, giving it safe tool access, and keeping it running under real traffic. This guide walks through the deployment side of that decision from a DevOps perspective.

    What AI Agent Development Services Actually Deliver

    When people talk about ai agent development services, they usually mean one of three things: a consultancy that builds a custom agent for your stack, a platform that hosts agent orchestration for you, or a hybrid where a team builds the agent and hands you a container to run yourself. The distinction matters because it changes what you’re responsible for operationally.

    A pure SaaS agent platform handles scaling, logging, and uptime for you, but you lose control over data locality and often pay per-execution pricing that gets expensive at volume. A self-hosted deployment built by an external team gives you full infrastructure control but means your DevOps org owns patching, scaling, and incident response from day one. Most production deployments end up somewhere in between: a vendor-built agent core running on infrastructure your team manages directly.

    Evaluating a Vendor’s Deployment Artifacts

    Before signing off on any provider, ask for the actual deployment artifacts, not just a demo. A serious vendor should hand you a Dockerfile or container image, a documented set of environment variables, and a health-check endpoint. If a vendor can only offer you a hosted API key with no self-hosting path, that’s a legitimate option for prototyping, but it’s a poor fit if data residency or long-term cost control matters to your organization.

    Questions to Ask Before Committing

  • Does the agent run as a stateless service, or does it require persistent local storage between requests?
  • What LLM provider(s) does it call, and can you swap providers without a rewrite?
  • Are tool integrations (web search, code execution, database access) sandboxed, or do they run with the agent’s full permissions?
  • What’s the expected p95 latency per agent turn, and does that match your product’s UX requirements?
  • Core Infrastructure Requirements for AI Agents

    Regardless of who builds the agent, the deployment target looks similar across most stacks. You need a container runtime, a reverse proxy or gateway, a place to store conversation state or vector embeddings, and a queue if the agent performs long-running tool calls asynchronously.

    A minimal but production-viable stack looks like this:

    version: "3.9"
    services:
      agent:
        image: your-org/ai-agent:latest
        restart: unless-stopped
        environment:
          - LLM_API_KEY=${LLM_API_KEY}
          - VECTOR_DB_URL=postgres://agent:secret@vectordb:5432/agent
        ports:
          - "8080:8080"
        depends_on:
          - vectordb
      vectordb:
        image: pgvector/pgvector:pg16
        restart: unless-stopped
        environment:
          - POSTGRES_USER=agent
          - POSTGRES_PASSWORD=secret
          - POSTGRES_DB=agent
        volumes:
          - vector_data:/var/lib/postgresql/data
    volumes:
      vector_data:

    This is intentionally close to the pattern used for any containerized service with a database dependency. If you’re new to Compose in general, the fundamentals of variable handling and secure secrets management are covered in managing environment variables the right way and secure config management, both of which apply directly to agent deployments since API keys and model credentials should never be hardcoded into an image.

    Choosing Where to Run the Agent

    For teams evaluating ai agent development services against a build-it-yourself approach, the underlying compute decision is usually a VPS or a managed Kubernetes cluster. A single agent workload with moderate traffic runs comfortably on a mid-tier VPS. Providers like DigitalOcean or Hetzner are common choices for teams that want predictable pricing without committing to a full Kubernetes setup on day one. If you’re unfamiliar with the tradeoffs of running your own box versus a managed platform, the guide on unmanaged VPS hosting covers what you’re signing up for in terms of patching and monitoring responsibility.

    Orchestrating Agent Workflows With n8n

    A large share of practical agent deployments aren’t a single monolithic service — they’re a workflow engine coordinating LLM calls, tool invocations, and data lookups. n8n has become a popular choice here because it’s self-hostable, has native nodes for common LLM providers, and lets you visually wire together the steps an agent takes without writing a full backend from scratch.

    If you’re building or evaluating agent workflows this way, how to build AI agents with n8n is a good starting point for understanding the node-based approach, and the general n8n self-hosted installation guide covers the Docker setup you’ll need before wiring in any agent logic. For teams comparing n8n against other automation platforms as part of an ai agent development services decision, n8n vs Make is worth reading, since licensing and execution-pricing models differ significantly between the two.

    Managing Agent Templates and Reuse

    Once you have one agent workflow running reliably, the next problem is reuse across projects. n8n’s template system lets you export a working agent workflow and redeploy it with different credentials and prompts for a new use case. The n8n template guide walks through exporting, customizing, and redeploying workflows, which is directly applicable if your ai agent development services engagement produces more than one agent for your organization.

    Deploying Agents That Call External Tools

    The riskiest part of any agent deployment is tool access — giving the agent the ability to run code, query a database, or call an external API on the user’s behalf. From a DevOps standpoint, this should be treated the same way you’d treat any untrusted-input execution surface.

    A few practices that hold up in production:

  • Run tool execution in a separate, unprivileged container from the agent’s reasoning loop, so a compromised tool call can’t reach the agent’s credentials directly.
  • Log every tool invocation with its input and output, not just the final agent response, so incidents are debuggable after the fact.
  • Set hard timeouts and resource limits on any tool-execution container; an agent that loops indefinitely on a bad tool call will otherwise consume unbounded compute.
  • Never give an agent’s tool-execution identity broader database permissions than the specific queries it needs to run.
  • Debugging Agent Behavior in Production

    When an agent misbehaves in production — calling the wrong tool, looping, or returning an incoherent response — you need the same debugging discipline you’d apply to any distributed system. Structured logs per container, correlated by request ID across the agent, the tool-execution service, and the database, are non-negotiable. If your stack runs on Docker Compose, the practices in Docker Compose logs debugging guide apply directly: tail logs across all agent-related services simultaneously rather than jumping between terminals.

    Scaling and State Management

    Agents that maintain conversation memory or long-running context need a persistent store, and that store becomes the bottleneck as usage grows. Postgres with a vector extension (as shown in the Compose example above) is a common choice because it lets you store both structured session state and embeddings in one place, avoiding the operational overhead of running a separate vector database.

    If your agent deployment already includes Postgres for this purpose, the setup and tuning guidance in Postgres Docker Compose setup guide applies whether the database is backing a normal web app or an agent’s memory layer. For agents that need a fast ephemeral cache — rate-limit counters, short-term session state, or a queue for pending tool calls — Redis Docker Compose setup guide is a reasonable complement to the primary datastore.

    Rebuilding and Redeploying Without Downtime

    Agent behavior changes frequently — prompt tweaks, new tools, updated models — which means your deployment pipeline needs to support fast, safe rebuilds. Treat an agent’s prompt and tool configuration as versioned artifacts alongside the code, and rebuild the container on every change rather than mutating a running instance. The Docker Compose rebuild guide covers the mechanics of doing this without unnecessary downtime, which matters more for agents than typical services because a bad prompt deploy can silently degrade output quality without throwing errors.

    Monitoring and Observability for AI Agents

    Standard infrastructure monitoring (CPU, memory, request latency) still matters for agent deployments, but it’s not sufficient on its own. You also need to track model-specific signals: token usage per request, tool-call failure rates, and response latency broken out by whether the agent made an external tool call or answered directly from context.

    Set up alerting on token usage growth specifically — a prompt or tool bug that causes the agent to enter a retry loop can burn through LLM API budget far faster than it triggers a conventional infrastructure alert. Route these metrics through whatever observability stack you already run rather than building agent-specific tooling from scratch; consistency across services makes on-call rotations easier.


    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Do I need Kubernetes to deploy an AI agent in production?
    No. A single VPS running Docker Compose is sufficient for most agents until you have multiple replicas or need automated failover. Kubernetes adds real value once you’re running several agent services with independent scaling needs, but it’s not a prerequisite for a reliable first deployment.

    How is working with ai agent development services different from hiring a general software contractor?
    The main difference is domain expertise around prompt design, tool-calling safety, and LLM provider quirks — things a general contractor may not have hit before. Operationally, though, you should still expect the same deliverables: a working container, documented environment variables, and a deployment you can run and maintain yourself.

    Should agent infrastructure be isolated from the rest of my production stack?
    Generally yes, at least at the network level. Agents that call external LLM APIs and execute tools represent a different risk profile than a typical CRUD service, so isolating their containers and limiting their database permissions reduces blast radius if something goes wrong.

    What’s the biggest infrastructure mistake teams make when deploying their first agent?
    Treating the agent like a stateless API endpoint and skipping proper logging of tool calls and intermediate reasoning steps. When something goes wrong, that missing context is exactly what you need to debug the failure, and it’s expensive to add retroactively.

    Conclusion

    Deploying an AI agent well is mostly a standard DevOps problem wearing new terminology: containerize it, isolate its tool access, give it a durable state store, and monitor it properly. Whether you build the agent internally or bring in ai agent development services to handle the model and prompt logic, your team still owns the infrastructure it runs on. Start with a minimal Compose-based deployment, get logging and monitoring right early, and only move to more complex orchestration once you have a concrete scaling reason to do so. For more on the underlying container mechanics referenced throughout this guide, the official Docker Compose documentation and Kubernetes documentation are worth keeping bookmarked as your deployment matures.

  • AI Agents Directory: Build & Self-Host One with Docker

    How to Build a Self-Hosted AI Agents Directory with Docker

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    If you’ve tried to keep track of every autonomous agent, LangChain wrapper, and CLI-based assistant your team has spun up, you already know the problem: there’s no single source of truth. Spreadsheets rot. Slack threads get buried. A proper ai agents directory — a searchable, self-hosted catalog of every agent, its capabilities, endpoints, and owner — solves this, and you can stand one up in an afternoon with Docker.

    This guide walks through the architecture, the Docker Compose stack, the database schema, and the operational concerns (backups, monitoring, hardening) you need before pushing this to production.

    Why Build an AI Agents Directory in the First Place

    Most teams don’t set out to build a directory — they back into needing one. First it’s two agents: a support-ticket triager and a changelog summarizer. Then it’s a dozen: deployment bots, code review assistants, data pipeline monitors, customer-facing chat agents. Without a catalog, you end up with duplicate agents solving the same problem, orphaned agents nobody remembers building, and zero visibility into which agents have access to which credentials.

    A self-hosted directory gives you:

  • A single searchable index of every agent, tagged by capability, owner, and status
  • An audit trail of which agents have access to which APIs or secrets
  • A discovery layer so engineers stop reinventing agents that already exist
  • A place to document rate limits, cost per run, and failure modes
  • Public directories exist (browsing GitHub’s ai-agents topic is a good way to survey the landscape), but for internal, proprietary, or credential-bearing agents, self-hosting is non-negotiable.

    The Problem With Scattered Agent Listings

    Without a directory, agent metadata lives in README files, internal wikis, and tribal knowledge. When someone leaves the team, that knowledge often leaves with them. A directory forces structure: every agent needs a name, an owner, a status, and a description before it’s considered “registered.” That structure is what makes the catalog searchable and auditable instead of just another list.

    Core Architecture for a Self-Hosted Directory

    Keep the stack boring and operationally simple. You don’t need a microservices architecture for this — a single web app, a relational database, and a reverse proxy will comfortably handle thousands of agent records and moderate traffic.

    Choosing Your Stack

    A solid default:

  • App layer: Node.js (Express or Next.js) or Python (FastAPI) serving a REST API and a lightweight frontend
  • Database: PostgreSQL — relational, supports full-text search out of the box, and well-documented at postgresql.org
  • Reverse proxy: Nginx or Caddy for TLS termination
  • Container runtime: Docker Compose for single-host deployments; move to Swarm or Kubernetes only if you outgrow one VPS
  • If you haven’t settled on a proxy yet, our reverse proxy comparison guide breaks down when Caddy’s automatic HTTPS beats a hand-rolled Nginx config.

    Database Schema for Agent Metadata

    Start with a schema that captures the essentials without over-engineering:

    CREATE TABLE agents (
        id SERIAL PRIMARY KEY,
        name VARCHAR(255) NOT NULL UNIQUE,
        slug VARCHAR(255) NOT NULL UNIQUE,
        description TEXT NOT NULL,
        owner_email VARCHAR(255) NOT NULL,
        status VARCHAR(50) NOT NULL DEFAULT 'active', -- active, deprecated, experimental
        repo_url TEXT,
        endpoint_url TEXT,
        tags TEXT[] DEFAULT '{}',
        created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
        updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
    );
    
    CREATE INDEX idx_agents_tags ON agents USING GIN (tags);
    CREATE INDEX idx_agents_search ON agents USING GIN (to_tsvector('english', name || ' ' || description));

    The GIN index on the tsvector column is what makes full-text search fast without bolting on Elasticsearch. For a directory of a few thousand entries, Postgres full-text search is plenty — don’t reach for a dedicated search engine until you actually need it.

    Deploying the Stack with Docker Compose

    Here’s a minimal but production-shaped Compose file: app, database, and reverse proxy, all networked together with named volumes for persistence.

    docker-compose.yml Walkthrough

    version: "3.9"
    
    services:
      db:
        image: postgres:16-alpine
        restart: unless-stopped
        environment:
          POSTGRES_DB: agents_directory
          POSTGRES_USER: ${DB_USER}
          POSTGRES_PASSWORD: ${DB_PASSWORD}
        volumes:
          - pgdata:/var/lib/postgresql/data
          - ./schema.sql:/docker-entrypoint-initdb.d/schema.sql:ro
        networks:
          - internal
    
      app:
        build: ./app
        restart: unless-stopped
        environment:
          DATABASE_URL: postgres://${DB_USER}:${DB_PASSWORD}@db:5432/agents_directory
          NODE_ENV: production
        depends_on:
          - db
        networks:
          - internal
          - web
    
      proxy:
        image: caddy:2-alpine
        restart: unless-stopped
        ports:
          - "80:80"
          - "443:443"
        volumes:
          - ./Caddyfile:/etc/caddy/Caddyfile:ro
          - caddy_data:/data
        depends_on:
          - app
        networks:
          - web
    
    volumes:
      pgdata:
      caddy_data:
    
    networks:
      internal:
      web:

    Notice the database sits only on the internal network — it’s never exposed to the web network or the host’s public interface. The app service bridges both networks, and only the proxy touches ports 80/443. This is the same segmentation pattern we cover in more depth in our Docker networking fundamentals guide.

    Environment Variables and Secrets

    Never bake credentials into the image. Use a .env file excluded from version control:

    # .env
    DB_USER=directory_admin
    DB_PASSWORD=$(openssl rand -base64 24)

    Generate the password once and store it in a secrets manager or your team’s password vault — not in Slack, not in a commit. Bring the stack up with:

    docker compose --env-file .env up -d

    Check that everything started cleanly:

    docker compose ps
    docker compose logs -f app

    Adding Search, Tags, and Filtering

    Once agents are in Postgres, exposing search is a single query using the tsvector index built earlier:

    SELECT id, name, description, tags
    FROM agents
    WHERE to_tsvector('english', name || ' ' || description) @@ plainto_tsquery('english', $1)
    ORDER BY updated_at DESC
    LIMIT 20;

    On the frontend, pair this with tag-based filtering (WHERE tags @> ARRAY[$1]) so users can narrow results to, say, deployment or customer-support agents. Keep the API simple: a GET /api/agents?search=&tag=&status= endpoint covers 90% of real usage without needing a query DSL.

    Monitoring, Backups, and Uptime

    A directory that goes down silently defeats the purpose — people will stop trusting it and fall back to spreadsheets. Two things matter here:

  • Uptime monitoring: A service like BetterStack will alert you the moment the directory or its API goes unreachable, before your team notices and starts asking questions in Slack.
  • Automated backups: Postgres data is the only thing that actually matters in this stack. Automate pg_dump on a cron schedule and ship the output somewhere off-host.
  • #!/usr/bin/env bash
    set -euo pipefail
    
    TIMESTAMP=$(date +%Y%m%d-%H%M%S)
    docker compose exec -T db pg_dump -U "$DB_USER" agents_directory | gzip > "backups/agents_directory_${TIMESTAMP}.sql.gz"
    
    # Keep the last 14 days of backups
    find backups/ -name "*.sql.gz" -mtime +14 -delete

    Wire that into a nightly cron job and sync the backups/ directory to object storage. If you’re not already backing up your other self-hosted tools, our VPS backup strategy guide covers the same pattern for other services.

    Hardening and Scaling in Production

    For a single-team internal tool, one modest VPS is enough. DigitalOcean droplets and Hetzner cloud instances both work well here — Hetzner tends to win on raw price-per-core if your team is EU-based or latency-insensitive, while DigitalOcean’s managed database add-on is worth considering once you outgrow a single-container Postgres instance.

    Basic hardening checklist before you expose this beyond localhost:

  • Put the directory behind your VPN or SSO proxy — don’t expose agent metadata (especially endpoint_url and owner emails) to the open internet
  • Enforce TLS via Caddy’s automatic HTTPS or a Let’s Encrypt cert in Nginx
  • Rotate the Postgres password and restrict pg_hba.conf to the internal Docker network only
  • Run docker scout or trivy against your app image periodically to catch known CVEs in base images
  • Rate-limit the search API to prevent it from being scraped wholesale
  • Once you have real traffic and want to track SEO performance for a public-facing version of the directory (some teams do publish a curated, non-sensitive subset), a tool like SE Ranking can help you monitor how discovery pages perform in search results.


    Recommended: Ready to put this into practice? SE Ranking is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Q: Do I need a vector database to build an AI agents directory?
    A: No. Unless you’re doing semantic search over thousands of long-form agent descriptions, Postgres full-text search with a GIN index is fast enough and much simpler to operate.

    Q: Should the directory expose agent credentials or API keys directly?
    A: No — store only references (e.g., a secrets-manager key name or vault path), never the actual secret value, in the directory database itself.

    Q: Can I run this on a single small VPS?
    A: Yes. A 2GB RAM droplet from DigitalOcean or a comparable Hetzner CX instance comfortably runs Postgres, the app, and Caddy for an internal directory serving a small-to-mid-size engineering team.

    Q: How do I keep the directory from going stale?
    A: Add a CI step that fails a pull request if a new agent is deployed without a corresponding directory entry — treat directory registration as a deployment requirement, not an afterthought.

    Q: What’s the difference between this and a public directory like the GitHub topic page?
    A: Public directories are for discovering third-party or open-source agents. A self-hosted directory tracks your own internal agents, including private endpoints, owners, and operational status that should never be public.

    Q: Is Docker Compose enough, or do I need Kubernetes?
    A: Compose is enough for the vast majority of internal tools like this. Move to Kubernetes only if you need multi-node redundancy or you’re already running K8s for everything else and don’t want a one-off Compose host.

    Wrapping Up

    A self-hosted AI agents directory doesn’t need to be complicated: Postgres for storage and search, a thin app layer for the API and UI, Caddy or Nginx for TLS, and a nightly backup job. The value isn’t in the tech stack — it’s in forcing every agent your team ships to have an owner, a description, and a status before it goes live. Start with the schema above, get it running with Docker Compose this week, and add search and tagging once real usage tells you what people actually search for.

  • AI Support Agents: Self-Hosted Docker Deployment Guide

    How to Deploy AI Support Agents on Your Own Infrastructure

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    AI support agents have moved past the novelty phase. Most teams evaluating them today aren’t asking “does this work,” they’re asking “can I run this on my own infrastructure without handing a third-party vendor my customer data.” This guide walks through a self-hosted deployment of AI support agents using Docker, a vector database for context retrieval, and a monitoring stack to keep the whole thing observable in production.

    We’ll build a stack that includes an LLM-backed agent service, a vector store for retrieval-augmented generation (RAG), a Postgres database for conversation logs, and reverse-proxy/TLS termination — all wired together with Docker Compose and ready to deploy on a VPS.

    Why Self-Host AI Support Agents

    Most SaaS helpdesk-AI products are just a thin UI wrapped around an LLM API call plus a vector database. You’re paying a markup for orchestration you can run yourself in an afternoon. Self-hosting gives you three concrete advantages:

  • Data control — customer conversations, tickets, and embeddings never leave infrastructure you own.
  • Cost predictability — you pay for compute and API tokens, not per-seat SaaS pricing that scales with support volume.
  • Customization — you can swap models, tune retrieval logic, and integrate directly with internal tools (Jira, Zendesk, internal APIs) without waiting on a vendor’s roadmap.
  • The tradeoff is operational: you own uptime, patching, and scaling. If your team already runs Docker in production, this is a manageable lift, not a new discipline.

    Core Architecture for Self-Hosted AI Support Agents

    A production-grade deployment of AI support agents typically needs four components:

    1. An agent orchestration service (built with something like LangChain or a lightweight custom FastAPI wrapper around an LLM API).
    2. A vector database for semantic search over your knowledge base (docs, past tickets, product manuals).
    3. A relational database for structured data — conversation history, escalation state, user metadata.
    4. A reverse proxy handling TLS and routing, since you’ll expose a webhook or chat widget endpoint publicly.

    Here’s a minimal docker-compose.yml that ties these together:

    version: "3.9"
    services:
      agent:
        build: ./agent
        environment:
          - OPENAI_API_KEY=${OPENAI_API_KEY}
          - VECTOR_DB_URL=http://qdrant:6333
          - POSTGRES_DSN=postgresql://agent:agent@postgres:5432/support
        depends_on:
          - qdrant
          - postgres
        restart: unless-stopped
    
      qdrant:
        image: qdrant/qdrant:latest
        volumes:
          - qdrant_data:/qdrant/storage
        restart: unless-stopped
    
      postgres:
        image: postgres:16-alpine
        environment:
          - POSTGRES_USER=agent
          - POSTGRES_PASSWORD=agent
          - POSTGRES_DB=support
        volumes:
          - pg_data:/var/lib/postgresql/data
        restart: unless-stopped
    
      caddy:
        image: caddy:2-alpine
        ports:
          - "80:80"
          - "443:443"
        volumes:
          - ./Caddyfile:/etc/caddy/Caddyfile
          - caddy_data:/data
        depends_on:
          - agent
        restart: unless-stopped
    
    volumes:
      qdrant_data:
      pg_data:
      caddy_data:

    This is intentionally minimal — no message queue, no worker pool — but it’s enough to run a functioning agent that answers support queries backed by real context, not hallucinated guesses.

    Wiring Up Retrieval-Augmented Generation

    AI support agents are only as good as the context they retrieve. Without RAG, you get a generic chatbot that confidently makes things up about your product. The retrieval layer is what turns a general-purpose LLM into something that actually knows your documentation.

    A basic ingestion script for loading your knowledge base into Qdrant looks like this:

    from qdrant_client import QdrantClient
    from qdrant_client.models import VectorParams, Distance, PointStruct
    from openai import OpenAI
    import uuid
    
    client = QdrantClient(url="http://localhost:6333")
    openai = OpenAI()
    
    client.recreate_collection(
        collection_name="support_docs",
        vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
    )
    
    def embed(text: str) -> list[float]:
        response = openai.embeddings.create(model="text-embedding-3-small", input=text)
        return response.data[0].embedding
    
    docs = [
        "To reset a user's password, run scripts/reset_password.sh <user_id>.",
        "Streaming device pairing failures are usually resolved by re-flashing firmware v2.3+.",
    ]
    
    points = [
        PointStruct(id=str(uuid.uuid4()), vector=embed(doc), payload={"text": doc})
        for doc in docs
    ]
    
    client.upsert(collection_name="support_docs", points=points)

    At query time, the agent embeds the incoming support question, retrieves the nearest neighbors from Qdrant, and injects them into the LLM prompt as context. This single step is what separates a usable AI support agent from a liability.

    Deploying on a Production VPS

    Once the stack works locally, deploy it to a VPS with enough RAM to run Postgres, Qdrant, and your agent container comfortably — 4GB is a reasonable floor, 8GB if you expect concurrent conversations at scale. We’ve had good results running these stacks on DigitalOcean droplets and Hetzner dedicated instances, both of which offer predictable pricing that beats managed AI-support SaaS tiers once you cross a few hundred conversations a month.

    A basic deployment flow:

    # On the VPS
    git clone https://github.com/yourorg/ai-support-stack.git
    cd ai-support-stack
    cp .env.example .env
    # edit .env with your OPENAI_API_KEY and domain
    docker compose up -d
    docker compose logs -f agent

    If you’re already running other Docker workloads on the same host — as covered in our Docker Compose production guide — you can slot this stack in alongside existing services without conflict, as long as ports and volumes are namespaced properly.

    Monitoring and Alerting for AI Support Agents

    AI support agents fail in quiet ways: a stalled embedding job, an LLM API rate limit, a vector DB running out of disk. None of these throw a loud error a user notices immediately — they just degrade answer quality until someone complains. You need uptime and log monitoring from day one.

    We run BetterStack for uptime checks against the agent’s health endpoint and log aggregation across the Compose stack. A simple health check endpoint in your agent service:

    from fastapi import FastAPI
    
    app = FastAPI()
    
    @app.get("/health")
    def health():
        return {"status": "ok"}

    Point an external uptime monitor at /health and alert on 3 consecutive failures. Combine that with structured logging (JSON logs shipped to your monitoring provider) so you can trace a bad answer back to a specific retrieval failure or API timeout. For teams already using our self-hosted monitoring stack walkthrough, the same Prometheus/Grafana setup extends cleanly to cover agent latency and token usage metrics.

    Securing the Public-Facing Endpoint

    Your agent’s chat widget or webhook is public by definition, which makes it a target for prompt injection attempts and abuse. A few non-negotiable controls:

  • Rate-limit the public endpoint per IP and per session to prevent token-cost abuse.
  • Sanitize and cap input length before it reaches the LLM call.
  • Never let the agent execute arbitrary retrieved content as instructions — treat retrieved documents as data, not commands, in your prompt template.
  • Put Cloudflare in front of the public endpoint for DDoS protection, bot filtering, and WAF rules before traffic hits your VPS.
  • Store API keys and DB credentials in environment variables or a secrets manager, never committed to the repo.
  • These controls matter more for support agents than most web apps, because the attack surface includes the prompt itself — a malicious user can attempt to override your system prompt through crafted input, a class of attack commonly called prompt injection.

    Scaling Beyond a Single VPS

    As conversation volume grows, the first bottleneck is usually the LLM API rate limit, not your infrastructure. Before scaling horizontally, check your provider’s rate limits and consider a queue (Redis or RabbitMQ) in front of the agent to smooth bursts. Only add a load balancer and multiple agent replicas once you’ve confirmed the bottleneck is compute, not API throughput — premature horizontal scaling here just adds operational complexity without fixing the actual constraint.

    When you do scale out, the Postgres and Qdrant services should move to managed instances or dedicated hosts rather than living in the same Compose file as your stateless agent replicas. Keep state and compute separated so you can scale the agent tier independently.


    Recommended: Want to explore DigitalOcean yourself? DigitalOcean is a direct vendor link (not an affiliate/tracked link).

    FAQ

    Do AI support agents need a dedicated GPU?
    No. Most self-hosted AI support agents call a hosted LLM API (OpenAI, Anthropic, etc.) rather than running inference locally, so your server only needs CPU and RAM for the orchestration layer, vector database, and Postgres. A GPU is only necessary if you’re self-hosting the language model itself, which most support-agent deployments don’t need.

    How much does it cost to run AI support agents in-house versus SaaS?
    Costs scale with token usage on the LLM side plus a modest VPS bill (typically $20-80/month for the infrastructure). For most teams under a few thousand conversations a month, this comes in well below the per-seat pricing of commercial AI helpdesk platforms.

    Can AI support agents integrate with existing ticketing systems like Zendesk?
    Yes. Most integrations work through the ticketing platform’s webhook or REST API — the agent receives a new-ticket event, generates a response or suggested reply, and posts it back through the same API. This requires custom glue code but no changes to your core agent stack.

    What happens if the LLM API goes down?
    Build a fallback path: if the primary API call times out or errors, queue the request and return a “we’ll get back to you” holding message rather than leaving the user with a hung request. Some teams configure a secondary model provider as a failover.

    How do I prevent the agent from giving wrong answers confidently?
    Strong retrieval (RAG) grounded in verified documentation reduces this significantly, but you should also add a confidence threshold — if retrieval similarity scores are low, have the agent escalate to a human rather than guessing.

    Is self-hosting AI support agents worth it for a small team?
    If you’re already comfortable with Docker and have moderate support volume, yes — the setup described here takes a day or two to deploy and gives you full control. If you have no DevOps capacity at all, a managed SaaS product may be the faster starting point until volume justifies the switch.

    Wrapping Up

    Self-hosted AI support agents aren’t complicated once you break them into their real components: an orchestration layer, a vector store, structured storage, and a reverse proxy — all things you likely already run in some form. The Docker Compose stack above is a starting point, not a finished product; expect to iterate on prompt templates, retrieval tuning, and escalation logic as real conversations reveal edge cases your test data didn’t cover.

  • Customer Support AI Agents: Self-Hosted Setup Guide

    Customer Support AI Agents: The Complete Self-Hosted Deployment Guide

    Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.

    Every SaaS vendor wants to sell you a subscription-based customer support AI agent. If you run infrastructure yourself, you already know the pitch is usually a markup on an LLM API call wrapped in a dashboard. This guide skips the sales deck and shows you how to actually deploy customer support AI agents on your own servers — with Docker, an open-weight model, and a proper production stack around it.

    We’ll build a self-hosted support agent that can answer tickets, pull context from a knowledge base, and escalate to a human when it’s out of its depth — all running in containers you control.

    Why Self-Host Customer Support AI Agents

    The case for running your own stack instead of a hosted SaaS agent comes down to three things: cost at scale, data control, and vendor lock-in.

  • Cost: Per-seat or per-conversation pricing on hosted platforms scales linearly with ticket volume. A self-hosted stack scales with your compute bill, which is usually cheaper past a few thousand tickets a month.
  • Data control: Support tickets contain PII, account details, and sometimes payment data. Routing all of that through a third-party SaaS adds a compliance surface you have to audit.
  • Portability: When your agent logic lives in Docker containers instead of a vendor’s proprietary workflow builder, you can move providers, swap models, or fork the whole thing without a migration project.
  • None of this means SaaS agents are bad — for a five-person team with light ticket volume, paying for hosted convenience is the right call. This guide is for teams that already run infrastructure and want the agent to be just another service in the stack.

    What a Support Agent Stack Actually Needs

    A production-grade customer support AI agent isn’t just an LLM call. The minimum viable architecture has four pieces:

  • An inference backend serving the model (local model via Ollama, or a hosted API you call from your own service)
  • A retrieval layer that pulls relevant docs, past tickets, or account data into the prompt
  • An orchestration service that owns conversation state, escalation logic, and tool calls
  • A gateway/reverse proxy that handles TLS, rate limiting, and auth in front of all of it
  • We’ll containerize each of these separately so you can scale, update, or replace them independently.

    Building the Docker Compose Stack

    Here’s a working docker-compose.yml that stands up the whole thing: Ollama for local inference, a Postgres instance with pgvector for retrieval, a small FastAPI orchestrator, and Caddy as the reverse proxy.

    version: "3.9"
    
    services:
      ollama:
        image: ollama/ollama:latest
        volumes:
          - ollama_data:/root/.ollama
        deploy:
          resources:
            reservations:
              devices:
                - driver: nvidia
                  count: 1
                  capabilities: [gpu]
        restart: unless-stopped
    
      vectordb:
        image: pgvector/pgvector:pg16
        environment:
          POSTGRES_USER: support_agent
          POSTGRES_PASSWORD_FILE: /run/secrets/db_password
          POSTGRES_DB: support_kb
        volumes:
          - pgdata:/var/lib/postgresql/data
        secrets:
          - db_password
        restart: unless-stopped
    
      orchestrator:
        build: ./orchestrator
        environment:
          OLLAMA_HOST: http://ollama:11434
          DATABASE_URL: postgresql://support_agent@vectordb:5432/support_kb
        depends_on:
          - ollama
          - vectordb
        restart: unless-stopped
    
      caddy:
        image: caddy:2-alpine
        ports:
          - "443:443"
          - "80:80"
        volumes:
          - ./Caddyfile:/etc/caddy/Caddyfile
          - caddy_data:/data
        depends_on:
          - orchestrator
        restart: unless-stopped
    
    volumes:
      ollama_data:
      pgdata:
      caddy_data:
    
    secrets:
      db_password:
        file: ./secrets/db_password.txt

    Note the secrets block — never bake database passwords into environment variables in plain compose files that end up in git. If you haven’t standardized on a compose layout for production yet, our Docker Compose production guide covers secrets, health checks, and restart policies in more depth.

    Choosing an LLM Backend

    You have two real options for the inference layer: run an open-weight model locally with Ollama, or call a hosted API like OpenAI or Anthropic from your orchestrator. For customer support specifically, a mid-size instruction-tuned model (7B–14B parameters) handles ticket triage and FAQ-style answers well, and keeps latency and cost predictable.

    Pull and serve a model with Ollama like this:

    docker exec -it ollama ollama pull llama3.1:8b-instruct-q4_K_M
    docker exec -it ollama ollama run llama3.1:8b-instruct-q4_K_M "Summarize this ticket: customer can't reset password, gets 500 error"

    If your ticket volume is high or you need stronger reasoning for escalation decisions, route complex cases to a hosted model and keep the local model for first-pass triage. This hybrid approach is what most teams settle on after a few weeks in production — full local inference for cost control, with a fallback for edge cases.

    Retrieval: Grounding the Agent in Your Actual Docs

    An LLM without context will hallucinate policy details. The retrieval layer is what keeps answers grounded in your actual knowledge base, refund policy, and past resolved tickets. A minimal ingestion script using LangChain and pgvector looks like this:

    from langchain_community.vectorstores import PGVector
    from langchain_community.embeddings import OllamaEmbeddings
    from langchain.text_splitter import RecursiveCharacterTextSplitter
    
    embeddings = OllamaEmbeddings(model="nomic-embed-text", base_url="http://ollama:11434")
    
    splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100)
    docs = splitter.split_documents(load_kb_articles())  # your loader
    
    PGVector.from_documents(
        documents=docs,
        embedding=embeddings,
        connection_string="postgresql://support_agent@vectordb:5432/support_kb",
        collection_name="kb_articles",
    )

    Run this as a scheduled job (cron container, or a systemd timer if you’re managing the host directly) so the knowledge base stays current as you update docs.

    Orchestration and Escalation Logic

    The orchestrator is the part that turns “LLM plus documents” into an actual support agent. At minimum it needs to:

  • Retrieve relevant context for the incoming ticket
  • Decide whether the agent can answer confidently or should escalate to a human
  • Log every conversation turn for audit and QA
  • Expose a webhook endpoint your helpdesk (Zendesk, Freshdesk, or a custom ticket system) can call
  • A simplified FastAPI handler:

    from fastapi import FastAPI
    from pydantic import BaseModel
    
    app = FastAPI()
    
    class Ticket(BaseModel):
        ticket_id: str
        message: str
    
    @app.post("/agent/respond")
    async def respond(ticket: Ticket):
        context = retrieve_context(ticket.message)
        answer, confidence = generate_answer(ticket.message, context)
    
        if confidence < 0.6:
            escalate_to_human(ticket.ticket_id, reason="low confidence")
            return {"status": "escalated"}
    
        log_interaction(ticket.ticket_id, ticket.message, answer)
        return {"status": "answered", "reply": answer}

    The confidence threshold is the single most important tuning knob in the whole system. Set it too low and the agent gives wrong answers with false authority; set it too high and you’re paying for an LLM that escalates everything, defeating the point.

    Reverse Proxy, TLS, and Rate Limiting

    Don’t expose the orchestrator directly. Put Caddy or Nginx in front of it for TLS termination and basic abuse protection. A Caddyfile for this stack:

    support-agent.yourdomain.com {
        reverse_proxy orchestrator:8000
        rate_limit {
            zone dynamic {
                key {remote_host}
                events 30
                window 1m
            }
        }
    }

    Caddy handles Let’s Encrypt certificate issuance automatically, which is one less thing to manage compared to hand-rolling Nginx plus Certbot. If you’re weighing the two, see our comparison of Caddy vs Nginx as a reverse proxy for the tradeoffs.

    Monitoring the Agent in Production

    An AI agent that silently degrades is worse than one that’s obviously down — customers get bad answers instead of an error page. Track at minimum:

  • Response latency per request (p50/p95/p99)
  • Escalation rate over time (a sudden spike usually means a model or retrieval regression)
  • Token usage / GPU utilization if running local inference
  • Failed webhook deliveries to your helpdesk
  • Wiring Prometheus into the orchestrator with a /metrics endpoint and a Grafana dashboard gets you most of this in an afternoon. If you don’t already have a monitoring stack running, our self-hosted monitoring stack guide walks through the Prometheus/Grafana/Alertmanager setup end to end.

    Security Hardening Checklist

    Customer support agents touch account data by definition, so treat this stack like any other system handling PII:

  • Terminate TLS at the proxy, never serve the orchestrator over plain HTTP even internally
  • Store DB credentials as Docker secrets, not environment variables in compose files
  • Restrict the Ollama API port (11434) to the internal Docker network only — never publish it
  • Redact or hash PII before it’s written to logs
  • Rotate API keys for any hosted LLM fallback on a schedule, not manually when someone remembers
  • Our general Linux server hardening checklist applies directly to the host running these containers, on top of the container-specific points above.

    Scaling Beyond a Single VPS

    A single well-specced VPS handles a surprising amount of ticket volume — a 4-8 vCPU box with a GPU (or CPU-only inference on a smaller quantized model) can serve a few thousand conversations a day. Past that, scale the orchestrator horizontally behind Caddy’s load balancing, and keep Ollama on a dedicated GPU box since that’s the actual bottleneck.

    For the VPS layer itself, DigitalOcean droplets are a solid default if you want managed load balancers and easy horizontal scaling without babysitting bare metal. If you’re running the GPU inference box yourself and want better price-per-core on dedicated hardware, Hetzner is worth pricing out — their dedicated servers are considerably cheaper than cloud GPU instances for steady-state local inference. And once the stack is live, BetterStack is a straightforward option for uptime monitoring and incident alerting on the public-facing endpoint, separate from your internal Prometheus setup.

    Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.

    FAQ

    Do customer support AI agents need a GPU to run?
    Not strictly. Small quantized models (7B parameters, 4-bit quantization) run acceptably on CPU for low-to-moderate ticket volume, though response latency will be higher — expect several seconds per response instead of under a second. A GPU becomes worthwhile once you’re serving concurrent conversations at scale.

    How do I stop the agent from hallucinating policy details?
    Ground every answer in retrieved documents rather than the model’s parametric knowledge, and set the confidence threshold conservatively so uncertain answers escalate to a human instead of being sent as-is. Never let the agent answer billing or refund questions without a retrieved policy document backing the response.

    Can I connect this to Zendesk or Freshdesk instead of building my own ticket UI?
    Yes — both platforms support outbound webhooks and custom app integrations. Point your helpdesk’s webhook at the orchestrator’s /agent/respond endpoint and post the reply back through their API instead of building a separate frontend.

    Is self-hosting actually cheaper than a SaaS support AI tool?
    Usually, once you’re past a few thousand tickets a month. Below that volume, the engineering time to build and maintain the stack often costs more than a SaaS subscription would. Run the math on your actual ticket volume before committing.

    What happens if the local model goes down?
    Build a fallback path in the orchestrator that routes to a hosted API when the local Ollama container is unreachable, and alert on it immediately via your monitoring stack. Treat local inference availability the same way you’d treat any other single point of failure.

    How do I handle multiple languages?
    Most modern open-weight instruction-tuned models handle major languages reasonably well out of the box. For anything beyond the top 10-15 languages, test explicitly with real sample tickets before trusting the agent — quality drops off fast outside high-resource languages.

    Wrapping Up

    Self-hosting customer support AI agents isn’t more complicated than any other production service you already run — it’s Docker containers, a reverse proxy, a database, and sane monitoring. The parts that actually require care are the retrieval grounding and the escalation threshold, since those determine whether customers get good answers or confidently wrong ones. Start with the compose stack above, tune the confidence threshold against real historical tickets, and scale the pieces independently as volume grows.