Essential cookies keep the site working and remember this choice. They are always on. Read the cookie policy for every cookie we use.

Optional cookies

Engineering10 min read

How to build an AI customer support agent

How to build an AI customer support agent that answers email, end to end. Gmail, FastAPI, Supabase, Pydantic AI with Claude, evals, Docker deployment and an admin dashboard.

An AI customer support agent that works in production is mostly not AI. It is an inbox connection, a database that records every action, a few hard guards, and plain Python that decides what gets sent. The model drafts. Code decides.

This is the system we use at Datalumina. We have automated the full customer support of six companies with it, and I rebuilt it from scratch in a three-hour course. A customer emails a question, the agent reads the thread, looks up the answer in the knowledge base and replies. Anything it isn't sure about goes to a human. The course covers the full stack, from the Gmail API to a deployed server and an admin dashboard.

The model proposes, code decides

The course uses a sample e-commerce shop that sells paddle boards and shelving, modeled on a real client that sells on Amazon. Those products generate the questions real support inboxes get. Sizing, missing parts, damaged panels, slow leaks.

Here is how the pieces fit together:

ComponentJobBuilt with
PollerChecks the inbox on an intervalPython loop, Gmail API
DatabaseStores events, tickets, messages and the knowledge baseSupabase (Postgres), SQL migrations, psycopg
GuardsSkip mail the agent must never answerPlain Python
ClassifierSorts the email and flags when a human is neededPydantic AI, Claude Haiku
Support agentDrafts the reply using knowledge base toolsPydantic AI, Claude Sonnet
Decision layerSends the reply or escalatesPlain Python
API and dashboardLets the client see tickets and edit the knowledge baseFastAPI, React, shadcn/ui
DeploymentRuns the API and poller on a serverDocker Compose on a VPS

I use Gmail so anyone can follow along at no cost. Real companies usually run a ticketing system like Zendesk, Help Scout or Gorgias, and swapping the Gmail connector for one of those is straightforward. The same goes for the model provider. I use Anthropic, but any capable LLM works.

Connect the inbox and log every event

Getting Gmail access is the slowest part of the whole build. API keys, permissions and OAuth tokens are where most of the time goes, because you can't one-shot this part with a coding agent. You create a project, enable the Gmail API, configure the consent screen, add the gmail.modify scope (read, send and label), and generate a refresh token with a small script. Keep in mind that tokens in testing mode expire after seven days.

The poller is a simple loop that checks for new messages, handles each one, and sleeps. Webhooks are the better production practice because they react instantly, but Gmail webhooks need a public HTTPS address and a domain. Polling avoids all of that. I poll every 30 seconds in the demo. In production, every 10 minutes is usually fine.

Every message and every action gets stored. The database has four tables, events, tickets, messages and knowledge_base, and it is the structure we use in all our customer support products. A ticket holds the conversation, and each email in the thread is a message under it. The events table is a ledger. It records what came in, what the system did and why. That pattern is the foundation for almost any AI automation project. Something happens in system X, you save it, you act on it, and you keep a record. Swap emails for invoices and the design stays the same.

I manage the schema with Supabase migrations in the repository. That keeps changes ordered, like Git for the database, and it means coding agents can read exactly what the database looks like.

Guards stop the agent from answering itself

Without guards, the agent replies to an email, sees its own reply as a new message, and replies again. So before any AI runs, Python checks a few skip conditions. It skips messages sent by us, messages from our own address, and auto-replies, newsletters and mailing lists. Skipped messages still get logged as an event with the reason.

You could try to handle this in a prompt. I want it to be physically impossible for the AI to answer these messages, and only code can promise that.

Classify with a small model, answer with a larger one

The AI part is a small pipeline. A smaller model classifies, a larger model answers with tools, and deterministic code makes the final decision.

Everything starts with strict Pydantic models. The classifier returns a section (paddle boards, shelving, returns and warranty, shipping or other), the language, a needs-human flag, a reason and an acknowledgement. The support agent returns a structured reply like this:

Python
from typing import Literal
 
from pydantic import BaseModel
 
 
class AgentReply(BaseModel):
    enough_information: bool
    promises_action: bool
    issue_type: str | None
    action: Literal["reply", "escalate"]
    reasoning: str
    reply: str

Structured output is what separates a fragile AI workflow from one you can control in code. In my experience, most of the work after the first prototype is this data modeling and the logic around it.

Both steps are Pydantic AI agents, which is my favorite lean library for working with LLMs in Python. Each one needs a model, instructions and an output type, and the output type matters most. The classifier runs on Claude Haiku. The support agent runs on Claude Sonnet and gets tools to list the knowledge base sections and read one.

The knowledge base is a set of markdown files (brand and tone, escalation rules, paddle boards, returns and warranty, shelving, shipping) split into sections and stored in Postgres. At this size you could paste it all into one prompt. Splitting it lets the agent read only what it needs, which saves tokens as the business grows. A paddle board question doesn't need the shelving docs. Brand and tone always goes into the system prompt.

I don't put everything into one big prompt for one powerful model. It would probably work at first. But you can't trace why it made a decision, it wastes tokens, and as products and customers grow it starts to hallucinate. When the agent struggles, add branching before you reach for a better model, for example a dedicated agent per product line. And some decisions don't need a model at all. Missing hardware for shelving means shipping a hardware kit. A wrong finish means an exchange. A few if statements handle that better than another paragraph in the system prompt.

Let code make the final call

The agent never gets direct control over sending. A decide function runs after the AI steps and escalates to a human when:

  • the email has an attachment, which in our client projects usually means a photo a person should look at
  • the classifier flags that a human is needed, for example an angry customer or a refund request
  • the agent itself chooses to escalate, or says it doesn't have enough information
  • the issue type is on the escalation list
  • the reply promises an action

That last rule comes from real launches. Agents answering a simple product question would add "I will also check with our support team" or "I will issue the refund", or say someone would call back. An AI agent should never promise an action it can't actually take, and that is very hard to prompt against, because customers are unpredictable. So the agent fills a promises_action field, and code escalates whenever it is true.

This decision layer is also how you go live safely. You can start with a single question type the agent is allowed to answer and escalate everything else. The support team sees the answers, gains confidence, and you widen the scope from there. That approach got our clients live in two to three weeks.

Stuck building prototypes?

Get our production stack

What you're missing is the architecture, evaluation, and judgment to ship a real project.

See Curriculum

Test the pipeline with evals

Sending test emails by hand is slow, and LLMs don't give the same output every run. Evals tell you whether the pipeline is any good, and whether a change makes it better or worse.

Because every step returns structured output, you can write tests against it. The course uses a golden dataset of emails, each with the expected section and the expected action (reply or escalate), and runs it with pydantic-evals. Each case checks two things, the section the classifier picked and the action the pipeline took.

The first run caught two problems. Some cases failed on transient API errors, and one email about a wrong shelf finish got classified as returns and warranty instead of shelving. The improvement loop is to change one thing (the classifier instructions, the agent instructions or a knowledge base entry) and run the whole set again. I let a coding agent fix both, with one warning. Don't overfit. If the agent pastes the failing example into the system prompt, the test passes and nothing generalizes. The fix it made was a general rule, that a problem with a specific product belongs to that product's section. After two rounds the set passed at 100%.

On a real project, the client's inbox gives you hundreds of tickets to turn into a much larger dataset. For the broader approach, read how to set up LLM evals.

Deploy it to a server you control

Deployment is the part I think is most valuable to learn, and it is a skill people look for when they hire. I run everything on a small VPS with Docker. One container serves the FastAPI app and another keeps the poller running, both defined in a Docker Compose file.

A few security choices matter more than the rest:

  • SSH key plus an IP allowlist. The firewall only accepts SSH on port 22 from your own IP address. Combined with the key, that works like two-factor authentication.
  • A read-only deploy key. Don't log in to your whole GitHub account on a server. A deploy key gives read access to one repository and nothing else.
  • Locked-down secrets. The .env file on the server is readable only by the deploy user.
  • Database network restrictions. In Supabase, allow database connections only from your own IP and the server's IP.

Updating is a git pull and docker compose up -d --build. Railway, Render and the big clouds do the same thing under the hood, and in my experience managed options often cost around 10 times more, while you also share your data with another company. The tradeoff is that you manage a server. If you want a production backend that is already wired up, GenAI Launchpad is the boilerplate we start our own client builds from.

Give the client an admin dashboard

Clients need to change rules, add products and fix knowledge base entries without emailing you for a new deployment. So the last step is an admin dashboard. React with shadcn/ui, built in the same repository and served by the same container. It shows the event log, opens each ticket with its full thread, can trigger an inbox check, and lets the client edit knowledge base entries that save straight to the database.

For internal tools like this, we usually build on shadcn/ui and add the client's logo and colors. The dashboard isn't exposed to the internet. You reach it through an SSH tunnel to localhost:8080, which avoids opening a public port. A public URL needs a domain and HTTPS, and that is the natural next step after the course.

Next step

For how these layers connect beyond a single build, start with how to build production AI systems.

If you want to go further than one project, the GenAI Accelerator covers AI system design, design patterns, RAG, monitoring and deployment with domains and HTTPS in depth. And almost any business with a manual support process could use what you build here. If you want to turn projects like this into client work, the Data Freelancer program is built for that.

FAQ

What is an AI customer support agent?

An AI customer support agent reads incoming customer messages, finds the answer in a knowledge base, and replies or hands the conversation to a person. In a production setup, the model drafts the reply and deterministic code decides whether it is sent or escalated.

Should an AI support agent send replies automatically?

Only for the question types you have tested and trust. Start by letting it answer one or two types and escalate everything else, then widen the scope as the support team gains confidence. Never let it send replies that promise refunds, callbacks or other actions it can't take.

Do I need vector search for a support knowledge base?

Not always. In this build, the knowledge base is split into sections in Postgres and the agent reads the sections it needs through tools. For a knowledge base of this size that is enough. For much larger ones, a RAG setup with vector or hybrid search is the next step.

Which models should I use for a customer support agent?

I use a small, fast model to classify each email and a larger model to draft the reply. In this build that is Claude Haiku for classification and Claude Sonnet for the agent. With Pydantic AI you can swap in another provider by changing the model string.

Can I use this with Zendesk or another help desk instead of Gmail?

Yes. Gmail is used so anyone can follow along for free. The connector is one class in the code, so you can replace it with Zendesk, Help Scout, Gorgias or another ticketing system while the rest of the pipeline stays the same.

Written by

Dave Ebbelaar

Dave Ebbelaar

Senior AI Engineer

AI engineer and founder of Datalumina. Dave helps developers build production AI systems and turn technical skills into client work.