How I'd Build Customer Support AI: Routing, Escalation, and Hallucination Guardrails

A production customer support AI — intent routing, RAG over product docs, escalation triggers, tone compliance checking, and the guardrail architecture that catches the 3% of cases that can destroy trust.

Customer support AI has the lowest tolerance for hallucination of any LLM use case. A wrong answer about a refund policy, a billing error, or account access costs real money and real trust. The architecture reflects this: multiple gates, explicit uncertainty handling, and a human escalation path that's faster than your average hold time.

Intent Routing First

Not all support queries need an LLM. Account status, order tracking, and common FAQ answers can be resolved with structured lookups. Use a lightweight intent classifier (fine-tuned Llama 3.1 8B at <50ms) to route before any expensive inference: structured-lookup intents go to your API/database, open-ended intents go to the RAG+LLM pipeline.

• Tier 1 (structured): order status, account details, simple FAQ → rule-based or API lookup
• Tier 2 (RAG): policy questions, troubleshooting, multi-step processes → RAG + LLM
• Tier 3 (human): complaints, billing disputes, account compromise, VIP customers → immediate escalation

RAG Over Product Documentation

The retrieval layer for support needs metadata filtering that general search doesn't. Products have versions. Return policies have effective dates. A chunk from the 2023 return policy should not surface for a 2025 query. Metadata schema: product_version, effective_date, jurisdiction, document_type. Every query should include a metadata filter for current effective date.

Escalation Triggers

Escalation should be proactive, not just available. Hard triggers (always escalate): account compromise signals, legal threats, explicit escalation requests, health/safety mentions. Soft triggers (escalate if confidence below threshold): billing disputes over $X, repeated failed resolution attempts (3+ turns without resolution confirmation), any sentiment classifier score below -0.6.

The Hallucination Guardrail Architecture

Three layers of hallucination defense for support AI:

• 1. Retrieval grounding: every factual claim in the response must cite a retrieved chunk. Claims without a source citation are flagged and suppressed (NLI-based grounding check).
• 2. Confidence gating: if retrieval similarity scores are below threshold (no good match found), the system responds with 'I don't have that information' + escalation offer — never hallucinate an answer to fill the gap.
• 3. Output validation: policy-specific validators check for prohibited patterns (quoting wrong refund percentages, incorrect SLAs, wrong contact information).

Tone Compliance

An LLM that sounds robotic, passive-aggressive, or condescending destroys the product experience even when the answer is correct. Fine-tune on your best human agent responses to internalize tone. Add an output-stage tone classifier that scores for empathy, directness, and concision — flag responses that score below threshold for human review before sending.

Metrics That Matter

• Resolution rate: % of sessions that close without escalation (target by query tier)
• Escalation precision: of escalated sessions, % that genuinely needed human (low = false alarms wasting agent time)
• Hallucination rate: % of responses containing at least one ungrounded factual claim (target: <0.5%)
• CSAT delta: support AI CSAT vs. human agent CSAT — your system is succeeding when the gap is <5 points

• Intercom AI Support Architecture
• Zendesk AI Design Principles