Skip to content

Agent Routing

Routing is how agents decide which path to take — classify the user's intent, route to the right tool or sub-agent, and hand off when needed. Good routing prevents the wrong agent from handling the wrong task.

Table of Contents

Installation / Setup

pip install langchain-openai langgraph

Core Syntax

Routing types

Type How it works Example
Intent classification LLM classifies input → routes "book flight" → travel agent
Keyword/rule-based Pattern matching → routes "SQL:" → database tool
Semantic routing Embedding similarity → routes Similar past queries → same handler
Dynamic routing Agent decides at runtime ReAct agent picks tools
Human gate Pause for human decision High-risk action → approval

LLM-based classification routing

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

def classify_intent(query):
    response = llm.invoke(f"""Classify this query into one category:
    - coding: programming questions
    - research: information lookup
    - analysis: data analysis
    - chat: general conversation

    Query: {query}
    Category:""")
    return response.content.strip().lower()

def route(query):
    intent = classify_intent(query)
    if intent == "coding":
        return code_agent(query)
    elif intent == "research":
        return research_agent(query)
    elif intent == "analysis":
        return analysis_agent(query)
    else:
        return chat_agent(query)

LangGraph conditional routing

from langgraph.graph import StateGraph, START, END
from typing import TypedDict

class State(TypedDict):
    query: str
    intent: str
    response: str

def classify(state: State):
    intent = classify_intent(state["query"])
    return {"intent": intent}

def route_intent(state: State):
    return state["intent"]

graph = StateGraph(State)
graph.add_node("classify", classify)
graph.add_node("code", code_agent)
graph.add_node("research", research_agent)
graph.add_node("chat", chat_agent)

graph.add_edge(START, "classify")
graph.add_conditional_edges("classify", route_intent, {
    "coding": "code",
    "research": "research",
    "chat": "chat",
})
graph.add_edge("code", END)
graph.add_edge("research", END)
graph.add_edge("chat", END)

Human-in-the-loop gate

from langgraph.types import Interrupt

def human_approval(state: State):
    """Pause for human approval before dangerous actions."""
    raise Interrupt(value={
        "action": state["proposed_action"],
        "reason": "This action requires human approval",
    })

# after human approves, resume with app.invoke(None, config)

Common Operations

Keyword-based routing

def keyword_route(query):
    q = query.lower()
    if any(w in q for w in ["sql", "database", "query", "table"]):
        return "database"
    elif any(w in q for w in ["search", "find", "lookup", "what is"]):
        return "search"
    elif any(w in q for w in ["code", "python", "function", "bug"]):
        return "code"
    else:
        return "chat"

Semantic routing

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")

# define route embeddings
routes = {
    "code": "programming, code, debug, function, python",
    "research": "search, find, lookup, information, what is",
    "analysis": "analyze, data, chart, graph, statistics",
}
route_embeddings = {k: model.encode(v) for k, v in routes.items()}

def semantic_route(query):
    q_emb = model.encode(query)
    scores = {k: np.dot(q_emb, v) / (np.linalg.norm(q_emb) * np.linalg.norm(v))
              for k, v in route_embeddings.items()}
    return max(scores, key=scores.get)

Escalation routing

def escalate_if_needed(query, confidence):
    """Escalate to human if confidence is low."""
    if confidence < 0.6:
        return "human_agent"
    return "ai_agent"

Practical Examples

Example 1 — Multi-department routing

def route_to_department(query):
    departments = {
        "billing": ["invoice", "payment", "charge", "refund"],
        "technical": ["bug", "error", "crash", "not working"],
        "sales": ["pricing", "demo", "trial", "plan"],
    }

    q = query.lower()
    for dept, keywords in departments.items():
        if any(kw in q for kw in keywords):
            return dept
    return "general"

Example 2 — Confidence-based routing

def route_with_confidence(query, llm):
    response = llm.invoke(
        f"""Classify this query and give confidence 0-1:
Query: {query}
Category: [code/research/chat]
Confidence:"""
    )
    lines = response.content.strip().split("\n")
    category = lines[0].strip()
    confidence = float(lines[1].strip().split(":")[-1])

    if confidence < 0.7:
        return "fallback_agent"
    return category

Common Patterns

Pattern: router with fallback

def safe_route(query, primary_router, fallback="general_agent"):
    try:
        route = primary_router(query)
        if route is None:
            return fallback
        return route
    except Exception:
        return fallback

Pattern: hierarchical routing

def hierarchical_route(query):
    # level 1: broad category
    category = classify_broad(query)    # "technical" / "business" / "general"

    # level 2: specific routing within category
    if category == "technical":
        return classify_technical(query)    # "code" / "debug" / "deploy"
    elif category == "business":
        return classify_business(query)     # "billing" / "sales" / "support"
    else:
        return "general_chat"

Pattern: routing with memory

def route_with_context(query, conversation_history):
    # check if continuing previous topic
    if conversation_history:
        last_topic = conversation_history[-1].get("topic")
        if is_followup(query, last_topic):
            return last_topic    # continue same route

    # new topic — classify fresh
    return classify_intent(query)

Best Practices

  • Classify first, act second — don't let the wrong agent handle the wrong task.
  • Use LLM classification for nuanced intents; keywords for speed.
  • Add confidence thresholds — route to fallback when uncertain.
  • Log routing decisions — helps debug misroutes.
  • Hierarchical routing for many categories — broad → specific.
  • Human gates for high-risk actions — never auto-execute destructive ops.
  • Fallback is mandatory — every route should have a default.

Common Mistakes

Mistake Fix
No fallback route Always have a default/general handler
Overlapping route definitions Make categories mutually exclusive
No confidence check Add threshold; route low-confidence to fallback
Routing without logging Log decisions for debugging
Ignoring follow-up context Check conversation history before re-classifying

Troubleshooting

Symptom Cause / Fix
Wrong agent handles query Classification is off — improve categories
Many queries hit fallback Categories too narrow — broaden them
Routing is slow Use keyword routing for speed; LLM for accuracy
Agent doesn't escalate Add confidence threshold check

Official Documentation

Last Updated

2026-09-06 — Initial version.