In previous posts I showed how DSPy replaced Agno/LangGraph as an assistant's cognitive router, adding an output contract, a metric, and an improvement process on top of the LLM.
What I did not show is that the same system contains another router with a completely different architecture. And that, before building the custom one, I evaluated an open-source library that almost became part of the project.
This post is the comparison I wish I had read before making those decisions.
The starting point: routing is not a single problem#
When you say intent router, you may mean very different things:
Problem A: The user typed "I want to cancel my order". Is it support, a return, cancellation before shipping, or something out of scope? A decision between a few broad candidates, with per-intent thresholds and ambiguity logic.
Problem B: The user typed "men's Nike sneakers up to R$ 300 with free shipping and good ratings". It is a catalog search. But with which tool? With which parameters? Scope + extraction of 10+ structured filters from a free-form sentence.
Problem C: You are starting a project, need to route between 8 intents, and do not want to build anything from scratch. You want something functional in 20 minutes.
Each of these problems may call for a different router.
Router 1: a custom embedding-based Semantic Router#
What it is#
A router that embeds the user's query, calculates cosine similarity against matrices of positive examples per intent, applies penalties using negative examples, and runs a decision pipeline with configurable thresholds.
How it works in practice#
Enlarge ↗The scoring for each intent:
Enlarge ↗This solves Problem A: broad classification between intents with clear semantics. No LLM call, with 20-50ms latency.
The pairwise reranker: where domain knowledge enters#
For pairs of intents that are frequently confused, a heuristic reranker decides based on text patterns:
Enlarge ↗The pairwise reranker is domain knowledge encoded as a rule. The embedding does not distinguish "I want to see my order status" from "I want to see men's sneakers"; their cosine scores for busca_catalogo may be close. The rule resolves this deterministically and auditably.
CFG_PARAMS in Redis: hot reload without deployment#
Enlarge ↗When routing behavior needs adjustment in production (a new term starts confusing intents, or a threshold needs calibration), you update CFG_PARAMS in Redis. No new deployment, no recompilation.
The work nobody documents: threshold calibration#
The custom semantic router has a problem that only appears in production: thresholds are defined manually, and the decision about which value to use has no direct mathematical backing.
How do you find the right values? Trial and error with a set of test queries, observing where the router starts accepting queries it should not (threshold too low) or rejecting legitimate queries that fall back to the FAQ (threshold too high).
This is manual calibration. And its cost gets worse over time.
When the encoder changes, whether through a new embedding model, a new Azure OpenAI version, or provider migration, the score distribution changes. Thresholds that worked with do not work with . You need to recalibrate from scratch.
When the utterance catalog grows, adding more positive examples per intent and more semantic diversity, the distribution changes again.
The in Redis solves the operational adjustment without deployment. It does not solve the process of discovering the right value.
When to use it#
Broad classification between well-defined intents (5-20 intents)
Latency is critical (<50ms per request)
You need configuration hot reload without deployment
The domain has intent pairs that are confused in known, predictable ways
You have positive and negative examples per intent, but not enough labeled data for an LLM
Router 2: DSPy#
What it is#
An LLM-first router that uses a declarative Signature to classify intent and extract structured parameters in a single inference, compiled with supervised examples and an asymmetric evaluation metric.
What concretely sets it apart#
The custom semantic router's output is:
Enlarge ↗The DSPy router's output is:
Enlarge ↗The same input query, but a completely different output. DSPy does not merely classify: it extracts all the parameters the search tool needs to execute the query in Redis Stack.
Doing this with embeddings is not feasible. You cannot extract preco_max=300.0 from a cosine vector.
The asymmetric metric: where business priority becomes code#
Enlarge ↗BootstrapFewShot rejects any demo with the wrong scope, regardless of how many fields were correct. This cannot be implemented in an embedding-based semantic router, where the concept of scope as a zero-tolerance criterion does not exist in cosine similarity.
How DSPy addresses the calibration problem#
In DSPy, the concept of a threshold does not exist in the same way, because the decision is not “is the score above X?”. The decision is “what is the best given the text, history, and compiled demos?”.
What acts as calibration is the metric itself together with BootstrapFewShot. By selecting which demos to include in the compiled prompt, the optimizer is implicitly calibrating the router's behavior to the domain, without you explicitly setting a threshold.
Enlarge ↗If you add new examples to the dataset and recompile, the model recalibrates with the new data. The limitation: recompiling incurs LLM costs. It is not an operation you perform through hot reload in production. It is an offline, controlled process with validation before deployment.
Coercion: protection against LLM unpredictability#
Enlarge ↗The semantic router never needs this. Embedding + dot product always returns a float. An LLM may return markdown, malformed JSON, or free text.
When to use it#
Routing requires structured parameter extraction in addition to intent classification
You have a supervised dataset (50+ examples already help)
You need portability between models (switching GPT-4o for Claude without rewriting)
The router output needs a verifiable, versionable contract
Latency of 2-3s per call is acceptable
Router 3: Aurélio AI semantic-router (evaluated, not implemented)#
Before building the custom router, I evaluated the open-source library from Aurelio AI, the group that includes part of the original semantic-router team. The API is clean and the mental model is the same: utterances per route, configurable encoder, cosine similarity.
Enlarge ↗The library works. What led me to build the custom version were three gaps specific to my use case:
No configurable pairwise reranker: for intent pairs confused in the domain ("
busca_catalogo" versus "detalhe_produto" when the user mentions a specific product), there is no structured mechanism to inject tie-breaking logic. Dynamic Routes (with an LLM) are the alternative, but add latency I did not want in the critical path.No configuration hot reload: to calibrate thresholds, add FAQ force terms, or adjust ambiguous-intent behavior in production, I would need a new deployment.
CFG_PARAMSin Redis was a project requirement.Reduced visibility into the decision pipeline: the topk with scores for all candidates, the pairwise reranker and its rule, the
disambiguation_rulein the structured log... these are important for debugging production degradation. The library returns RouteChoice withsimilarity_score. Sufficient for many cases, but not for the level of auditability I needed.
What the library does better: automatic threshold calibration#
The area where Aurélio AI has a concrete advantage over the custom version is precisely the calibration problem discussed above. The library has an automatic fitting method using a labeled dataset:
Enlarge ↗Fitting uses the similarity scores themselves to find the optimal cutoff per intent, without manual trial and error. When the encoder changes, you run fitting again on the same dataset and the thresholds recalibrate automatically.
The limitation: fitting is only as good as the dataset. If it does not cover the domain's edge cases, thresholds will be optimal for what you tested and silently poor for what you did not.
For my case, this benefit did not offset the other three gaps. For a project without a hot-reload requirement and with semantically distinct intents, it may be enough.
When it makes sense#
Rapid adoption with simple, semantically distinct intents
Low volume and few confusing intent pairs
Automatic threshold calibration through a dataset takes priority over pipeline control
A small team without the capacity to maintain a custom routing pipeline
Dynamic Routes are sufficient for edge cases
The calibration comparison, side by side#
The threshold calibration problem appears differently in the three routers. It is worth comparing them side by side:
Enlarge ↗The custom semantic router puts the calibration work on the engineer. Aurélio AI automates it, but still depends on the quality of the training dataset. DSPy eliminates it as an explicit problem but turns it into another: maintaining a supervised dataset and a recompilation process.
None of the three is maintenance-free. The difference is where the work sits.
The comparison table#
Enlarge ↗How the two coexist in production#
In the assistant, the two routers exist in different layers:
Layer 1: custom semantic router in the supervisor: decides whether the message concerns catalog search, product details, support, order tracking, or something out of scope. Broad classification, no LLM, extremely low latency.
Layer 2: DSPy router inside the search subgraph: when Layer 1 routes to "busca_catalogo", DSPy decides which tool to call and with which parameters. This is where "men's Nike sneakers up to R$ 300 with free shipping" becomes:
Enlarge ↗The division is based on the problem, not preference. Layer 1 needs speed and hot reload. Layer 2 needs structured extraction with a verifiable contract.
The trade-offs the table does not capture#
Silent degradation: the custom semantic router degrades when utterances become outdated relative to users' actual vocabulary. You detect it through an increasing fallback rate. DSPy degrades when the provider's model is updated; you detect it by running the validation set before deployment. Aurélio AI degrades in the same way as the custom router, but you have to recalibrate.
Actual operating cost: the custom semantic router incurs an embedding cost per request (cheap and predictable). DSPy incurs embedding + LLM costs, but parameter extraction in the same inference eliminates a subsequent LLM call to parse the query; the net cost is sometimes comparable. Aurélio AI has the same cost as the custom router outside Dynamic Routes.
Knowledge transfer: the custom semantic router has many lines of decision-pipeline code. Each line is a decision with a reason. When someone new joins the project, this context needs to be transferred explicitly. DSPy has a compiled JSON and a dataset; the "why" is in the examples and weights. Aurélio AI has public documentation.
To summarize:#
Custom semantic router: speed, full pipeline control, and hot reload. The price is manual threshold calibration and high maintenance cost.
DSPy: when the routing decision needs to extract structured parameters, with a verifiable contract, portability between models, and implicit calibration through compilation.
Aurélio AI: when you want the semantic router mental model without building the pipeline from scratch, with automatic calibration through . Assess whether the absence of hot reload and a pairwise reranker is acceptable for your use case before choosing.