The Complete Pipeline: From User Text to the Formatted Response

Over the past few months, I have published separate posts about each component of the e-commerce conversational assistant: episodic memory, routers, DSPy, anaphora, Redis Stack, guardrails, observability, and QueryBuilder.

This post closes the loop by showing the complete path of a query: from the text the user types to the response they receive, passing through each component in its execution order.

Once again, I will use the e-commerce domain as an example. The architecture is the same one I have documented throughout the series.

The example query#

Turn 2, in a session that already had a question about women's sneakers in the previous turn.

Step 1 — Supervisor: macro-routing without an LLM#

The message arrives at the supervisor. Before any LLM, the embedding-based semantic router classifies the macro-intent:

Score 0.81, above the 0.72 threshold for busca_catalogo, unambiguous. The supervisor routes to the catalog-search subgraph without calling any LLM.

Latency for this step: ~35ms.

Step 2 — Setup node: business context in parallel#

The subgraph starts with setup_node. Using asyncio.gather, it loads everything the subgraph will need in parallel:

In a sequential system, these two calls would take ~300ms each. In parallel, they take as long as the slower call: ~320ms at P50.

The graph state leaves setup with user_preferences, browsing_history_summary, and account_context populated. If the client has bought sneakers before, that is in the context when the LLM generates the response.

Latency for this step: ~320ms.

Step 3 — Router node: anaphora + DSPy#

router_node is where the cognitive decision happens. But before reaching DSPy, the anaphora resolver checks whether the message has implicit references:

"men's Nike sneakers up to R$300 with free shipping and good ratings" has no pronouns and no implicit continuation. The resolver does not modify the message.

If the user had said "and this one with free shipping?", the resolver would have expanded it to "and the Women's Nike Air Force 1 with free shipping?" before passing it to DSPy.

Now DSPy:

The scope is busca_catalogo; router_node selects CatalogSearchTool and constructs the arguments from RouterOutput:

Latency for this step: ~2.1s (query embedding + DSPy LLM).

Step 4 — Tools node: FT.SEARCH in Redis Stack#

The tool receives the RouterOutput arguments and calls QueryBuilder:

Redis returns 8 products (9 found; 1 indicates has_more=True). Each document arrives with 5 fields (Slim projection):

The tool formats the output for the LLM:

The [HAS_MORE_PAGES=true] token in the tool output will be read by generate_node to determine whether it should prepare the pagination UX.

Latency for this step: ~45ms.

Step 5 — Generate node: DSPy Advisor with Refine#

generate_node assembles the context and calls CatalogAdvisorModule:

CatalogAdvisorModule uses dspy.Refine(N=3, threshold=1.0). On the first attempt, the LLM generates:

The reward function detects "great choice," which violates the qualitative-opinion rule. Score: 0.75. Refine generates again.

Second attempt:

Score: 1.0. Refine stops.

Latency for this step: ~3.6s (2 LLM calls).

Step 6 — Critique node: final validation#

critique_node receives the generated response and validates it:

Fast path active for scope="busca_catalogo". Synchronous checks in <5ms:

Strip premature PDF offer → none found
Quick regex (imperative) → passed
Missing disclaimer → no mandatory disclaimer; not applicable
Perspective check → passed

Latency for this step: <5ms.

The complete trace#

What each component decided#

The semantic router decided it was a catalog search, without an LLM, in 35ms.
The anaphora resolver decided the message had no implicit reference in <1ms.
The DSPy router decided the scope, tool, and 8 filter parameters in 2.1s.
The QueryBuilder transformed those parameters into a safe FT.SEARCH query in <1ms.
The Redis Stack executed the search and returned 5 fields × 8 products in 45ms.
The DSPy Refine generated the response, detected a violation, and corrected itself in 3.6s.
The critique_node confirmed that nothing slipped through in 5ms.

Each component has a single responsibility; none knows what the previous one did. The graph architecture ensures that they connect through state.

What cost the most, and where there is room for improvement#

DSPy dominates latency: 2.1s in the router + 3.6s in generate = 5.7s out of 6.1s total.

The DSPy router can only be accelerated with a faster model or the pagination fast path, which bypasses DSPy completely when the user asks for "more" results. Generate can be accelerated by reducing Refine's regeneration rate: if the Signature is more explicit about the opinion rule, the first attempt passes, saving ~1.8s.

Redis Stack is irrelevant to total latency: 45ms to search and return 8 paginated documents. This is where Fat/Slim/Hybrid modeling paid back its construction cost.

Closing the series#

This series covered the system end to end:

→ Fat/Slim/Hybrid in Redis Stack: how to model data so the LLM does not get lost

→ DSPy: prompts as compilable code: the 4 fundamental concepts

→ Agno → DSPy migration: what changes when the router is already an LLM

→ Anaphora: resolving it before the LLM, with EntityStack and regex

→ Three routers: comparing DSPy, a custom router, and Aurélio AI

→ Regulatory guardrails: critique_node with three defense layers

→ Observability with Langfuse: traces, metadata, and degradation detection

→ dspy.Refine: runtime self-correction with a reward function

→ QueryBuilder: Pydantic → safe FT.SEARCH

→ Complete pipeline: the synthesis view

Each article stands on its own. Together, they form the documentation of a real production architecture, with the trade-offs, silent bugs, and decisions that are not visible in the code.