“Let’s leave it for the model to think about” is the most expensive
architecture decision you can make. An LLM reasons about language; it does not reason with
guarantees about rules, arithmetic, temporality or constraints. And the
difference between leaving everything in the model and dividing the reasoning is the
difference between an agent that seems reliable and one that is.
Introduction
We have spent the whole series building layers: knowledge, memory, context.
And there is a question that supports all of them and that nobody wants to answer,
because the answer is uncomfortable: who reasons?
Because when an agent makes a decision, there are two “machines” in the room: the
LLM (which reasons about language, with generalization and without guarantees) and the
infrastructure (which resolves rules, arithmetic, temporality and
constraints, with guarantees and without generalization). The temptation — and the
most common error — is to put everything in the LLM: “the model is very intelligent, let it
solve everything”.
Today we set the division, because it is one of the architecture decisions
most important in the whole series, and the one that most separates an agent that is
production from one that is demo.
The problem
A bank has deployed a fraud detection agent that “analyzes
transactions and detects what is suspicious”. In the demo, it detected the classic patterns
well. In production, two opposite and serious failures:
- False negatives due to numerical reasoning. A transaction of 9,999 €
was just below the 10,000 € threshold that the model “knew”. The
model let it pass because it “did not exceed the threshold”, without applying the rule
real, which is accumulative: three transactions of 9,999 € in a week do
exceed the aggregated threshold. The model does not add; it “approximates”. - False positives due to ignoring time. A trip abroad with three
charges in three days marked “suspicious”, without the model applying that the
customer had declared the trip (episodic memory) and that the card
was in “travel” mode. The model saw three charges in three days and
generalized “fraud”, ignoring the temporal and declarative context.
In both cases, the model was not “too dumb”. It had been asked to
make reasonings that are not its strength: exact arithmetic over
thresholds, accumulation over time, and application of conditional rules.
Things that an infrastructure resolves with guarantees and that an LLM resolves
with approximation.
The concept
The architecture decision is to divide the reasoning between two
executors with different capabilities:
What the LLM should reason about (and it does it well):
– Interpret language: understand the question, the document, the intention.
– Generalize: recognize unseen patterns, extrapolate from similar cases.
– Synthesize: combine heterogeneous information into a coherent answer.
– Formulate: draft, summarize, explain, translate between domains.
– Soft reasoning: probability judgments, “this looks odd”,
prioritization by relevance.
What the infrastructure should resolve (and it does it with guarantees):
– Business rules. “If the discount > 15 %, it requires the director’s
approval”. Deterministic, verifiable, auditable.
– Arithmetic and thresholds. Sums, totals, exact comparisons,
accumulations. The LLM approximates; the infrastructure calculates.
– Constraints and restrictions. “This endorsement is invalid if there is an
open claim”. Verification, not guessing (the ontology, day 4).
– Graph reasoning. Multi-hop traversals: “what depends on X?”. The
LLM does not traverse graphs; the infrastructure does (day 7).
– Temporal reasoning. Currency, expiration, accumulation over time,
“what is in force now”. The LLM has no clock nor reliable
timeline.
– Planning and state verification. “Has this step already been done?”.
State query, not the model’s memory.
The general principle: the LLM reasons about language and generalization; the
infrastructure resolves what demands exactness, determinism, guarantees and
audit. And the agent’s architecture is, to a large extent, the map of
“this the model does, this the infrastructure does”.
Why does it matter so much? Because the failures of the two executors are of
different nature, and only one of the two is acceptable in production:
- The LLM’s failure is approximation: it gives a plausible answer that is not
exact. It is a silent failure: it does not warn that it has made a mistake. - The infrastructure’s failure (poorly designed) is deterministic: it either
applies the rule or it does not. A deterministic failure is detectable, testable and
auditable.
In an environment where a badly made decision has a cost (a fraud that passes,
a discount outside policy, an invalid endorsement), the silent failure of the
LLM is unacceptable. That is why what demands exactness must leave the
LLM and enter the infrastructure.
Architecture
In the series diagram, reasoning is the layer that coordinates the
two executors:
AGENTS
↓
REASONING ← TODAY: LLM / infrastructure division
┌──────┴──────┐
↓ ↓
LLM INFRASTRUCTURE
(language, (rules, arithmetic,
generalization, constraints, graph,
synthesis) temporality, state)
↓
MEMORY + CONTEXT + KNOWLEDGE
The reasoning layer is not the LLM: it is the coordinator that decides, for
each part of the decision, who executes it. The LLM proposes; the
infrastructure verifies, calculates and restricts. And the result of that
collaboration is what the agent presents as a decision.
Use case
The bank, with its fraud agent. We break it down:
- Problem. The fraud agent gives false negatives (it does not add
accumulative thresholds) and false positives (it ignores the temporal and
declarative context), because all the reasoning is in the LLM. - Decision. Classify a transaction (or a pattern of transactions) as
fraudulent or legitimate, and propose the action (block, warn, let
pass). - Knowledge needed. The fraud thresholds and their
accumulative rules; the customer’s spending profile; the previous
declarations (trips); the rules by card type. - Data. Transactions, customer profile, declarations, fraud history
of the customer. - Relationships. Transaction → customer → profile; transaction → type → fraud rule
; trip declaration → covers → transactions at destination; transaction →
card → mode. - Context. The customer declared a trip; the card
is in “travel” mode; the transaction is at the declared destination; there are three transactions of
9,999 € in the week. - Memory. Episodic: “a trip to X from the 1st to the 10th was declared”; the customer’s history
is clean (no previous frauds). - Reasoning. Infrastructure: calculate the weekly accumulation
(3 × 9,999 = 29,997 € > aggregated threshold) → triggers the rule; check
that the trip transaction is covered by the declaration and the
“travel” mode → it is not fraud. LLM: synthesize “the pattern of three transactions
of 9,999 € is an attempt at structuring (evading the threshold); the trip’s
spending is legitimate due to the declaration” and draft the justification. - Action. Block/investigate the structuring pattern (rule
accumulative, infrastructure); let the trip spending pass (declared,
infrastructure); open the investigation with the justification (LLM). - Infrastructure. The division of reasoning: the infrastructure
resolves the arithmetic, the accumulative rule and the temporal verification;
the LLM interprets the pattern, generalizes (structuring) and drafts. Without the
division, the model approximated and failed at both extremes.
Trade-offs
- Cost. Dividing the reasoning requires building the
infrastructure part (rules engine, threshold calculator, verification of
temporality), which is additional work compared to “leaving it in the model”. - Complexity. Drawing the border “this is the LLM’s, this is the
infrastructure’s” is a design decision that has to be documented and
maintained. A badly drawn border produces failures in the gray zone. - Latency. Calls to the infrastructure (rules, graph, calculation)
add steps. But in risk domains, the latency of a
verification is cheap compared to the cost of a failure. - Maintenance. Business rules change (new threshold, new
constraint). The rules infrastructure has to be updated; the LLM
does not “learn” the change, you have to tell it. - Interpretability. The division improves interpretability: each
part of the decision has an executor and a verifiable result. But
it requires that both are registered (day 18). - Honest limitation: the border is not sharp. There are reasonings that are
half-LLM, half-infrastructure (a judgment that depends on a calculation). The
practice is to iterate: start with the most conservative division (more in the
infrastructure) and move to the LLM what is safe.
Implication for the company
For the CTO, the division of reasoning is the architecture decision that
defines the reliability of the agent: what remains in the LLM is approximated; what
leaves to the infrastructure is guaranteed. And in risk domains, that
decision is the difference between a system that can be audited and one that cannot.
For the CRO (risk) and compliance, the implication is direct: the rules
regulatory and of risk must live in the infrastructure, not in the
prompt. A regulator is not going to accept “the model decided” as an answer to
“why was this transaction blocked?”.
For top management, the message is one of control: an agent whose decision is
a traceable mix of “what the infrastructure calculated” and “what
the model interpreted” is an agent that can be explained. And an agent that can
be explained is an agent that can be deployed.
Conclusion
Reasoning is divided: the LLM reasons about language, generalization and
synthesis; the infrastructure resolves rules, arithmetic, constraints,
graph, temporality and state. The LLM’s failure is silent (approximation);
the infrastructure’s is deterministic (detectable and auditable). In
production, what demands exactness has to leave the model.
Today we know what an agent needs, how its knowledge, its
memory, its context and its reasoning are built. The question that remains, and it is the
natural one, is: with all those layers, what exactly is an enterprise
agent? Tomorrow we give it the complete anatomy.
Next article
Tomorrow: An Enterprise Agent Is Far More Than an LLM with Tools.
We will see the complete anatomy of an agent (LLM, memory, context,
knowledge, ontology, retrieval, tools, policies, identity, permissions,
observability, evaluation) and how those layers allow passing from a chatbot to
an agent.
Sources
- The distinction between neuro (LLM) and symbolic (rules, graph,
logic) reasoning is the core of the neuro-symbolic approach; see also
the content of Cirtra, “The Renaissance of Meaning” (2026). - Working definition of the ICE: see series, Day 1.
