An agent that acts and cannot be audited is not an asset: it is a risk
with an assistant’s face. Governance is not the final chapter of the project
nor the document that compliance signs: it is the property that makes it possible for
the agent to act. Without it, the agent cannot be deployed.
Introduction
Yesterday the agent started to act: it reserved a part, created a work
order, notified management. And with the action, the question that
everyone is going to ask appeared, and it is the correct question:
How do we know why an agent made a decision?
Because when an agent acts on the company’s systems, it is no longer an
experiment: it is an operational actor. And an operational actor has to be able to
account for itself. Not “approximately”, not “the model decided”, but
exactly: with what data, with what rules, with what context, and with what
authorization it acted.
That capacity to account for itself has a name, and it is today’s topic:
governance. And we are going to dismantle it not as a compliance chapter,
but as the property of the cognitive infrastructure that makes it
possible for the agent to exist.
The problem
A bank has deployed a risk agent that blocks transactions. One day,
it blocks a customer’s transaction. The customer complains. The
compliance team has to answer, in 48 hours, to the supervision:
- “Why was customer X’s transaction blocked on April 3?”
- “What rule was applied?”
- “What data was used?”
- “Who authorized the block (the agent, a human, both)?”
- “Is the rule that was applied in force?”
- “Can I see the complete chain of the decision?”
And the AI team answers: “The model decided it. It has access to the
customer’s data and to the risk rules.”
That answer is not enough. It is not a technology problem: it is that the
system was not designed to explain its decisions. The agent acts, but
it does not leave an auditable chain. And when a regulator (or a customer, or a
auditor) asks “why?”, the answer “the model decided it” is an
admission that the company does not control what its own system does.
In the European Union, the AI Act already demands, for high-risk
systems, the logging of operations and the capacity to trace the decisions. And it is not only
regulation: it is the condition of trust. A customer is not going to let a
system that cannot explain its decisions manage their money.
The concept
The governance of the cognitive infrastructure is not a set of
documents: it is a set of properties that the system has to have
so that its decisions are auditable, controllable and accountable. They are
eight, and today we dismantle them:
-
Provenance. Each piece of knowledge has a
verifiable origin: from what document, from what source, extracted by whom, with what
date. “This rule comes from policy P-2026, page 4, extracted on
1/1.” Without provenance, you cannot verify whether the knowledge is
correct or where it comes from. -
Lineage. The transformation chain of a data point from the
source to the decision. “The 10,000 € threshold comes from the
risk system, it passes through the knowledge layer and it is applied in rule R-7.”
Without lineage, you cannot track how a source data point was turned into
a decision. -
Ownership. Each piece of knowledge and each rule has
an owner: a person or a team responsible for its correctness and
update. Without ownership, there is no one to ask to correct a
bad data point or to update an obsolete rule. -
Freshness. Each knowledge has an
update date and a currency. “This policy was updated on 1/1 and is
in force until 31/12.” Without freshness, the agent applies obsolete rules
with the confidence of the in-force ones. -
Confidence. Each piece of knowledge has a
confidence level: how much the system trusts that it is correct. “This
supplier data comes from an unverified source, low confidence.” Without
confidence, the agent treats a dubious data point as a reliable one. -
Temporal validity. Each knowledge and each rule
has an interval of validity in time. “This constraint applied
from 2023 to 2025; from 2026 it is another.” Without temporal validity, the agent does
not know what rule applies now. -
Permissions. What each actor (human or
agent) can see and do, with the principle of least privilege. We saw it in Day 15 and 17;
in governance, it is the property that guarantees that the agent does not access
what is not theirs. -
Auditability + human oversight.
The capacity to reconstruct any decision (what context, what rules,
what action, who authorized) and the mechanism by which a human can
intervene, cancel or escalate. It is the culmination of the HITL (Day 17) as
a permanent property, not as a pilot phase.
The eight properties support each other: without provenance there is no lineage;
without ownership there is no freshness; without confidence there is no trust; without temporal
validity there is no currency; without permissions there is no control; without auditability
there is no oversight. And together, they produce the capacity that the bank’s
problem needed: reconstruct the complete chain of a decision.
The question that joins them
How do we know why an agent made a decision?
The answer, with the eight properties, is: “Because the decision has
provenance (where each data point comes from), lineage (how it was transformed),
ownership (who is responsible), freshness and temporal validity (which rule
was in force), confidence (how much is trusted in the data), permissions
(who could see and do) and auditability (the complete chain is
registered), and human oversight (a human could intervene and did it in this
step).”
That is the difference between “the model decided it” and a decision that can
be explained, audited and, if necessary, reverted.
Architecture
Governance is not a layer: it is a transversal property that crosses
all the layers of the ICE:
GOVERNANCE (provenance, lineage, ownership,
freshness, confidence, temporal validity,
permissions, auditability, human oversight)
┌──────────────────────────────────────────────┐
│ AGENTS · REASONING · CONTEXT · MEMORY │
│ KNOWLEDGE · ONTOLOGY · SEMANTICS · DATA │
└──────────────────────────────────────────────┘
Each layer produces decisions or uses knowledge, and each one
has to leave a trace (provenance, lineage) and be subject to control (permissions,
oversight). Governance is the “nervous system” that allows the ICE
to be accountable.
Use case
The bank, with its risk agent, now with governance. We break down
the same blocked transaction, but with the auditable chain:
- Problem. Customer X’s transaction was blocked and it cannot be
explained why. - Decision (to audit). The block of customer X’s transaction on
April 3. - Provenance. The data point “cash balance 48,000 €” comes from the
core banking system (verified source, high confidence), extracted on 3/4 at
10:12. The rule “block if withdrawal > 40,000 € in 24 h without a declared
origin” comes from the risk policy R-2026, section 4. - Lineage. The balance (core) → knowledge layer (customer X entity)
→ graph (customer X → transaction → withdrawal type) → rule R-7 (block) →
decision. - Ownership. The rule R-7 is owned by the compliance
team (owner: J. Pérez). The balance data point is owned by the core
banking team. - Freshness + temporal validity. The rule R-2026 was updated on 1/1/2026
and is in force (not obsolete). The balance data point is from the same day (fresh). - Confidence. The balance has high confidence (verified source). There are no
“declared origin” data (confidence: absent → the rule is applied
without exception). - Permissions. The risk agent can read balances and apply
block rules; it cannot unblock (that is for the compliance team). - Auditability + human oversight. The decision is registered:
transaction, rule R-7, data, context, and that the agent proposed the
block and the compliance supervisor confirmed it at 10:15. The
complete chain is reconstructible.
The answer to the supervision, now: “It was blocked because the transaction
exceeded the threshold of rule R-7 (policy R-2026, in force), with a
verified balance from the core (high confidence), without a declared origin. The agent
proposed it and the compliance supervisor confirmed it. The complete chain is
registered and reconstructible.” That can be defended.
Trade-offs
- Cost. The eight properties are additional work: you have to instrument
provenance, lineage, ownership, freshness in each layer. It is expensive, and it is
continuous, not one-off. - Complexity. Instrumenting the lineage across layers (graph, RAG,
memory, reasoning) is complex: each transformation has to leave
a trace. A partial lineage is worse than none, because it gives confidence in a
incomplete chain. - Latency. Registering provenance, lineage and auditability adds steps. In
risk domains, the latency of a registry is cheap compared to the cost of
not being able to audit. - Volume. An agent that acts a thousand times a day generates a thousand
auditable chains. The storage and the query of those chains is a
engineering problem. - Balance with autonomy. Too much human supervision slows
the agent down; too much autonomy makes it inauditable. The design has to
find the point where the supervision is attentive without being ritual. - Honest limitation: governance audits what the system did, not
whether the decision was the best. An auditable block can be a false
positive. Governance explains the decision; it does not guarantee that it was
correct. The correction of the false positives is for the lifecycle
(tomorrow).
Implication for the company
For the CRO and compliance, governance is what turns the agent from a
risk into an auditable asset: each decision can be explained, reconstructed and,
if necessary, reverted. And it is what a regulator is going to demand for
high-risk systems.
For the CISO, the governance properties (permissions, auditability,
oversight) are the operational defenses of the agent: without them, the agent is
an attack surface that acts without accounting for itself.
For the CDO, governance is the trust asset of the ICE: without
provenance, freshness and confidence, the knowledge is not reliable, and a
unreliable knowledge cannot be used to decide.
For top management, the message is one of responsibility: an agent that acts
and cannot be audited is a risk that the company does not control. Governance
is not the brake of the agent: it is what makes it deployable.
Conclusion
The governance of the cognitive infrastructure is not a chapter of
compliance: it is the set of properties (provenance, lineage, ownership,
freshness, confidence, temporal validity, permissions, auditability,
human oversight) that makes it possible for the agent to act and account for itself. And
the question that joins them — how do we know why an agent made a
decision? — is the one that separates a system that can be deployed from one that cannot.
Today the agent acts and can be audited. But there is a last problem that the
series has not touched: knowledge changes. The rules are revised, the
suppliers change, the data are updated. How do you keep alive a
cognitive infrastructure when the world it describes does not stop moving?
Tomorrow we see it: the knowledge lifecycle.
Next article
Tomorrow: Enterprise Knowledge Changes: How to Keep a Cognitive Infrastructure Alive. We will see the complete knowledge lifecycle:
incorporation, validation, update, versioning, correction,
enrichment, expiration, elimination and audit — and the
feedback loop that turns operation into new knowledge.
Sources
- (EU) Regulation 2024/1689 (European AI Act) — Logging and
traceability requirements of the decisions for high-risk AI systems
(articles on technical documentation and logging). - W3C PROV-O — Provenance model for data and knowledge.
- Working definition of the ICE: see series, Day 16.
