m@berryhill: ~/berryhill.dev/posts/the-governance-gap-after-ai-extraction.md
~ homeposts/about.md
m@berryhill in ~/posts$ cat the-governance-gap-after-ai-extraction.md
---
title:  The Governance Gap After AI Extraction
date:   2026-07-29
topic:  ai-governance
read:   12 min
words:  2,541
slug:   the-governance-gap-after-ai-extraction
views:  live post
tags:   [ai-governance, knowledge-systems, data-provenance, agent-operations]
---
essay · long read

The Governance Gap After AI Extraction

An extraction model returns a candidate claim. A five-field governance record decides whether that claim can safely travel.

table of contents
  1. A claim is not yet a knowledge record
  2. Provenance is necessary, but it is not governance
  3. A bounded implementation from cybersecurity
  4. The post-extraction trust lifecycle
  5. One claim, four state changes
  6. The Claim Governance Record
  7. How much governance is enough?
  8. The operating test

A model reads a document, returns valid JSON, and passes the schema check.

The extraction worked.

That does not mean the resulting claim belongs in a knowledge base, a retrieval index, an analyst workflow, or another agent's context.

This is the hidden completion error in AI extraction systems: treating a well-formed assertion as accepted knowledge. The model completed one operation. The system has not yet decided whether the assertion is supported, admissible for a particular use, or still safe to reuse after its evidence changes.

Extraction creates a candidate claim. Governance decides whether that claim can travel.

The practical fix is not necessarily a large compliance layer or a knowledge graph. It is a claim-level record that can answer five questions: What is the assertion? What evidence supports it? How was it validated? What is its current operational state? What changed over time?

I call that artifact the Claim Governance Record.

A claim is not yet a knowledge record

Consider a pipeline that extracts a product limitation from a support document:

Candidate claim: Product X does not support regional failover.

The sentence may be copied correctly. The JSON may match the schema. The model may report high confidence. None of those facts answers the questions a downstream operator actually has:

  • Which exact source passage supports the claim?
  • Which version of the document did it come from?
  • Did a rule, comparison, or reviewer validate it?
  • Is it admissible for customer-facing answers, internal triage, or neither?
  • Has a newer document superseded it?

Model confidence describes the model's output, not the quality of the evidence or the permission to reuse the claim. Schema validity establishes structure, not trust.

That distinction becomes more important when extracted claims can trigger action. A low-consequence tagging suggestion may need only a source link and an expiry date. A claim used to block a deployment, answer a customer, or drive a security response needs stronger evidence, explicit validation, and a reversible admission decision.

Governance depth should rise with the consequence of downstream reuse. That is an operating recommendation, not a measured law.

Model output versus governed claimA side-by-side comparison. The left panel contains a candidate claim and model confidence but leaves reuse undecided. The right panel binds the same claim to evidence, validation, an operational trust state, and append-only history before downstream use.MODEL OUTPUTGOVERNED CLAIMCandidate claimStructured outputModel confidenceReuse is still undecidedbind + decideCandidate claimEvidenceValidationTrust stateHistoryAdmissible for a defined useExtraction creates the assertion. Governance determines whether it can travel.
A valid schema can tell you what the model returned. It cannot tell you whether the result is authorized to travel.

Provenance is necessary, but it is not governance

Provenance tells you where a claim came from and how it was produced. It can identify the source span, document version, extractor, configuration, run, and timestamp.

Governance answers a different question: Is this claim currently admissible for this defined use, on what grounds, and what has changed since that decision?

The distinction matters because a perfectly traceable claim can still be disputed, obsolete, or unauthorized for a particular downstream view.

Claim governance is also narrower than chain of custody. Chain of custody follows an artifact through a workflow: who created it, who handled it, and where it moved. Claim governance follows an individual assertion through validation, admission, dispute, supersession, and revocation after extraction.

One does not replace the other. They solve different operational problems.

A bounded implementation from cybersecurity

A recent preprint, TRACE-CTI: Auditable Post-Extraction Governance of TTP Claims with Knowledge Graphs, makes this boundary concrete in cybersecurity threat intelligence.[^trace]

TRACE-CTI is not a new extractor. It receives mappings produced by existing systems between threat-report text and MITRE ATT&CK techniques. It then preserves a boundary among model predictions, corroborating evidence, governed assertions, and the knowledge exposed to downstream applications.[^trace]

In that design, a prediction-derived assertion stays outside the trusted view until an explicit validation event satisfies the active policy. The system retains native evidence granularity, extraction provenance, versioned trust decisions, and non-destructive revocation history.[^trace]

The evaluation used two public cyber-threat-intelligence corpora containing 65 reports and 5,303 sentences. It showed that seven operational questions—covering evidence, producer provenance, trust, versions, dependencies, disagreement, and review queues—were answerable from the stored snapshot and audit queries.[^trace]

That is useful architecture evidence within a specific domain: stored answerability, not proof of better analyst decisions, lower review cost, safer production outcomes, or generalizability to every extraction system. The evaluation did not exercise analyst validation or execute an actual revocation batch. The paper also treats agreement as policy evidence rather than truth and establishes no universal support threshold.[^trace]

The knowledge graph is an implementation choice, not the operating requirement. The paper notes that enriched relational or event-sourced systems could encode the same lifecycle semantics.[^trace]

The post-extraction trust lifecycle

The broader lifecycle below is my operator synthesis, not a framework the paper claims for every domain:

  1. Extract. Produce a candidate assertion from a source.
  2. Bind evidence. Preserve the exact supporting span or artifact, its identity, version, and production context.
  3. Validate. Apply a documented rule, comparison, procedure, or human review. Record both the grounds and the outcome.
  4. Admit. Allow the claim into a defined downstream view under a defined policy and use.
  5. Revise, supersede, or revoke. Change the active state without deleting the evidence or the reason an earlier decision was made.

Admission is not a declaration of universal truth. It is a governance decision: this claim may be used in this context under this policy based on these grounds.

A trusted view is therefore a governed downstream presentation or query state. It is not a truth database.

Likewise, trust_state is an operational status—such as candidate, disputed, trusted, superseded, or revoked—not epistemic certainty.

Automation can collect evidence, run comparisons, detect conflicts, and route exceptions. It cannot make accountability disappear. The operator still owns the policy that determines what counts as sufficient validation and where the admitted claim may be used.

The post-extraction trust lifecycleFive boxes connected left to right show Extract, Bind evidence, Validate, Trusted view, and Revoke or supersede. A return line labeled preserved history runs from the final state back beneath the lifecycle.POST-EXTRACTION TRUST LIFECYCLEExtractCandidate claimBind evidenceSpan + versionValidateGrounds + outcomeTrusted viewDefined policy + useRevoke orsupersedePreserve evidence + historyAdmission is reversible. The record of why it happened is not erased.
Trust should be reversible without making the evidence trail disposable.

One claim, four state changes

Here is an illustrative cybersecurity transition assembled from the paper's mechanisms. It is not a reported incident from the evaluated dataset.

An extractor maps a sentence in a threat report to a specific MITRE ATT&CK technique. The initial record is candidate.

The system binds that mapping to the exact sentence or span, report identity, extractor configuration, run, retrieved context, and timestamp. A declared policy then checks the mapping against a gold annotation, corroboration from distinct setups, or an analyst decision.

If the recorded grounds satisfy the policy, the assertion becomes trusted inside a defined threat-intelligence view. If they do not, it remains candidate, becomes disputed, or enters a review queue.

Later, suppose the source, extraction setup, or validation decision is found to be unreliable. The system appends a revocation event and removes the assertion from the current trusted view. It does not erase the old evidence or pretend the previous admission never happened.

The history still explains why the assertion was formerly admitted, when its status changed, and what downstream records may depend on it.

That final property is easy to underestimate. Deleting a bad claim cleans the current view but destroys the explanation an operator needs to investigate prior decisions.

The Claim Governance Record

The lifecycle can be implemented with a graph, an event stream, or a carefully designed relational record. The representation can vary. These five fields should survive:

The five-field Claim Governance RecordA numbered vertical list defines five required fields. Claim records the reusable assertion. Evidence records exact support and source version. Validation records grounds and outcome. Trust state records scoped operational status. History records append-only changes and decisions.CLAIM GOVERNANCE RECORDFive fields that survive the implementation1ClaimThe assertion at a granularity that can be admitted, disputed, or revoked.2EvidenceThe exact support, source identity, version, and relevant production context.3ValidationThe actor, rule, comparison, grounds, outcome, and limits of the check.4Trust stateThe current status for a defined policy, downstream view, and use.5HistoryThe append-only evidence, decisions, disputes, supersession, and revocation trail.
The storage model can change. These five operating facts cannot disappear with it.

1. Claim

The normalized assertion the system may reuse.

Record the claim at a granularity that can be admitted, disputed, or revoked independently. A paragraph-sized blob is difficult to govern if only one sentence becomes wrong.

2. Evidence

The exact source span, artifact, or observation supporting the claim.

Include source identity and version, not only a URL that may change. Preserve enough production context to distinguish what the source said from what the extractor inferred.

3. Validation

The actor, rule, comparison, or procedure used to evaluate the claim, plus the outcome and its limits.

“Validated” is not useful by itself. Was it checked against a reference set? Corroborated by independent sources? Reviewed by an analyst? Accepted because a low-risk policy permits single-source use?

4. Trust state

The claim's current operational status for a defined policy and use.

Candidate, disputed, trusted, superseded, and revoked are starting points. The important part is that the state is explicit, scoped, and queryable rather than implied by the claim's presence in a database.

5. History

An append-only lineage of evidence changes, validation events, admission decisions, disputes, supersession, and revocation.

History turns correction into an inspectable operation. It lets an operator ask not only “What do we believe now?” but “Why was this available yesterday, what changed, and which decisions used it?”

How much governance is enough?

Not every extraction needs the same machinery.

For temporary meeting-topic labels, a source reference and short expiry window may be enough. For a durable customer fact, preserve the exact source, validate against an authoritative system, scope admission to the relevant workflow, and retain supersession history. For a security claim that can drive containment, require stronger corroboration and route disagreement to a named reviewer.

Use the smallest representation that preserves the required controls.

A simple record may be sufficient when it can retain evidence, validation grounds, current state, and append-only history. A graph becomes valuable when operators need to traverse dependencies, compare overlapping assertions, or identify everything affected by a revoked source. Event sourcing becomes attractive when the state transitions themselves are the primary audit object.

Do not start by asking which database makes the architecture look sophisticated. Start by asking which decisions must remain explainable after the claim changes.

The operating test

A post-extraction system is not complete because the output parses.

It is complete enough to operate when it can answer:

  • Why is this claim available to this downstream user or agent?
  • Which exact evidence supports it?
  • Who or what validated it, under which policy?
  • Is its trust state operationally scoped rather than presented as truth?
  • What happens when the source, validation, or claim becomes unreliable?
  • Can the system remove it from the current view without deleting its history?

If those answers live only in logs, reviewer memory, or application code, the system has provenance fragments—not a governance layer.

The boundary is simple:

Extraction produces a candidate claim. Governance earns and maintains its permission to travel.

[^trace]: Saheed O. Ajayi et al., “TRACE-CTI: Auditable Post-Extraction Governance of TTP Claims with Knowledge Graphs,” arXiv:2607.24563v1, July 27, 2026, https://arxiv.org/html/2607.24563

m@berryhill in ~/posts$
$ cd ../ · back to posts/