The Judgement Layer: Remembering Human Decisions in AI-Drafted Documentation

Introduction

A large share of professional documentation is produced on a cycle. Transfer pricing reports, audit files, regulatory filings, credit reviews and policy-exception registers are all written every year, or every quarter, about mostly the same subjects. The facts move a little between cycles. The judgement mostly does not. A contract manufacturer is still a contract manufacturer. A comparable company rejected for functional dissimilarity is still dissimilar.

In this article we, the Fynto engineering team, describe the Judgement Layer, the component we built to capture every human decision made during a documentation cycle and replay it in the next one. We built it for transfer pricing documentation, which is our main use case and the worked example throughout this post. The design is not specific to tax and we discuss what it takes to apply it to other workflows at the end.

The Problem of Recurring Judgement

A transfer pricing report for a mid-size multinational group covers several dozen controlled transactions, a handful of legal entities each characterized in several transaction categories, and a benchmarking study that screens hundreds of candidate comparable companies down to a set of ten to twenty. Almost every one of those items involves a human call that has to be defended to a tax authority, sometimes years later.

Historically that judgement has lived in one place: the head of whoever prepared the file last time. The report records the conclusion but not the reasoning, the alternatives considered, or the evidence the decision was weighed against. When the preparer moves on, the reasoning goes with them.

An AI drafting agent makes this problem sharper. An agent that re-reads this year's ledgers, agreements and filings and re-drafts the analysis will, some fraction of the time, reach a different position than it did twelve months ago. Not because anything changed, but because extraction is probabilistic and the source documents are never quite the same. An unexplained year-over-year change in how a transaction is characterized, or which pricing method applies, is exactly what an examiner looks for.

Two fixes are commonly proposed, and we rejected both.

• Regenerate from scratch every cycle. This is the default behaviour of most AI drafting tools. The report reflects this year's evidence, and drift goes unnoticed until a reviewer happens to compare against last year's file.
• Roll last year's positions forward. This feels safe and is the more dangerous of the two. A stale position propagates indefinitely because nothing prompts anyone to revisit it, and the error compounds precisely in the personnel-turnover case the feature was meant to solve.

The Need for a Judgement Layer

We wanted a third behaviour. The system always drafts from this cycle's evidence. It also remembers, durably and with context, every judgement a human made last cycle. Where the two disagree, it stops and asks. Where they agree, it says nothing. Where a reviewer wants to reuse a prior call, that is one click away, and the click is itself recorded.

To start with we had the following requirements:

• Never change a position in the draft without an explicit human decision.
• Detect a contradiction between this cycle's draft and last cycle's judgement, and block finalization until a human resolves it.
• Remember the reason, in the reviewer's own words, and the evidence the decision was made against.
• Match subjects across cycles with near-zero false positives, accepting recall gaps instead.
• Keep an append-only history so a re-decision never overwrites the earlier one.
• Carry unresolved items forward so "leave open" never becomes "forgotten".
• Never let a failure in the memory subsystem block the reviewer's work.
• Support new kinds of judgement without changing the storage layer.

Architecture

At a high level the Judgement Layer is a ledger of decisions with a capture side and a recall side, sitting beside the existing report workflows rather than inside the drafting agents.

A drafting agent owns one section of the report and produces this cycle's positions from this cycle's evidence. It does not know the ledger exists. The review workflow, where a human confirms or corrects the agent's output, is the only writer and the only reader. This keeps the extraction independent of last year's answer, which is what makes the comparison meaningful.

Each decision is stored as one row with the same shape regardless of what it is about:

{
  "business_key": "far|acme services gmbh|it_support_services",
  "topic": "far_characterization",
  "fiscal_year": 2025,
  "prior_value": "limited_risk_service_provider",
  "chosen_value": "entrepreneur",
  "disposition": "confirmed_change",
  "rationale": "Acme took on DEMPE functions for the platform IP
                         in FY2025 per the amended ICA.",
  "status": "closed",
  "supersedes_decision_id": null,
  "context": {
    "agent_band": "limited_risk_service_provider",
    "agent_confidence": "low",
    "records": {
      "functions": ["Tier-2 helpdesk", "Platform maintenance"],
      "assets": ["…"], "risks": ["…"]
    },
    "source": "decision_queue",
    "user_id": "…"
  }
}

One row in the ledger. Names and text are illustrative.

The business_key is the subject's identity across cycles. The topic says what kind of judgement it is. The disposition is one of a closed set (confirmed_change, reverted, open, recorded) and says what kind of decision was made, not just what was chosen. The context records what the decision was about: the position the agent had proposed, its confidence, the evidence the reviewer was looking at, and which screen the decision came from.

Today the ledger holds four topics:

Under the hood

Following is the flow for one cycle. It runs when a reviewer opens a workflow's review step, and again on every decision they make.

1. Compute a business key for each subject. For a controlled transaction we use a source-aware ladder, taking the first rung available: a genuine ERP transaction reference; else the governing intercompany agreement's reference code; else the registrant id of the public filing it came from; else skip. Placeholder ids the pipeline assigns to rows with no external reference are detected by prefix and never used as a key, because they change every cycle for an unchanged transaction. For a functional-analysis unit the key is the normalized entity name plus category. For a comparable company it is the normalized company name. In every case the normalization is the same function the pipeline already uses to match records within one report, so cross-cycle matching is byte-identical to intra-report matching.
2. Load the most recent prior report of the same type and key it the same way. We match on report type and ignore schema version so that continuity survives schema evolution. A skipped year does not blank the comparison.
3. Require a singular match on both sides. The weaker rungs of the ladder are used only when the key resolves to exactly one subject in the current report and exactly one in the prior. An agreement governing three ref-less transactions between the same parties yields a non-unique key, and the usual disambiguator, the transaction type, is the very thing we are comparing. So we skip. This is a deliberate recall gap, never a false flag.
4. Compare positions, ignoring non-judgements. A characterization of insufficient_evidence or not_established is a coverage state, not a position a human took. A move to or from one of those is routed to the evidence-gap questions, not to the drift gate. Otherwise, if the displayed position differs, we attach an advisory to the subject.
5. Raise a gate for each advisory. The gate is a decision-queue item that must be answered before the report can be finalized. It names the prior position, who set it if it was a human override, the current position, and last cycle's rationale if there is one. It offers three options: confirm the change, revert to the prior position, or leave open. Revert restores the prior value server-side from the stored advisory. There is no language model between the reviewer's click and the row that gets written.
6. Carry open items forward. Before generating gates we also fetch every decision for this client whose latest status is open and attach an advisory to any matching subject that did not already flag a fresh change. The gate is worded "unresolved from FY2024" rather than "changed since FY2024". The same open items are listed in the client's risk register.
7. Recall prior judgements as offers. For subjects with no contradiction we still fetch the latest decision from any other report of this client, one batched query per topic, and attach it to the subject. On the review pane this appears beneath the current value as "FY2024: limited-risk service provider — 'no DEMPE functions'" with a Reuse button, and beneath each narrative element as "FY2024 — you wrote: …" with a Use this button. The recalled value is never pre-applied. "Any other report" includes sibling reports of the same year, so a regenerate an hour after a set of decisions does not forget them.
8. Write one row per decision. Every gate answer, dropdown pick and text save becomes a row. Picking a value on the review pane while a queue question about the same subject is pending answers that question, so the reviewer is not asked twice. Reusing a recalled value writes a row tagged reuse_prior. Clearing an earlier override writes a reverted row rather than deleting anything. A re-decision in the same report writes a new row pointing at the one it supersedes.

Every read and write in this flow is best-effort. If the store is unreachable the reviewer's action still applies, the audit event still fires, and a warning is logged. The whole layer sits behind one feature flag, is scoped by tenant and client on every query, and reaches the database only through the same persistence interface every other store in the product uses.

What we deliberately left out

• Auto-carry. The recalled value is an offer, never a default. Pre-applying it would silently re-impose a position the reviewer may be in the middle of changing, and would destroy the defensibility of any report that used it.
• Inferred identity. We do not let a model decide that this year's "Acme Services GmbH, IT support" is last year's "Acme GmbH, technical services". Identity comes from stable external references or the subject is skipped.
• Bulk confirmation. Routine items elsewhere in the product can be confirmed in a batch. A cross-cycle change cannot, and neither can a concluding judgement such as the arm's-length conclusion for a transaction category. One gate, one decision, one row.
• Feeding reasoning back into extraction. The ledger is not used as a prior for the drafting agents. That would make this year's extraction less independent, which defeats the purpose of comparing it to last year's.

Why this matters

• Identity is the hard part, and precision beats recall. A memory keyed to the wrong subject manufactures a false contradiction and trains reviewers to click through the gate. Derive identity from stable external references, share the normalization with the in-report matcher, and skip when unsure.
• Record what the decision was about, not just what was decided. "Last year you chose X" is a bare assertion. "Last year you chose X, against these functions and assets, when the agent proposed Y at low confidence" lets the reviewer judge whether anything has actually changed.
• Keep the model out of the decision path. Closed-choice decisions are plain endpoints that mutate persisted state. Free-text questions go to an agent. Mixing the two makes the audit trail unreadable.
• Build the store generic from the start. The fourth topic we added touched no storage code. A new kind of judgement is a key function, a disposition mapping and a gate.

Judgement is data

The Judgement Layer treats human judgement as data with a lifecycle of its own: captured at the moment it is made, keyed to a subject that survives across cycles, replayed as a question or an offer rather than an answer, and never overwritten. In transfer pricing this turns the annual report from a document that records conclusions into one that also carries the reasoning behind them, in the reviewer's own words, to whoever prepares the next one.

The same shape appears in any recurring, defensible document. An audit workpaper where a control assessed as effective last year now drafts otherwise. A regulatory filing where a product classification changed between submissions. A credit memo where a covenant was waived for a reason the next analyst should see. Each has a stable subject, a closed set of positions, a named human and a reason. Each is a topic, and the ledger already has the columns.

Terminology

Controlled transaction

A transaction between two entities of the same multinational group, which must be priced as if between independent parties.

Characterization

The classification of a transaction (for example royalty, service, sale of goods) or of an entity's functional profile (for example limited-risk service provider, entrepreneur).

FAR

Functions, Assets, Risks. The functional analysis that describes what each entity does, owns and bears in a transaction, and drives its characterization.

Comparable

An independent company whose financial results are used as a benchmark for an arm's-length price.

ICA

Intercompany agreement. The contract governing a controlled transaction. Its reference code is stable across cycles and is one rung of the identity ladder.

Business key

The deterministic cross-cycle identity of a subject, computed at decision time and stored with the decision.

Disposition

What kind of decision a row records: confirmed_change, reverted, open or recorded.

← All posts