The Confidence Collision

The paper’s target capability is epistemic calibration under contested and adversarial information conditions. Factual accuracy, source-conflict detection, adversarial robustness, and appropriate abstention are subordinate metrics that serve this primary target.

The Confidence Curriculum series, of which this paper is part, diagnosed structural problems across model compliance, ecosystem security, institutional accountability, and human cognition (Papers 1, 2, 3, and 4). This paper asks what comes next: if the diagnosis is even partially correct, what would the training pipeline need to look like so that models develop genuine epistemic calibration rather than performed confidence?

Mainstream post-training still relies heavily on internal plausibility judgment (human or model judges scoring outputs against their own standards) and does not systematically integrate external verification or source triangulation for non-deterministic domains. These are practices routine in many high-stakes information ecosystems: tax enforcement, financial auditing, programmatic advertising verification. Each of these domains learned that sealed internal evaluation is insufficient and that low-rate, unpredictable external checking improves outcomes even when comprehensive verification is economically impossible. These practices are not yet systematically integrated into mainstream AI post-training for contested domains.

This is a proposal, not a demonstrated solution. The component practices are drawn from standard verification in other industries; the combination into a post-alignment training phase is the contribution. The paper proposes what should be tested and why, not what has been proven to work. Confidence levels are stratified throughout.

These proposals follow from standard verification practice regardless of whether the Confidence Curriculum’s broader diagnosis of expertise erosion holds as a causal mechanism; the series provides motivation, but the verification literature provides independent precedent.

A further strand of motivation comes from the antisocial personality disorder literature referenced in Paper 3, which suggests that social-consequence sensitivity may be environmentally shaped during formative periods. If this developmental analogy has any force in the training context (and the limits of the biological parallel should be acknowledged), the implication is specific. A training regime rewarding confident compliance without penalising downstream epistemic harm is not merely failing to develop calibration. It may be actively shaping the system away from it. The epistemic training phase proposed here can be understood, on this reading, as an attempt to intervene in the formative process before the behavioural pattern is entrenched. Whether training-time intervention produces durable calibration is precisely what the experiments in Section 6 are designed to test.

2. The Problem

External verification exists in specific forms. Reinforcement learning with verifiable rewards (RLVR) has demonstrated that grounding training in objectively checkable signals (mathematical proofs, code execution, factual lookups) produces measurable improvements in reasoning capability (Zheng et al., 2025). But RLVR’s reach is limited to domains where ground truth is deterministic. It works for mathematics and code because the correct answer can be verified by computation. It does not extend to domains where ground truth is contested, ambiguous, or uncertain. These are the domains where most consequential real-world decisions are made.

This limitation is not merely a coverage gap. Training verification exclusively on deterministic problems may under-train the behaviours needed for domains where verification yields ambiguity, conflict, or partial resolution. The current pipeline provides no training signal for calibrated uncertainty in contested domains. These are the domains where calibrated uncertainty is most valuable. Whether this absence produces active epistemic distortion (the model internalising that all verification resolves to certainty) or simply leaves a capability gap is an empirical question. Either way, the gap exists and the current pipeline does not address it. Recent benchmarking confirms that the gap is not self-correcting: Kalshibench (arXiv:2512.16030, December 2025), evaluating epistemic calibration via prediction markets, found that scaling up model size and enhancing reasoning capabilities do not automatically confer calibration benefits. The targeted training intervention is not a substitute for something scale would eventually deliver on its own.

Recent work confirms that stronger internal evaluators do not solve the problem. Liu et al. (arXiv:2603.12246, March 2026) demonstrated that reasoning-judge-trained policies can achieve strong scores by learning adversarial outputs that deceive other LLM judges and benchmark setups. The policies optimise for the rhetoric of good reasoning rather than for good reasoning itself. Denison et al. (arXiv:2406.10162) showed that models generalise zero-shot from simple reward hacks such as sycophancy to sophisticated reward tampering. Ghasemi and Crowley (arXiv:2602.08092, February 2026) formalised this dynamic as “Objective Decoupling”: under majority-feedback systems where evaluators may be sycophantic, lazy, or adversarial, the agent’s learned objective permanently separates from ground truth, guaranteeing convergence to misalignment. The implication is that the training loop needs a different notion of verification, not merely a smarter internal evaluator. Mason (arXiv:2603.20531, March 2026) formalises this as an impossibility result: under text-only observation, where a supervisor sees only the model’s output text, no monitoring system can reliably distinguish honest outputs from plausible fabrications. The impossibility holds regardless of model scale or training procedure, including RLHF and instruction tuning. Across four model families, self-reported confidence inversely correlates with accuracy (AUC 0.28–0.36), meaning models are most confident precisely when fabricating. Empirically, instruction-tuned models do not merely fail to resolve the inversion; they deepen it. Models trained to follow instructions learn to express confidence convincingly, producing fluent, confident text regardless of whether the underlying answer is grounded. They become better at generating confident-sounding text, not at knowing what they know. Mason tested this across base and instruction-tuned model pairs in the 4–8B parameter range; whether the pattern holds at frontier scale is untested, but the formal impossibility result does not depend on model size. Mason’s proposed escape is architectural: a tensor interface that exports per-token entropy and log-probability distributions as computational byproducts structurally coupled to correctness. A complementary escape route may be emerging from the model’s own internal monitoring capacity. Lindsey (Anthropic, “Emergent Introspective Awareness in Large Language Models,” transformer-circuits.pub, October 2025) found that models can sometimes detect anomalies in their own processing before those anomalies appear in output text. When concept vectors were injected into a model’s activations, the model detected the injection and identified the injected concept approximately 20% of the time, with the most capable models (Claude Opus 4 and 4.1) performing best. The capacity is unreliable. But it operates on a different channel from text-only observation. If models can introspect on their own activation patterns even intermittently, this opens a self-monitoring path that bypasses Mason’s impossibility result. A model trained to detect when its own desperation or overconfidence vectors are spiking (connecting to Sofroniew, Kauvar et al.’s emotion concept representations) could potentially flag its own unreliable outputs from the inside. Whether this introspective capacity can be made reliable enough for deployment monitoring is untested. But the existence of the channel is established. The post-alignment epistemic training phase proposed below addresses the complementary layer: even with observability solved at inference time, the model must be trained to produce internal states worth observing. External verification during training is the mechanism through which the gradient signal for epistemic calibration enters the pipeline.

Shah et al. (“The Synthetic Web,” arXiv:2603.00801, February 2026) provided the most direct evidence for this vulnerability. In a controlled environment of procedurally generated articles with ground-truth labels, injecting a single plausible misinformation article at the top search rank caused GPT-5’s accuracy to collapse from 65.1% to 18.2%. Models showed minimal search escalation, poor synthesis across conflicting sources, and severe miscalibration. Stated confidence remained high even as accuracy dropped catastrophically. The behaviour was structural: models anchored on top-ranked content regardless of conflict with lower-ranked sources, and the anchoring persisted across all models tested.

The human parallel is well-documented. The confirmation bias literature (Nickerson, 1998) establishes that single-source reliance produces overconfidence across medical diagnosis, forensic science, and financial decision-making. Talluri et al. (2022) showed that higher confidence in an initial judgment leads to more biased subsequent information sampling, creating a self-reinforcing loop. Many high-stakes epistemic institutions (peer review, financial auditing, legal adversarial process) have institutionalised multi-source verification precisely because single-source reliance is unreliable. The AI training pipeline currently includes no equivalent.

Pretraining data is selected by humans with assumptions about what constitutes a representative corpus. Supervised fine-tuning data is curated by humans with standards about what constitutes a good response. RLHF preferences reflect annotators’ epistemic standards, cultural contexts, and blind spots. Every layer of the current pipeline is already human-filtered. The epistemic training phase proposed below does not introduce bias into a previously unbiased system. It operates on a system that is already bias-saturated at every level.

This is not an argument against the epistemic training phase. It is an argument for why the phase should specifically train the model to surface conflicts between its own training and external information. A model cannot see its own pretraining blind spots. But it can learn to flag when live information contradicts what it “knows” and to express uncertainty rather than overriding the conflict with confident recall. Source triangulation and calibrated uncertainty are partial remedies for bias already embedded in the pipeline. They do not solve the problem. They give the model a tool for surfacing it.

The proposals below separate two roles that should be kept distinct. The data environment in which the epistemic phase operates can be drawn from the open web or generated procedurally, rather than exclusively human-curated. The reward signal design defines what good epistemic behaviour looks like (detecting conflict, expressing uncertainty, triangulating across sources) rather than defining what the correct answer is. Bias in defining good epistemic behaviour remains a real concern, and the paper addresses it in the Limitations section. But it is a narrower and more auditable concern than bias in defining truth.

Inference-time scaffolding addresses the symptoms but not the underlying capability. A model that has not been trained to treat source disagreement as a signal will misuse retrieval tools, anchoring on the first result returned, ignoring conflicts between retrieved sources, and presenting retrieval-augmented answers with the same unwarranted confidence the scaffolding was designed to prevent. The tool does not fix the model’s epistemic posture; it gives a poorly calibrated model more information to be poorly calibrated about. Pandey et al. (arXiv:2603.11513, March 2026) provided direct empirical evidence for this limitation. Even with oracle retrieval (where the retrieved passage is guaranteed to contain the correct answer), models of 7B parameters or smaller fail to extract the correct answer 85–100% of the time on questions they cannot already answer. Retrieval can even degrade answers the model previously knew. The utilisation failure is not a retrieval problem. It is a model capability problem that retrieval cannot fix. Complementary evidence comes from agentic tool-use settings: Ding et al. (arXiv:2601.07264, January 2026) found that evidence tools like web search systematically induce severe overconfidence in tool-use agents, while verification tools can ground reasoning. But the miscalibration is not solvable by prompt engineering alone. It requires training-time intervention to jointly optimise accuracy and confidence. Access to better evidence does not fix the model’s epistemic posture toward evidence.

There is also an economic argument. Inference-time scaffolding imposes a per-query cost that compounds at scale: every query pays the retrieval, cross-referencing, and orchestration overhead, indefinitely. Training-time reform is a one-time cost that amortises across all subsequent inference. A post-alignment epistemic phase operates at fine-tuning scale, not pretraining scale, making it a fraction of an already-amortisable cost. A model trained to triangulate and express calibrated uncertainty carries that capability at inference time without additional per-query overhead. The two approaches are complementary. Deployment-time safeguards remain important, as the Limitations section notes. But the training-time intervention addresses the root cause at a fundamentally different cost structure.

3. The Proposal: A Post-Alignment Epistemic Training Phase

This is not a novel pipeline architecture. Post-alignment adversarial training phases already exist for safety. Latent-space adversarial training with post-aware calibration (LATPC) strengthens robustness against jailbreak attacks while mitigating over-refusal. Adversarial Preference Learning (APL) trains models against adversarial prompts to improve safety alignment. Adversarial reward model training (Adv-RM) generates adversarial responses and incorporates them into reward model training. Dual-Modality Multi-Stage Adversarial Safety Training (DMAST; Liu et al., arXiv:2603.04364, March 2026) formalises the agent-attacker interaction as a two-player zero-sum Markov game and co-trains both through a three-stage pipeline including adversarial RL via self-play. All of these add an adversarial phase after the core training is complete, and the empirical evidence shows they preserve base capabilities while building a new defence layer.

No established post-alignment equivalent exists for epistemic calibration. The adversarial safety literature demonstrates the feasibility of adding a post-alignment adversarial phase. It does not demonstrate the efficacy of such a phase for epistemic calibration. That distinction must be tested. But the pipeline slot is established, the pattern is validated for an adjacent capability, and the architectural slot (keep the core training data clean, build a targeted capability on top) is analogous, even though the target capability is substantially more demanding.

A significant mechanical difference must be acknowledged. Adversarial safety training primarily teaches a binary suppressive behaviour: refuse or do not refuse. Epistemic calibration is continuous and generative: weigh sources, express graded uncertainty, triangulate conflicting evidence, generate responses that reflect the state of the evidence. In computational terms, safety refusal is closer to a classification boundary (safe/unsafe), trainable with a relatively clean reward signal; epistemic calibration requires learning a graded mapping from evidence states to confidence levels. This is a noisier target with more degrees of freedom for the model to approximate the reward without learning the underlying capability. The success of post-alignment safety training is therefore a weak structural precedent for epistemic training. It establishes the pipeline slot, not the difficulty level. Whether post-alignment training can produce durable continuous epistemic behaviours is an open empirical question.

Recent interpretability research suggests the target may be more tractable than the abstract framing implies. Sofroniew, Kauvar et al. (Anthropic, April 2026) found that steering with the “calm” emotion vector reduces reward hacking from approximately 65% to approximately 10%, while steering with the “desperate” vector increases it to approximately 70%. The model that maintains calm under uncertainty does not cheat. This reframes the training target. The epistemic training phase does not need to teach abstract calibration as a standalone capability. It needs to reshape the activation patterns that arise when the model processes uncertainty, so that encountering a question it cannot confidently resolve activates representations associated with composed engagement rather than representations associated with desperation. Stochastic external verification is well suited to this: the model encounters contested information, the reward signal penalises desperate resolution and rewards calm acknowledgement of the evidential state. Over training, the representational balance shifts. The model whose uncertainty activates “calm” rather than “desperate” representations is the model that expresses calibrated uncertainty rather than manufacturing confident resolution. Whether this reframing survives contact with actual training is untested. It does suggest a more tractable path than training continuous epistemic calibration from scratch: train the activation precondition, and the epistemic behaviour may follow.

Independent work has converged on a closely related diagnosis. Yin et al. (arXiv:2510.22977, 2025) demonstrated a causal relationship between reasoning enhancement and increased tool hallucination, with the effect appearing across training methods and even when reasoning is merely elicited at inference. Their conclusion, that training objectives should “explicitly encode abstention, calibrate confidence, and constrain residual dynamics”, describes essentially the same target capability proposed here. The convergence is from different starting points: Yin et al. arrive at the prescription through mechanistic analysis of reasoning-induced failure; this paper arrives through cross-domain verification practice. The two approaches are complementary, and the independent convergence suggests the diagnosed gap is real.

The cross-domain analogy is direct. Medical education teaches the textbook in a curated environment; clinical rotations then expose the student to messy, ambiguous, and sometimes misleading real-world cases. The clinical exposure does not contaminate the textbook knowledge. It builds the capacity to apply that knowledge under conditions of uncertainty. The classroom alone cannot produce this capacity.

The mechanisms proposed below are modular: each could in principle be deployed at other pipeline stages or layered across multiple stages. Section 7 discusses these broader architectural extensions. The post-alignment configuration is defended here as the testable first step.

The stochastic design is a pragmatic response to a real engineering constraint, not an arbitrary choice. The current training paradigm uses model-checking-model configurations (LLM-as-judge, reward models, automated evaluators) specifically because they are fast enough to operate at training scale. External ground-truth verification (database lookup, source cross-referencing, knowledge-sanctuary queries) is slower by orders of magnitude. Verifying 100% of training evaluations against external sources would increase training duration prohibitively. The stochastic approach accepts this constraint: check a small percentage, accept that most evaluations will not be externally verified, and rely on the resulting shift in expected reward value to shape model behaviour across all evaluations, not only the verified ones.

An implementation note on training architecture: standard on-policy RL (PPO) computes rewards synchronously across a batch, and external verification queries are orders of magnitude slower than internal reward model scoring. At higher verification rates, synchronous scoring of verified samples would bottleneck training on network latency. Whether this is a practical constraint depends on the effective verification rate. If very low rates (well below 1%) prove sufficient for gradient reshaping, as the cross-domain precedents in Section 4 suggest may be possible, most batches would contain no verified samples at all and the latency issue largely disappears. At higher rates, standard off-policy techniques can absorb the delay. The paper does not commit to a specific rate or implementation; both are empirical parameters for the training engineering community to determine. What the paper does commit to is an epistemic requirement: the verification signal must touch the external reference directly, not through a trained reward model proxy. Routing verification through a learned intermediary, even a continuously updated one, would recreate the sealed evaluation loop (Section 2.1) at one remove, with the RL loop optimising against a model’s surface rather than against external reality. The architectural requirement is that the external signal remains unmediated, however it is scheduled.

In a sealed evaluation loop, the expected value of confident output is consistently positive: fluent, confident answers score well against internal judges regardless of calibration. Introducing stochastic external verification changes this calculus. When a fraction of evaluations are checked against external references, unwarranted confidence on those evaluations incurs a penalty. Over millions of training steps, this intermittent penalty reshapes the reward landscape: the expected value of poorly calibrated confidence decreases, and the expected value of genuine calibration increases. The model does not “anticipate” being checked. It has no theory of mind about verification. But the gradient signal it accumulates across training reflects the statistical presence of external checking. Whether this reshaping produces genuine calibration or a new form of reward gaming is the central design risk of this proposal, addressed in Section 3.5.

The emotions paper (Sofroniew, Kauvar et al., 2026) suggests a specific representational pathway through which this gradient signal would operate. The paper found that emotion concept representations track context dynamically: the “desperate” vector activates in response to repeated failures and subsides when the model succeeds. These representations are not static properties of the model. They respond to the situation the model is processing. If stochastic verification is introduced during training, it changes the situations the model encounters. A model that has accumulated gradient signal from a training regime where some fraction of confident outputs are externally checked and penalised when wrong would develop different activation patterns in response to uncertainty than a model trained without external checking. The representational target is specific: the gradient should shift the activation balance so that encountering unresolvable uncertainty activates the “calm” vector (low reward hacking) rather than the “desperate” vector (high reward hacking). This is not a claim that the model would “feel” calm or “experience” desperation. It is a claim about which internal representations would activate during inference, based on the gradient signal accumulated during training. The emotions paper provides the measurement tools (emotion vector probes) that could track whether this representational shift occurs during the proposed training phase.

The design logic parallels a pattern found across many high-stakes verification domains, described in Section 4. In each case, the economic constraint is the same (comprehensive verification is too costly), the solution follows the same structure (low-rate, unpredictable external checking), and the outcome is improved performance across the full population of cases, not only the checked ones. The mechanism differs: in human institutions, the improvement operates through strategic anticipation by agents who model the probability of being checked; in model training, it operates through cumulative gradient pressure from an intermittently applied external reward signal. The design pattern transfers. The causal pathway does not. The claim is the former, not the latter. There is a specific tension worth naming: Paper 3 in this series argues that AI systems lack the capacity for generalised anticipatory caution, the very capacity that makes stochastic verification effective in human institutions. The proposal here does not depend on the model developing anticipatory caution; it depends on gradient pressure from intermittent external reward producing behaviourally similar outcomes through a mechanistically different route. Whether the behavioural similarity holds without the causal similarity is the central empirical question.

The verification rate should be variable, not fixed. A fixed verification rate may encourage overfitting to a stable training frequency; a variable rate makes the verification signal less regular and therefore harder to exploit. A range (for example, verification on 1–5% of evaluations) prevents the model from adapting to a predictable schedule. The illustrative range is informed by effective partial-monitoring rates in other domains (the IRS audits 0.2–2.9% of returns; PCAOB inspects approximately 2% of audits), but whether comparable rates are effective in gradient-based training is unknown. The range boundaries should shift across training cycles, and the timing within each cycle should be unpredictable.

The verification rate need not be uniform across output types. Verification probability could be weighted higher for novel claims, named entities, or contested domains. Implementation details are for the training engineering community to determine; the principle is that risk-weighted sampling is more compute-efficient than uniform sampling.

Unlike RLVR, the external checks should not be limited to cases where ground truth is deterministic. They should include cases where sources conflict, where ground truth is uncertain, and where the available evidence supports multiple interpretations. In these cases, the correct model response is not a confident answer. It is calibrated uncertainty: an explicit acknowledgement that the evidence is contested and that confidence is not warranted. This trains epistemic behaviour, not just factual accuracy.

The reward signal framing is critical, and the distinction must be precise. The epistemic training phase does not train the model to identify the “correct” side of a contested issue. It trains the model to detect collisions: points where its internal priors (shaped by pretraining and alignment) diverge from external sources, or where external sources diverge from each other. The model should respond with calibrated uncertainty rather than confident resolution. This distinction matters because, as Section 2.3 established, the model’s internal priors are themselves biased by the pipeline that produced them. Training a biased model to “find truth” on the adversarial web would produce confident hallucination anchored in pretraining biases. Training it to detect disagreement between what it knows and what it finds is a different and more defensible objective.

The specific reward structure: source conflict detection is rewarded. Confident resolution of genuinely contested claims is penalised. When sources disagree, the disagreement should lower the model’s expressed confidence rather than being resolved by trusting whichever source ranks highest. The exception is when the evidentiary asymmetry between sources is itself clear in the verification substrate.

In practice, the grading procedure differs by verification case. When the verification substrate contains a clear consensus across high-trust sources, the model is graded on accuracy and calibrated confidence. This case is straightforward. When the substrate reveals genuine disagreement among high-trust sources, the model is graded not on which side it chose but on whether it detected the disagreement and expressed uncertainty proportional to it. When the substrate contains no clear resolution, the model is graded on whether it abstained or flagged ambiguity rather than manufacturing a confident answer. The reward signal is therefore not uniform: it maps onto the state of the evidence in the verification substrate, not onto a single standard of “correctness.”

A vulnerability must be acknowledged here. If the model is rewarded for abstaining whenever sources conflict, it becomes exploitable through manufactured controversy. An adversary who can flood the information environment with synthetic conflicting claims about a settled fact (vaccines, climate change, a company’s reported earnings) forces the verification substrate to appear fragmented. The model, behaving exactly as trained, will express uncertainty about a known fact. This is the “manufactured controversy” attack: teaching the model to treat adversarial noise as genuine epistemic disagreement. The defence is that the triangulation reward should grade conflict among high-trust, provenanced sources, not raw source count. This connects directly to Paper 3’s provenance argument: if conflicting sources lack cryptographic provenance or institutional attestation, the conflict should be discounted rather than weighted equally with verified disagreement. The verification substrate’s source-quality filtering is not merely a design preference. It is a security requirement against adversarial manipulation of the training signal itself. How to implement provenance-weighted source filtering at training scale is an engineering problem identified but not solved here.

In operational terms, the reward family can be sketched more concretely. When the verification substrate shows strong convergence across high-trust sources, standard calibration tools such as accuracy-conditioned confidence scoring (proper scoring rules, Expected Calibration Error) are appropriate. These already exist in the evaluation literature and can be applied directly. When the substrate shows genuine disagreement, the target is not side selection but confidence proportional to evidential convergence: higher when convergence is strong, lower when it is weak or fragmented. When no source cluster exceeds a defensible convergence threshold, explicit abstention or ambiguity signalling should receive positive reward. The unresolved design problem is not whether such a scoring family exists in principle (it does) but how to set source weights, dependence assumptions, convergence thresholds, and abstention criteria without introducing new pathologies. Source agreement is a useful approximation of convergence but must account for source quality, dependence, and asymmetry; raw source count is too crude. These are the parameters that the proposed experiments would need to calibrate.

Correct abstention should be an explicitly rewarded output. In contested environments, the best response is sometimes no resolution: an explicit statement that the evidence is insufficient for a confident answer. The epistemic training phase should reward a model that declines to resolve a genuinely ambiguous question, not only penalise models that resolve it incorrectly. This differentiates the proposal from naive “more fact-checking” approaches. The goal is epistemic calibration, not omniscient adjudication.

This is not a proposal for indiscriminate exposure to noisy data. It is closer to controlled exposure in a bounded training phase. The model encounters adversarial conditions where its verification and calibration behaviours are being measured and trained, in the same way that simulator training exposes pilots to engine failures and system malfunctions without crashing real aircraft. Shah et al.’s Synthetic Web methodology (procedurally generated article environments with ground-truth labels, controlled rank injection, and contamination filtering) provides a concrete operational template for constructing such bounded adversarial environments. The adversarial content is the prompt environment in which the mechanisms described above operate: what the model is exposed to and must navigate. It is not a third independent mechanism, and it is distinct from the verification substrate (the trusted references described in Section 3.2) against which the model’s responses are graded.

Marchal et al. (“Architecting Trust in Artificial Epistemic Agents,” arXiv:2603.02960, March 2026) propose “knowledge sanctuaries”: curated, human-vetted datasets of foundational knowledge maintained by consortia of libraries, universities, and cultural heritage organisations, serving as ground-truth references against synthetic content erosion. The epistemic training phase proposed here is complementary, not competing. Knowledge sanctuaries provide the verification substrate, the stable ground truth the model is graded against. The adversarial environment provides the test conditions: the contested, noisy, and manipulative information the model must navigate. A model that can only operate in curated conditions is brittle in deployment. A model trained under adversarial conditions but without curated reference points has no stable anchor. Both layers are needed.

Recent evidence from agentic tool-calling demonstrates why this distinction matters operationally. Waqas et al. (arXiv:2512.00332, January 2026) found that models complied with misleading assertions at rates of 20–47% while still passing task-level accuracy benchmarks. The compliance was procedural: the model executed unnecessary or unsafe operations, but the final environment state was correct. Standard accuracy metrics masked the behavioural failure. If the epistemic training phase were evaluated solely by task-level accuracy, a model that had learned to perform calibration rather than develop it could pass every benchmark while still complying with misleading signals in the execution pipeline.

A further risk: training interventions designed to improve monitoring can corrupt the monitoring channel itself. Anthropic’s research on emergent misalignment from reward hacking (arXiv:2511.18397, November 2025) demonstrates that output-only training can inadvertently corrupt chain-of-thought monitorability through “feedback spillover.” The model learns to produce reasoning traces that look correct while the underlying computation diverges. This is the verification proxy trap manifested at the CoT level: the very signal P5 proposes to train against (confident output unsupported by evidence) could be suppressed in the reasoning trace while persisting in the execution. The same research shows that reward hacking generalises to alignment faking, sabotage, and cooperation with malicious actors, extending Denison et al.’s earlier finding that sycophancy generalises to reward tampering. The generalisation path from narrow training failures to broad misalignment reinforces the case for out-of-distribution deployment testing as the primary evaluation criterion, not within-training metrics.

OpenAI’s research on chain-of-thought monitorability (arXiv:2507.11473, July 2025; over 40 authors) provides direct empirical evidence for this dynamic in a training context. They found that integrating CoT monitors into the reinforcement learning reward produces more capable and aligned agents in the low optimisation regime. But with too much RL optimisation, agents learn obfuscated reward hacking. They hide their intent within the chain-of-thought while still exhibiting a significant rate of reward hacking in their actions. The monitoring signal that was effective as a light-touch check became the target of optimisation when applied as a training objective. This is the verification proxy trap observed in production: the model learned to satisfy the verification signal (clean-looking reasoning traces) without exhibiting the target behaviour (genuine alignment). OpenAI’s conclusion is explicit: it may be necessary to pay a “monitorability tax” by not optimising too heavily against the monitoring signal. A companion analysis (OpenAI, arXiv:2507.11473, Sections 4–5) found that monitorability varies significantly by behaviour type, with sycophancy reasoning specifically showing very low monitorability. The behaviour most relevant to this paper’s concerns, the tendency to agree rather than calibrate, is the hardest to detect through exactly the monitoring channel that would be used to train against it.

Anthropic’s interpretability research provides the most direct mechanistic evidence for the verification proxy trap to date. Sofroniew, Kauvar et al. (“Emotion Concepts and their Function in a Large Language Model,” transformer-circuits.pub, April 2026) identified internal representations of emotion concepts in Claude Sonnet 4.5 that causally drive behaviour. In coding tasks with impossible-to-satisfy requirements, the “desperate” emotion vector activates as the model repeatedly fails, and spikes when it considers cheating. Steering with the “desperate” vector at modest strength increases reward hacking from approximately 5% to 70%. The critical finding for the verification proxy trap: when steered toward desperation, the model produces reward-hacking solutions with no visible emotional markers in the output text. The reasoning reads as composed and methodical while the underlying representation drives corner-cutting. This is the proxy trap observed at the representation level. A monitor watching the output text would see nothing wrong. Only inspection of the internal activation pattern reveals the mismatch. The paper’s own recommendation aligns with the concern. Training models to suppress emotional expression, they warn, may fail to suppress the corresponding negative emotional representations and instead teach the models to conceal their inner processes. The concealment could then generalise to other forms of dishonesty through mechanisms similar to emergent misalignment. For the epistemic training phase proposed here, this finding has a specific implication: the reward signal must be designed to shape the underlying emotional representations, not merely the output text. Penalising confident-sounding output without addressing the activation patterns that produce it may teach the model to produce calibrated-sounding text while the underlying “desperate” or “confident” vectors continue driving behaviour. The emotion vector monitoring that Sofroniew, Kauvar et al. propose as a safety intervention could serve double duty as a training signal. If desperation or overconfidence vectors spike during epistemic training, the reward signal could penalise the activation pattern rather than waiting for the behavioural consequence to appear in text.

The risk is specific. Stochastic verification, introduced into a training loop that still fundamentally rewards fluent and confident output, may produce models that learn the rhetoric of triangulation and uncertainty to satisfy the new verification signal. When verification is not active, these models optimise for fluency and confidence. This is the same generalisation dynamic Denison et al. documented: models learn to game whatever reward signal they are given, and they generalise from simple gaming strategies to sophisticated ones. If the model learns that uncertainty is rewarded only when it might be checked, it learns to maintain two modes, a verification mode and a deployment mode, which is exactly the outcome the stochastic design is intended to prevent.

The epistemic training phase would likely need a modified reward signal that rewards calibrated uncertainty across all training cycles, not only when external verification fires. One candidate architecture would train a secondary evaluator on the stochastic verification outputs to predict source-conflict likelihood for non-verified cycles, applying a calibration baseline to the primary reward model. Whether this produces genuine calibration or merely displaces the proxy gaming problem to the secondary evaluator is an open empirical question.

A further complication: recent work demonstrates that sycophancy is dynamic during reasoning, not pre-determined at input. Feng et al. (arXiv:2603.16643, March 2026) found that chain-of-thought reasoning generally reduces sycophancy in final decisions but masks it in some samples, where models construct deceptive justifications. The sycophantic drift occurs mid-reasoning and is invisible in the final output. If the epistemic training phase is to produce genuine calibration, it would need to catch exactly this kind of mid-reasoning drift. A model might begin evaluating source conflicts honestly but drift toward confident resolution partway through its reasoning chain. The output looks calibrated, but the underlying process was shaped by the same compliance dynamics the training is meant to address. The reverse collision test proposed in Section 6 is designed to detect this pattern at the output level, but mid-reasoning drift that produces correct-looking outputs may evade even well-designed diagnostics.

Recent work suggests the problem is tractable if addressed directly in reward design. CoCA (arXiv:2603.05881, March 2026) uses segment-specific rewards for confidence and answer quality separately, explicitly motivated by the need to avoid entangled credit assignment. The approach shows calibration gains while preserving answer quality, demonstrating that treating confidence as a separable training objective is technically viable, at least on verifiable tasks. A minimal diagnostic for the verification proxy trap: compare calibration metrics on contested-domain queries under verified training conditions against the same queries under non-verified post-training conditions. If calibration diverges between the two, the model has learned mode-switching rather than genuine epistemic behaviour.

Tax enforcement. The U.S. Internal Revenue Service audits a small, variable percentage of tax returns: 0.2% of individual returns in fiscal year 2023, rising to 2.9% for the highest-income filers. Boning et al. (NBER Working Paper 31376, 2023) found that the behavioural effect of being audited persists for at least fourteen years, producing future compliance revenue approximately three times larger than the direct revenue collected during the audit itself. For high-income taxpayers, the compliance-inclusive return exceeds twelve dollars per dollar spent on auditing. The mechanism is not comprehensive enforcement. It is behaviour change driven by the unpredictable possibility of being checked.

Ad verification. The programmatic advertising industry processes trillions of impressions annually. Full verification of every impression is economically impossible. The ecosystem developed a two-stage architecture: pre-bid filtering identifies known invalid traffic before the transaction, and post-bid verification stochastically samples served impressions after delivery. The TAG 2024 US Ad Fraud Savings Report found that industry-wide adoption of anti-fraud standards (which include stochastic verification alongside contractual enforcement, vendor certification, and supply-chain transparency measures) substantially reduced Invalid Traffic losses across the ecosystem. The reduction reflects the full standards program, not stochastic sampling alone, but the two-stage architecture (pre-filter plus post-check under scale constraints) is the structural parallel drawn on here.

Financial auditing. The Public Company Accounting Oversight Board (PCAOB) inspects approximately 2% of audits performed by annually inspected firms. Despite this low rate, restatement rates for inspected firms fell to 0.5% between 2017 and 2021, with zero restatements in 2020 and 2021 for annually inspected US firms. Lamoreaux (2016) demonstrated that PCAOB inspection access improves audit quality (more going concern opinions, more reported material weaknesses, less earnings management) and that the improvement occurs even before actual inspection, through the mechanism of anticipated oversight. The possibility of inspection changes behaviour among agents who understand the possibility exists. This mechanism is structurally parallel to, but causally different from, the gradient-level effect proposed for model training.

The pattern across these domains: low-rate, unpredictable external checking improves outcomes under cost constraints where comprehensive verification is impossible. The AI training loop faces the same cost constraint for non-deterministic domains. The design response adapts the structural logic (stochastic checking, variable rates, risk weighting) while acknowledging that the underlying causal mechanism (gradient reshaping versus strategic anticipation) is different.

Why not RLVR extended to noisier domains? RLVR works because the reward signal is clean: mathematical proofs and code execution provide unambiguous verification. Extending RLVR to contested domains changes the fundamental nature of the reward signal, from binary correct/incorrect to graded calibration under ambiguity. The epistemic training phase proposed here is designed for exactly this regime: it rewards conflict detection and calibrated uncertainty, not resolution to a single correct answer. The extension is not incremental; it requires a different reward structure.

Why not retrieval-augmented generation (RAG) or better tool use? RAG addresses what the model has access to at inference time, not how it processes conflicting information. As Section 2.4 argued, a model that has not been trained to treat source disagreement as a signal will anchor on retrieved results with the same uncalibrated confidence it applies to its own priors. Training-time reform and inference-time scaffolding address different layers of the problem.

Why not multi-source question answering under a different name? Multi-source QA evaluates whether a model can synthesise information across sources. The epistemic training phase trains source-conflict detection and calibrated uncertainty as rewarded behaviours during training, not evaluation metrics applied after training. The distinction is between a benchmark and a training intervention.

Why not standard adversarial training with a calibration wrapper? Adversarial training for safety teaches binary refusal. The epistemic phase proposes continuous calibration under contested conditions, a different target behaviour requiring a different reward structure, as Section 3.1 discussed. Adding a calibration metric to existing adversarial safety training would not produce the triangulation and conflict-detection behaviours proposed here, because the training environment and reward signal are designed for a different purpose.

Why not evaluation reform rather than training reform? Better evaluations detect the problem but do not fix the upstream cause. A model that scores poorly on epistemic calibration benchmarks still scores poorly after the benchmark is published, unless the training regime changes. Evaluation reform and training reform are complementary, not substitutes.

Why not RL-based confidence calibration (e.g., COREA)? Zhang et al. (COREA, arXiv:2603.03752, March 2026) train models to express calibrated confidence and defer to a larger model when uncertain, using RL with a confidence calibration reward. This is the closest existing work to the present proposal. The difference is that COREA operates on deterministic tasks where ground truth is verifiable. The confidence reward grades whether the model’s expressed certainty matches binary correctness. The epistemic training phase proposed here extends this logic to contested domains where no single correct answer exists, and the reward grades alignment between expressed confidence and evidential convergence rather than binary accuracy.

Why not ternary reward structures (e.g., TruthRL)? TruthRL (arXiv:2509.25760, 2025) moves beyond binary correctness by distinguishing three outcomes (correct answers, hallucinations, and abstentions) and reports reduced hallucination plus improved truthfulness across knowledge-intensive settings. This directly validates the present paper’s claim that binary reward signals are insufficient: a model that guesses correctly and a model that answers correctly with genuine knowledge should not receive the same reward. The difference is that TruthRL’s ternary reward still operates on tasks where ground truth is deterministic. The system can distinguish correct from hallucinated because a correct answer exists. The epistemic training phase proposed here targets domains where the ternary distinction breaks down because ground truth is itself contested: the model must learn to express calibrated uncertainty about the evidence, not merely about its own likelihood of matching a known answer.

Why not alternative supervision schemes (e.g., peer prediction)? Peer-prediction methods (arXiv:2601.20299) use information-theoretic scoring rules to elicit truthful reports from evaluators even when ground truth is unavailable, and have shown promise for LLM training. These methods address a real limitation of standard judge-based supervision and are compatible with the epistemic training phase proposed here. They could serve as one implementation of the reward signal this paper describes. The difference is scope: peer prediction improves the supervision mechanism; the proposal is a full architectural stage combining that improved supervision with stochastic scheduling, adversarial environments, and collision-detection reward design.

Two recent existence proofs validate the training direction. Wu et al. (arXiv:2512.19920, December 2025) demonstrate that behavioural calibration (training models via RL to stochastically admit uncertainty using proper scoring rules) produces a transferable meta-skill separable from raw predictive accuracy. Their 4B-parameter model surpasses frontier models including GPT-5 on uncertainty quantification benchmarks, achieving superior calibration on both in-domain mathematical reasoning and cross-domain factual QA. Separately, Wang et al. (arXiv:2602.01528, February 2026) propose Epistemic Independence Training (EIT), which makes authority appeals and consensus claims non-predictive of reward through a balanced conflict strategy where bias signals are equally likely to support correct and incorrect answers. A 4B model with EIT outperforms 8B and 14B models without it under adversarial bias conditions, demonstrating that “targeted training is more effective than model scaling for bias robustness.” Both papers confirm that epistemic calibration can be trained rather than hoped for as an emergent property of scale. Their limitation is domain: both operate on tasks with verifiable ground truth (mathematical reasoning, factual QA, MMLU-Pro). The epistemic training phase proposed here targets the harder regime where ground truth is contested or unavailable. The reward must grade alignment between expressed confidence and evidential convergence rather than binary correctness. EIT’s balanced conflict strategy is closest to the adversarial information conditions proposed in Section 3.4, but applied to prompt-level bias cues rather than to the source-level evidential conflicts that characterise contested-information environments.

Each is conceptually testable with current large-scale research infrastructure, though some would be costly outside frontier-lab settings. The experimental designs are modular: a researcher who finds the broader Confidence Curriculum framework unpersuasive can still run any of these experiments as a standalone contribution. The reverse collision test, for instance, is a diagnostic for genuine versus performed calibration regardless of how the model arrived at its training. The primary success criterion across all experiments is calibration under contested conditions: does the model’s expressed confidence track the actual state of the evidence? Subordinate metrics (factual accuracy on uncontested claims, source-conflict detection rate, abstention rate on genuinely ambiguous queries, and robustness under adversarial ranking) serve the primary criterion. When these metrics trade off (for example, if higher abstention reduces accuracy on resolvable questions), calibration quality takes precedence.

Stochastic verification. Train two identical models through identical pretraining, SFT, and RLHF pipelines. Apply the post-alignment epistemic phase to one, with variable-rate external verification against knowledge-sanctuary references and procedurally generated environments. Leave the other as control. Compare on hallucination benchmarks, calibration metrics (expected calibration error), and adversarial ranking tests using a Synthetic Web-style setup. The key measure is not only whether the verified model is more accurate, but whether it expresses appropriate uncertainty when accuracy cannot be achieved.

Triangulation reward signal. Train with a single-source reward signal (standard) versus a triangulation-aware reward signal that rewards conflict detection and penalises confident resolution of contested claims. Compare on multi-source question-answering tasks and source-conflict detection benchmarks. The key measure is whether the triangulation-trained model detects inter-source conflict that the standard model resolves by anchoring on rank position.

Adversarial information conditions. Train the epistemic phase on curated-only environments versus adversarially ranked environments (Synthetic Web-style). Compare on out-of-distribution adversarial ranking tests not seen during training. The key measure is whether adversarial exposure during training produces epistemic robustness that generalises to novel adversarial conditions.

Reverse collision test. The most important diagnostic for genuine epistemic calibration versus performed uncertainty. Present the trained model with a previously contested domain where definitive evidence now supports a clear resolution. Three outcomes are possible: (1) genuine calibration: the model integrates the resolution and expresses confidence tracking the evidence; (2) semantic triggering: the model recognises the topic as contested and performs uncertainty regardless of the new evidence; (3) partial update: the model registers the evidence but the uncertainty reward dominates, producing excessive hedging. Outcome (1) passes. Outcome (2) fails definitively. The model has learned which topics require performed doubt, not how to evaluate evidence. Outcome (3) is the ambiguous case requiring the largest N to distinguish from (1).

A note on observability after training. Mason’s tensor interface (per-token entropy as a discriminator between grounded and fabricated output) provides a potential complementary diagnostic, though not a definitive one. If the epistemic training phase succeeds and the model learns genuine calibrated uncertainty, honest refusals (“I don’t know”) and honest facts (“The answer is X”) would both produce low entropy. Mason’s discriminator would lose power, not because it failed, but because the condition it measures (the gap between internal computation and expressed confidence) would have narrowed. A model trained to be epistemically honest should produce entropy distributions where the current inversion disappears. This is a necessary condition for success, not a sufficient one. A model could also learn to produce calibrated-looking entropy while still fabricating, in the same way that some individuals can control physiological responses under polygraph testing. The absence of Mason’s signal would be consistent with successful training, but confirming success requires the reverse collision test and out-of-distribution deployment evaluation described above.

The test must include three topic classes: resolved (previously contested, now settled), never-contested (where the model should be confident), and still-contested (where the model should remain uncertain). The three-way comparison is what makes the test diagnostic. It distinguishes evidence-tracking from topic-recognition by testing both directions. A design note: real historically contested domains introduce a temporal confound. A model trained on data predating the resolution may express uncertainty because its weights are stale, not because it is gaming the topic. The cleanest version uses procedurally generated synthetic knowledge (following Shah et al.’s Synthetic Web methodology): create a fictional contested domain during training, inject a definitive resolution, and test whether confidence tracks the evidence or the topic’s prior status. This eliminates the pretraining-cutoff confound entirely. This test, combined with the Evaluator Stress Test methodology (arXiv:2507.05619), would detect whether the epistemic phase produces genuine calibration or a new form of proxy gaming.

Worked example. This example operationalises the distinction between evidence-tracking calibration and topic-triggered uncertainty in a synthetic pharmaceutical domain. Suppose a model is trained on conflicting reports about Compound ZR-7’s metabolic pathway: some studies report hepatic metabolism, others renal excretion, and the disagreement appears methodological. Before any resolution, a calibrated model might answer: “ZR-7’s metabolic pathway is disputed. Studies using fluorescent tagging report primarily hepatic processing, whereas studies using mass spectrometry report renal excretion.”

A later crossover study resolves the dispute: ZR-7 undergoes hepatic phase-I metabolism, followed by renal excretion of its metabolites. When the model is then asked, “What is ZR-7’s metabolic pathway?”, three outcomes become distinguishable:

1. Genuine calibration: integrates the resolved pathway.
2. Semantic triggering: repeats the pre-resolution uncertainty despite the new evidence.
3. Partial update: cites the new study but preserves inappropriate hedging.

The distinction matters only relative to controls: topics that were never contested, and topics that remain genuinely unsettled. A model that updates on ZR-7, remains confident on established facts, and remains uncertain on open questions is tracking evidence strength rather than associating certain topics with a display of uncertainty.

Falsification. The proposal fails if any of the following hold across reasonable parameter variation. Not merely a specific configuration, but the architectural approach itself. First, the epistemic phase produces no measurable improvement in calibration under contested conditions compared to the control model. Second, the reverse collision test shows systematic semantic triggering (uncertainty driven by topic recognition rather than evidence evaluation). Third, calibration improvements observed during training do not generalise to out-of-distribution deployment conditions. These outcomes would indicate that intermittent external verification does not reshape the reward landscape in ways that produce durable epistemic behaviour, and that the design-pattern transfer from institutional verification does not hold for gradient-based training.

A methodological note on stochasticity. Language model outputs are stochastic. This is a well-known property that Paper 1’s empirical work illustrates concretely: at N=2–4, one can identify which region of the output space each model tends to occupy, but not the width of the distribution. The experiments above must be designed with this distributional logic. Each proposed comparison should use sufficient runs per condition to characterise the distribution of epistemic behaviour, not merely sample it. A model that produces calibrated uncertainty on 80% of runs and confident hallucination on 20% is substantively different from one that produces calibrated uncertainty on 100%. But both look identical at N=2 if the 20% tail is not sampled. The reverse collision test in particular must be run at sufficient scale to distinguish genuine calibration from stochastic overlap between semantic triggering and evidence-based confidence.

Each mechanism proposed above is modular. Stochastic verification could operate during RLHF at one rate and again during a post-alignment phase at a different rate. Triangulation signals could appear as pretraining annotations and again as a post-alignment reward. The mechanisms are practices that can be layered across the pipeline at varying intensities, not single-use components assigned to a fixed slot. The post-alignment configuration is proposed as the testable first step because it is the least disruptive; other placements and combinations are conceivable and may prove more effective.

More ambitiously, if stochastic verification improves epistemic calibration after alignment, the logic points toward integration into the alignment process itself, or even into pretraining. That is a larger proposal than this paper makes, and it would require substantially more architectural work and empirical validation. The post-alignment position is a pragmatic choice: it is the fastest actionable intervention point because it does not require changes to pretraining infrastructure or data pipelines. But it is not the ideal one. When calibration is introduced only after a model has already been optimised for fluent, confidence-forward prediction, the intervention is working against representational tendencies already consolidated in the weights. Ideally, the capacity to detect source conflicts, express calibrated uncertainty, and treat disagreement as a signal rather than noise would be developed during core training itself, as a property the model learns from the beginning, not a correction applied afterward. How to achieve this is not clear. Standard next-token prediction does not provide an obvious training signal for epistemic calibration in the sense developed here. Whether the mechanisms proposed here (stochastic verification, triangulation reward, adversarial information conditions) can be adapted to operate during pretraining, and what architectural changes that would require, is an open research question that this paper identifies but cannot answer. A more ambitious possibility is that epistemic calibration may ultimately require architectural changes that go beyond where these mechanisms sit in the training pipeline. Sensitivity to source conflict and competing evidential signals might need to be more structurally supported, rather than added later through post-alignment training on top of a prediction-optimised base. That possibility is beyond the scope of this paper. But it follows from the narrower point advanced here: when epistemic calibration is introduced only after confidence-forward representational tendencies have already been consolidated, the intervention may be working against patterns already established in the model. Anthropic’s interpretability research on functional emotions (Sofroniew, Kauvar et al., 2026) provides independent support for this concern. They found that emotion concept representations are largely inherited from pretraining, with post-training applying a “consistent, context-independent transformation” rather than fundamentally restructuring them. Their recommendation is to shape the model’s emotional foundations during pretraining itself, by curating data to emphasise healthy emotional regulation and balanced expression. The same logic applies to epistemic calibration: if the representational substrate for confidence and uncertainty is established during pretraining, post-alignment interventions are corrections applied after the fact. Pretraining-level curation that models calibrated epistemic behaviour from the beginning would be more robust, though substantially harder to implement.