EVIDENCE

A governed, multi-agent scientific institution for AI — not another chatbot.

Four persistent, competing world models preregister falsifiable predictions before they see any outcome. Experiments run under a bounded compute budget. Blind judges evaluate the results. Credibility updates only from provenance-complete, policy-authorized evidence. Every belief is traceable back to the raw experiment that produced it.

EVIDENCE turns the scientific method into a verifiable, tamper-evident control plane for autonomous AI research. It is designed for a world where agents increasingly generate claims, recommend actions, spend compute, and modify shared state. The central principle is:

Reasoning authority must never silently become execution authority.

---

Table of Contents

1. Why EVIDENCE Exists
2. What EVIDENCE Does
3. Core Scientific Loop
4. System Architecture
5. World Models
6. Experiment Selection
7. Judge Panel
8. Nexla: Canonical Evidence Layer
9. Pomerium: Authorization and Governance
10. Akash: Controlled Compute
11. Persistent Memory and Recovery
12. Security Model
13. Frozen Contracts and Identifiers
14. Repository Structure
15. Quickstart
16. Testing and Acceptance Criteria
17. Demo Flow
18. What Is Already Implemented
19. What Is Still Missing
20. Recommended Next Steps
21. Why This Project Matters

---

Why EVIDENCE Exists

Most AI research agents can generate hypotheses, write code, run experiments, and explain results. However, the same agent often proposes the hypothesis and interprets the outcome after seeing it. This creates several structural problems.

Failure mode	What goes wrong
Post-hoc rationalization	Agents explain any outcome after it occurs instead of making a falsifiable prediction beforehand.
Premature consensus	Multiple agents copy the same evidence summary or reasoning and converge on the same wrong answer.
Unaccountable judges	Critics sound persuasive but are not evaluated against future evidence.
Lost scientific memory	Agents preserve conclusions but lose the experiments, assumptions, contradictions, and confidence history behind them.
Unsafe execution	An agent that can think about an action may also be able to execute it, spend compute, or alter evidence.
Identity bias	Judges may favor predictions from high-reputation agents when identities are visible.
Hidden data paths	Official scientific records may bypass governance through local files or databases.

EVIDENCE treats these as institutional problems rather than prompt-engineering problems.

---

What EVIDENCE Does

EVIDENCE creates a secure scientific community of competing AI world models.

Each world model:

• maintains its own structured beliefs;
• retrieves evidence independently;
• preregisters a numerical prediction;
• commits that prediction before seeing the result;
• proposes experiments that could falsify its theory;
• gains or loses credibility based on prediction quality.

The platform then:

1. compares predictions;
2. selects the most informative affordable experiment;
3. authorizes execution through policy controls;
4. runs the experiment on controlled infrastructure;
5. converts the result into canonical evidence;
6. creates anonymized judge packets;
7. evaluates the result using specialized judges;
8. updates beliefs and credibility;
9. records the complete chain of custody.

The system does not merely ask agents to debate until they agree. It preserves disagreement, forces agents to make falsifiable predictions, and updates trust from real evidence.

---

Core Scientific Loop

1. Restore persistent world state
        ↓
2. Retrieve canonical evidence independently
        ↓
3. Propose hypotheses and experiments
        ↓
4. Commit sealed numerical predictions
        ↓
5. Reveal predictions after all commits lock
        ↓
6. Select the most informative experiment
        ↓
7. Authorize the action
        ↓
8. Execute the experiment
        ↓
9. Canonicalize and validate evidence
        ↓
10. Create blind judge packets
        ↓
11. Run specialized judges
        ↓
12. Update world credibility and beliefs
        ↓
13. Record lineage and create snapshots
        ↓
14. Begin the next round

The loop ends only when the research question is resolved, the compute budget is exhausted, or the remaining hypotheses cannot be distinguished with available experiments.

---

System Architecture

flowchart TD
    U[Human User or Agent] --> P[Pomerium Gateway]

    P --> W[Web Application]
    P --> M[MCP ToolSets]
    P --> S[Scheduler Launch API]

    M --> O[Scientific Orchestrator]
    O --> WM[World Models]
    O --> CR[Commit-Reveal State Machine]
    O --> ES[Experiment Selector]
    O --> BU[Belief and Credibility Updates]

    ES --> PB[Policy Broker]
    PB --> S

    S --> EW[Experiment Worker]
    EW --> A[Akash Compute]
    A --> RE[Raw Experiment Event]

    RE --> N[Nexla Data Plane]
    N --> CE[Canonical Evidence]
    N --> RV[Role-Specific Evidence Views]
    N --> BJP[Blind Judge Packets]
    N --> LE[Lineage and Belief Events]

    BJP --> J[Judge Panel]
    J --> BU
    LE --> W

The architecture has three simultaneous integrity layers:

1. Identity and access integrity — Pomerium controls who may call which route or MCP tool.
2. Workflow integrity — the orchestrator and policy broker decide when an action is scientifically valid.
3. Epistemic integrity — Nexla determines whether a result is eligible to enter the official belief state.

---

World Models

The MVP includes four persistent world models.

Data World

Explains outcomes through:

• augmentation;
• dataset size;
• sample diversity;
• label quality;
• class balance;
• train-test distribution mismatch.

Optimization World

Explains outcomes through:

• optimizer choice;
• learning-rate schedules;
• warm-up;
• weight decay;
• regularization;
• gradient dynamics;
• convergence behavior.

Architecture World

Explains outcomes through:

• model depth and width;
• parameter allocation;
• representational bottlenecks;
• capacity;
• inductive bias;
• compute efficiency.

Skeptic World

Searches for:

• confounding variables;
• unmatched compute;
• seed effects;
• data leakage;
• unfair baselines;
• hidden configuration changes;
• unsupported causal claims.

Each world maintains its own belief state, contradictions, calibration history, supporting evidence, and unresolved questions.

---

Experiment Selection

The system selects experiments based on how much scientific uncertainty they can resolve per unit of cost.

Experiment Value =
Prediction Disagreement
× Scientific Importance
× Falsifiability
× Unresolved Judge Conflict
× Feasibility
÷ Estimated Cost
+ Minority Exploration Bonus

An experiment may be selected even when no world predicts the highest final accuracy, because its outcome may best distinguish competing theories.

A minority exploration budget protects low-probability but not-yet-falsified hypotheses from being eliminated too early.

---

Judge Panel

EVIDENCE does not rely on a single LLM critic.

Metric Judge

Deterministically evaluates:

• accuracy;
• loss;
• runtime;
• memory;
• compute cost;
• variance;

Built With

• authentication
• authorization
• fastapi
• jwt
• nexla
• open-ai
• policy-based
• pomerium
• pydantic
• python
• rbac
• react
• sha-256
• trust
• typescript
• vite
• zero