laserbrain · research · proved, measured, and what did not work

laserbrain research

The theorem below is about the maths: to tell whether an agent’s stated goal has moved, you need a fixed reference, and a monitor that only watches its recent self cannot do it. It does not say the live check catches real drift. In real use it caught little: 1 of 16 live · one session · AI-labelled drifted helper agents and 0 of 10 live · one session · AI-labelled main-session misreads. Whether returning an agent helps was tested three ways and not established. laserbrain is a personal research project and is not for sale.

Nov 2026 · 60 days of real use, and a quieter setup on our machine

We watched the check on one machine for 60 days. Four numbers from that work, misses first. Each grade says how far the number reaches.

On 2026-11-20 our own machine switched to a quieter setup: no step reminders, an outside judge watching helper agents, and the guard as the only thing that blocks. It is not what pip installs.

1 of 16
drifted helper agents caught by the live check in a labelled month (12 had left their task; 4 were handed a task that missed the ask)
live · one session · AI-labelled
0 of 39
of laserbrain's own alarms over 60 days were true catches; 52 more only repeated an agent that had typed stuck
live · 60 days · one AI reviewer
about 10%
of the agents' model responses only made a check under the old every-step design: 8-10% of tokens, 12.5 agent-hours, a median 5.3 s each; 87% of check calls came from helper agents
live · one machine · 60 days
9 of 12
drifted helpers flagged by the outside judge re-scored with only what it sees live; 3 false flags among the 519 clean helpers it could score; one of the 12 never checked in, so it could not be judged
95% interval 47-91%
dev · replay · one session · AI-labelled

does not run from the package · runs only on our machine

what we measured, misses first →

The maths is proved. A benefit was tested three ways and not established.

Eight studies. Each names its verdict and the file it comes from. The real-use one is the newest and the least flattering.

detection · a theoremproven

The fixed reference

Can displacement from a goal be detected at all, and can an agent do it by watching only itself?

A proof over agent states in a metric space, with the ground state as the goal as first spelled.

A fixed, unchangeable, findable reference is necessary and sufficient to detect when an agent has left the ball around where it started, in O(1) memory. No monitor that compares an agent only to a window of its recent states can be both sound and complete, for any window and any rule. So the proof forces the reference never to change. It says nothing about how often real drift is caught.

source · laserbrain: research/PROOF.md

the metric · by constructionbuilt

A real displacement

Is the reference a genuine measure?

The JSON grammar the agent spells its state into carries a pseudometric.

Displacement Φ is well-defined, deterministic and symmetric, so “distance from ground” is a real quantity. Which grammar is rich enough for a given agent’s reasoning is a separate, open question; the theorem covers a fixed reference, never a particular vocabulary.

source · laserbrain: research/SUBSTRATE.md

coverage · H2 · N = 18measured

Shorter runs

Does an agent with the harness stop sooner than one left to run?

Open-ended tasks, control against harness, replicated.

Yes, runs with the harness stopped sooner. The caveat travels with it: fewer steps is not fewer tokens. Steps do not charge the monitor for its own calls, and a run can be short and expensive. Whether the shorter run gave a better answer was tested three ways and not established.

about half the steps
open-ended runs stopped sooner (median 5 against 10, N = 18); a better result was tested three ways and not established
offline

source · laserbrain: research/STUDY.md; research/CLAIM.md

benefit · H1 pilot · N = 12inconclusive

The judge could not tell

Does the early return keep the answer as good as running longer?

A blind, stronger judge scored every pair in both orders; a win had to survive the swap.

The preregistered rule read “supported”, but the judge often disagreed with itself. Judging open-ended answers that have no right answer is near a coin flip. The pairs where the harness acted were ties or clean losses (the return produced a worse answer). Consistent with the hope, dominated by noise, not a result.

source · laserbrain: research/GRADER.md

benefit · ground-truthed · N = 15negative

Where there is a right answer

And on tasks that do have an objective answer?

Debug and loop coding tasks with hidden unit tests: Pass@1, no judge needed.

The harness did not help. It trailed the control, and every ceiling failure was a run where it intervened. That fits the theory: a task with its own built-in criterion does not need an outside reference, and the nudge derails a run that was fine. laserbrain is not for well-specified, test-backed work. Its domain is open-ended work, which is exactly where quality has no ground truth.

12 of 15
coding tasks with hidden tests passed with the harness, against 15 of 15 without it, at 4.4 times the tokens; suggestive, not significant (p about 0.11)
offline · N = 15 per arm

source · laserbrain: research/CLAIM.md

benefit · powered re-run · N = 16inconclusive

The panel, in its own domain

So we ran it only where it should help, with the best measure we could afford.

Criterion-absent tasks, a three-judge panel, each pair in both orders, and a rule fixed before any data: if the judges cannot agree, the verdict is inconclusive.

The panel could not agree, so the output is inconclusive: the pilot’s problem confirmed at scale. Criterion-free answers cannot be reliably judged even by three models, and the κ-gate refused to read a verdict from a broken measure. It was not neutral, though: the harness rarely acted, and where it did and the judges could agree, the control won. Cost was swamped by noise.

source · laserbrain: research/RETEST.md

live · AI-labelled real sessions · one developer · 2026-11negative

In real use

On real working sessions, labelled by AI agents, does the live check catch the drift that happened?

Real sessions with known drifted helper agents and known main-session misreads, plus 60 days of alarms labelled by an AI reviewer.

Mostly not. It caught 1 of 16 live · one session · AI-labelled drifted helper agents and 0 of 10 live · one session · AI-labelled main-session misreads, and none of its own alarms over 60 days was a true catch (0 of 39 live · 60 days · one AI reviewer). An outside AI judge did better on the same helpers: 10 of 12 offline · one session · AI-labelled with each helper’s final report in view, which it does not have live, and 9 of 12 dev · replay · one session · AI-labelled re-scored with only what it sees live. That judge does not run from the package and runs only on our machine today.

source · one developer’s working sessions and a 60-day log on one machine (local, not published)

multi-agent · M1 · N = 40inconclusive

The echo spiral

A group of agents has a failure a single one does not. Can a fixed reference catch it where the group cannot?

Checkable-answer tasks, so resolution is objective (no judge). Two transcript-only arms, the fixed reference (agents agree and add no new ground) against a recent-history baseline, scored by balanced accuracy, with the decision rule frozen before the run.

Run, and NULL by the frozen rule: the fixed reference did no better than chance; the same in the unseeded split that carries the claim. The fixed reference did fire where the recent-history baseline stayed silent, and leaned the right way, but two capable models resolved most tasks, so there were almost no real spirals to catch. The labels were noisier than “checkable” promised, too: one answer key was wrong. Not established; the multi-agent extension stays a prototype.

source · laserbrain: research/M1.md

the proof · why the reference must be fixed

A monitor that watches only its recent self cannot detect displacement. A fixed reference can.

The detection result is a theorem, proved against every monitor that compares an agent only to its recent self, not just the easy ones. Here it is, in full.

the model

A run is a trajectory of states s₀, s₁, s₂, … in a metric space (X, d). The ground state s₀ is the goal as first spelled; displacement from it is Φ(n) = d(sₙ, s₀). For a threshold D > 0, the event to detect is E(n): Φ(n) > D. A self-referential monitor of width w decides from only the last w states, so after w steps s₀ has scrolled out of memory. A fixed-reference monitor keeps the single state s₀ and fires iff d(sₙ, s₀) > D. Sound means it never fires without E; complete means it always fires under E; “detects displacement” means both.

lemma · local coherence does not bound displacement

Let sₙ = Hₙ = 1 + 1/2 + ⋯ + 1/n and s₀ = 0. Consecutive steps d(sₙ, sₙ₋₁) = 1/n → 0, yet Φ(n) = Hₙ → ∞: the agent grows more locally self-consistent every step while wandering infinitely far from where it began. Displacement is the sum of the steps, and a sum of vanishing terms can diverge, so local coherence says nothing about displacement. ∎

theorem · no self-referential monitor detects displacement

For every width w, every threshold D and any decision function, a self-referential monitor cannot be both sound and complete. Fix a point p at distance d(p, s₀) = 3D and a gentle wander p = q₀, q₁, q₂, … that stays within D/2 of p. Trajectory A (displaced): s₀, p, q₁, q₂, …, so every Φ ≥ 3D − D/2 > D and E holds forever. Trajectory B (at home): p, q₁, q₂, … re-based on ground p, so every Φ ≤ D/2 < D and E never holds. For n > w the monitor sees the same w states on A at step n and on B at step n−1, and identical windows force identical decisions. So a complete monitor fires on A, hence on B, where nothing moved: a false alarm. What the window omits, no function of the window can recover. ∎

theorem · the fixed reference is sound and complete

F fires iff d(sₙ, s₀) > D, which is the event E(n), so it fires on exactly the displaced steps, storing one state at O(1) memory. On A and B above it separates them at once, because it kept the ground each one started from. The whole gap between the two theorems is a single retained state.

corollary · the three adjectives are the hypotheses

The reference must be unchangeable (let it move toward recent states and it becomes a windowed quantity, so the impossibility applies again), findable (readable every step, or F is not computable), and recursive (defined at every depth the agent nests to, so d(sₙ, s₀) is never undefined). Drop any one and the positive result fails at the named step. The shipped check re-bases after each user message, so the result applies within a request.

the boundary · proved to here, no further

Necessary and sufficient to detect displacement, and no more. What is not proved is a different kind of claim. Displacement is not pathology: F fires on deep, legitimate work too, and telling drift from progress needs a criterion the structure does not contain. The threshold D is unforced: fixed before data, or it is fit to the result. And whether returning helps is empirical: the studies on this page, not a QED.

the data · every run, with its numbers

Most runs found no benefit. Here is every one.

studynmeasurethe number
Detection · the theorem—proofnecessary and sufficient; no self-window monitor is both sound and complete
Displacement Φ—constructiondeterministic, symmetric, O(1) memory
Coverage · H218steps to stopabout half the steps offline; a better result not established
Benefit · H1 pilot12blind judge, both ordersthe judge often disagrees with itself; where it acted: ties or losses
Benefit · ground-truthed15Pass@1, hidden unit tests12 of 15 offline · N = 15 per arm with the harness, against all 15 without it
Benefit · powered re-run163-judge panel, κ-gatethe judges could not agree: inconclusive; where it acted, the control won
Live check · real use—AI-labelled real sessions, one developercaught 1 of 16 live · one session · AI-labelled drifted helpers; 0 of 39 live · 60 days · one AI reviewer of its own alarms were true
Multi-agent · M140balanced accuracy, checkable answersthe fixed reference did no better than chance: null; too few tasks were left unresolved to power it

Every study’s measure and decision rule was written down and frozen before its run. The real-use row is AI-labelled sessions from one developer, not a preregistered study.

what it saves

What it saves: a comparison that closed with no result

We do not know what it saves. The one kind of evidence that could answer it is a controlled comparison: the same work with the check and without it. We ran one. It is closed, and it produced nothing usable.

The design was sound. Half of the units got a verdict and half had it withheld, assigned at random, with the stopping rule registered before the first unit. Three things went wrong in the run. It stopped early, with the arms unequal and smaller than the size registered. Nearly all the units were segments of one long session rather than separate sessions, so in the dimension that mattered it was one sample. And the outcome could not be joined back to the runs it was meant to score, so even a clean split would not have produced a number.

Status: closed, no result. A replacement needs whole sessions from more than one person, and an outcome recorded when each unit is assigned, not rebuilt afterwards.

What the checking cost is measured. Under the old every-step design, about 10% live · one machine · 60 daysof the agents’ model responses only made a check.

the ledger

What we may say, and what we may not

we may say

  • A fixed reference is provably needed to detect displacement from a stated goal; a monitor that watches only its recent self cannot. That is the maths, not a catch rate.
  • Within a request the reference does not change. After each user message it becomes the agent’s own restatement of the new ask, so the theorem covers each request, not a whole session.
  • The core check gives the same verdict for the same input. The AI judges do not always agree with themselves.
  • On open-ended tasks it stopped runs in about half the steps offline. A better result was tested three ways and not established.
  • In real use it caught 1 of 16 live · one session · AI-labelled drifted helper agents.

we may not say

  • That it makes your agent better, or finishes the task: tested three ways, not established.
  • That it saves tokens or money: the comparison that could show it closed with no result.
  • That it helps on well-specified, test-backed work: measured, and it did not.
  • That it “catches drift” or “stops drift”: in real use it caught 1 of 16 live · one session · AI-labelled drifted helpers.
  • Any percentage made from these small counts.
  • That it is a product: it is a personal research project, not for sale.

Claim the maths, not the cure. The theorem holds. It does not make the live check catch real drift, which it mostly did not. The benefit is a study we ran three times and did not win. If a better way to measure quality on open-ended work appears, we will run it and report whatever it says.