Two days before an ecosystem grant application was due, our risk report on the Midnight network said the one blocker left would clear "within weeks": the next compiler release was "overwhelmingly likely" to expose the gate primitive our circuit needed. One command falsified that sentence. The release was already out. It was installed on the machine that wrote the report. The primitive was still unbound — and, as a further re-run showed a day later, it had never existed in the language at all. Our circuit had failed to compile on syntax we had invented, and the report had turned that failure into a story about someone else's roadmap.
The report never reached the partner. What caught it was not seniority, taste, or luck; it was the structure the spike ran inside, and one rule in particular, which is the sentence this article exists to hand over: evidence is not evidence until someone who did not produce it has re-run it. The rest is the machinery that makes that rule cheap enough to follow under a deadline.
five assumptions, four rungs, no way to fail
We wanted to know whether our product — a service that detects drift by probing access rights, where a failed probe is the signal — could be rebuilt on Midnight, a zero-knowledge network where contract logic compiles to circuits and the chain verifies proofs instead of reading data. We had no toolchain installed, no circuit written, no infrastructure, no testnet account. Every estimate we had was derived from reading architecture documents — which is to say, every estimate was an assumption wearing a number.
So the spike's first artifact was not code but the assumptions themselves, written down — five, each load-bearing for the build decision: the toolchain stands up on our hardware; a circuit of our own is days of work, not weeks; a headless, no-GUI service path exists for proving and submitting; cost and latency fit the product's cadence; and a rejected proof is machine-detectable, because our product is the reject path — drift detection is a probe that fails on purpose.
Then a ladder of four small briefs, one rung per cluster of assumptions: stand up the toolchain and run the platform's own example; write our own minimal circuit and demonstrate both verdicts, accept and reject; drive both verdicts through a scripted service-account sidecar that emits JSON; then take it to the public testnet and write the risk report. Each rung carried a timebox — five and a half days across the ladder — and, more important, a kill-criterion written into the definition of done. The second rung's read: two days without a compiling circuit of our own producing both verdicts means the days-not-weeks assumption is wrong — stop and report.
That last clause is the design decision the whole spike rests on. Hitting a kill-criterion and reporting it is a successful outcome. The spike's product is evidence, not success; green, yellow, and red are all deliverables. A spike framed this way cannot fail as a project — which sounds like a comfort, and is actually a trap, because it removes every incentive to shade a result except the ones you brought with you.
the first report read greener than its artifacts
The ladder ran. The toolchain stood up on our arm64 machines in about an hour — a number that first shipped as "first proof in about an hour" and had to be corrected, because no proof had run: an hour bought a working toolchain and a compiled example, not the round trip. The correction is small, but it is the failure class in miniature: under a deadline, an honest team rounds "the part that worked" up to "the thing we wanted."
The pattern repeated at every altitude. The circuit report described its rejection path in the present tense — rejection "is machine-detectable via the indexer" — fifteen lines above the admission that nothing had compiled. A verdict earned by reading SDK source was scored as if the code had run. And when the public-testnet attempt crashed, the report quoted the crashed probe's memory setting — a 48 GB heap cap — as the cost of the sync that had actually succeeded under a 12 GB cap. A cap is not a measurement; quoting the failed run's cap as the successful run's cost inverted the two. The true working set was unmeasured either way, because a V8 old-space cap does not bound what a WASM-heavy process actually uses.
None of this was fabrication. It was optimism compiled into prose by the people with the most context and the least distance, wearing the report's own vocabulary of measurement. Our mechanical checks passed it: the brief's verify table included greps for the right words, and the words were all there. A grep is not a judge. The report contained the strings; it did not contain the substance.
someone who didn't build it re-ran it
What the process demands next is the part that saved us: the implementer stops at "implemented" and never grades their own work. An independent verifier — a different session, no stake in the outcome — re-ran the ladder from the written command sequences, row by row against each brief's verify table, on their own machine.
The re-run reproduced the good half exactly: the toolchain install and the example
compile re-derived cleanly, down to byte-identical proving keys — four separate
verifier runs eventually confirmed that half, four times. Then it reached the
claims. The compiler that was "weeks away" was already installed, and one compile
against it — compact compile +0.31.1 withcounter.compact — returned
unbound identifier onlyIf, exit 255: the primitive the roadmap story
hung on, still missing on the release that was supposed to add it. Three claims in
the reporting were falsified that way, by re-running the spike's own toolchain. And
the verdict on the determinative rung was not "failed" — it was "never ran": no
proof generated, no verdict observed, accept or reject.
The reports were rewritten with the retractions left visible in place, and the
rewrite carried a discipline the first version lacked: every verdict now names its
evidence class — measured, inspected, modeled, or unmeasured — and no verdict may
read greener than its artifact. Under that rule the corrected suite converged
fast. Our own circuit compiled on three compiler versions once the invented syntax
was replaced with the primitive that had existed all along. Both verdicts ran
on-chain locally with real proofs: the authorized key's probe accepted and
incremented the ledger counter; the unauthorized key aborted at the assertion
before a proof existed, with a distinct exit code and a stable machine-readable
verdict — {"verdict":"rejected","reason":"circuit assert violation: unauthorized"} —
which is exactly the detectable rejection the product needs.
The headless sidecar measured its latency as a distribution, not an anecdote: ten
runs, 16.1 to 22.9 seconds against a local devnet. The public-testnet leg stayed
blocked by an upstream wallet bug, so its cost figure shipped as what it was —
modeled, not measured — and the overall verdict shipped yellow, not green. The
recommendation followed the color, and only the corrected suite was cleared to
leave the building.
| assumption | final verdict | evidence class |
|---|---|---|
| the toolchain stands up on our hardware | green | measured |
| a circuit of our own is days, not weeks | green | measured |
| a headless, no-GUI service path exists | green | measured |
| cost and latency fit the product | yellow | latency measured (local devnet); on-chain cost modeled |
| a rejected proof is machine-detectable | green | measured (local devnet) |
The corrected scorecard. Every verdict names its evidence class, and none reads greener than its artifact.
the rules that travel
Nothing above needs our tooling. It needs a text file, a timer, and a colleague.
Write the unknowns down as assumptions before you install anything, and give every rung of the spike a timebox and a kill-criterion in the definition of done — including the sentence "hitting this and reporting it is a successful outcome." The wrong first move on an unknown platform is writing product code; the right one is deciding what evidence would change your plan.
Prove the reject path. If your product depends on detecting failure, a demo that only shows success proves nothing — demand both verdicts, and record the exact observable form the rejection takes, because that observable is the product.
Separate the implementer from the verifier. The person who ran the spike writes the report but does not get to grade it; someone with no stake re-runs the ladder from the written commands before any number leaves the room. Label every figure with its evidence class — measured, inspected, modeled, unmeasured — and hold one line: no verdict reads greener than its artifact. When a claim and its artifact disagree, the report is held, corrected in place with the retraction visible, and re-verified before it moves.
the hedge
This is one spike, by one team, on one technology — a sample of one for the method, whatever it proved about the platform. The latency figures are ten runs against a local devnet on one machine; they characterize our laptop's proving performance, not the network's. The on-chain cost leg was never measured, and the final report says so. The mechanical gates in this story caught nothing by themselves — word-presence checks passed a report whose substance failed, and the catch came only when a person actually re-ran commands; a verify table nobody re-runs is decoration. And independent verification is not free: the ladder was re-run in full several times in the weeks that followed — real cost, paid from the spike's own budget. We think it bought the only version of the report worth having.
The spike could not fail as a project, and it did not: it produced the decision-grade artifact twice — once wrong, once corrected. What made the second version worth acting on was not the ladder, the timeboxes, or the verdict colors. It was that every claim in it had been re-run by someone with no stake in it being green. Evidence is not evidence until someone who did not produce it has re-run it.