assay · which model does which job
Model tiering
Running a fleet means constantly answering a question single-agent work never asks: which model should do this piece? Left to the moment, that question is answered by convenience — whatever session is open takes whatever comes next. Assay answers it in advance instead, as a property of the work rather than of the worker, and writes the answer into the brief where a dispatcher can read it.
The asymmetry the policy is built on
Tiering is usually framed as a cost trick. That framing gets the mechanism right and the reason wrong. The reason is an asymmetry in how errors travel:
- Errors made while authoring compound. A wrong risk answer, a missed dependency, a mis-identified head of the critical path — each of those propagates through every implementer that reads the brief and every review that trusts its framing. Nothing downstream is positioned to catch it, because everything downstream takes the brief as its premise.
- Errors made while implementing get caught. That is what the verification and review gates are for. An implementation defect meets a Verify table, a reviewer that did not write it, and a non-author re-running the same checks on merged mainline.
So the policy is not "use the cheap model where you can get away with it." It is: put the expensive judgment where nothing downstream can repair it, and put the cheap execution where three separate mechanisms are already watching. Cheap implementation is safe because of the gates. Remove the gates and the same policy becomes a way to ship worse work faster.
The default: effort keys the tier
Every brief already carries an effort estimate for scheduling. That same field keys the execution tier, so the common case needs no extra decision:
| Effort | How it runs | Why |
|---|---|---|
S |
May run inline, at whatever tier the current session is. | Dispatch overhead dominates the work. Handing off a ten-minute job costs more than doing it. |
M / L |
Planned at the author's tier, then dispatched to a cheaper implementer behind the verify and review gates. | Executing substantial work inline at a strong tier is the cost leak the discipline exists to avoid — and the gates make the handoff safe. |
One consequence is worth stating on its own, because it is the part people try to route around: a session cannot downlevel itself. There is no way for a strong session to decide it will "act cheap" for the next hour. Dispatch is the only way down — which means the tier of a piece of work is always a fact about a real, separate session, observable from outside, rather than an intention held privately inside one.
The override: when effort is the wrong signal
Effort measures size, and size is usually a good proxy for difficulty. Usually. A brief can be small and still need judgment that a cheap tier cannot supply — a one-file change to an authorisation path, say. For those, a brief carries an optional field asserting a minimum execution tier:
exec-tier: strong exec-tier-why: correctness depends on cross-artifact reasoning about the shared value
Three properties keep it from becoming a preference field:
- It is derived, not chosen — a deliberate mirror of how the risk gate works. The author answers three questions about the work, and any yes makes it
strong: does the Task need design decisions the stated facts do not fully pre-specify? Does correctness depend on reasoning across components or artifacts — a shared value, an end-to-end flow, a pattern swept across many sites? Is it code where a subtle error would survive the brief's own tests? - It only ever tightens. The field can raise a brief above its effort-derived default. It cannot be used to widen one downward — there is no way to mark a large brief as cheaper than the effort rule makes it.
- It obliges the receiver to refuse. A session running below the asserted tier is required to hand the brief back rather than attempt it. Refusing is the cheaper path: a cheap draft does not merely waste its own run, it anchors the strong session that reviews it.
The field is optional, and its absence is a real default rather than a missing value — an unmarked brief is any, meaning the effort rule governs.
What the board does with it — and what it deliberately does not
The generator treats the field with a light touch, on purpose.
- It validates the value. An unrecognised tier is a lint problem naming the file and the offending string.
- It asks for the reasoning. A brief claiming
strongwithout the one-line rationale draws a notice pointing at the derivation questions — enough to make an unexplained escalation visible, not enough to block a merge on prose. - It renders a marker. A
strongbrief shows as[exec:strong]in the work queue and in the awaiting-verification board, so a dispatcher scanning the queue sees the constraint without opening the file. - It is never a score input. The tier does not raise or lower a brief's position in the queue. Difficulty is not importance, and letting the tier field touch the ranking would quietly convert "this needs a careful implementer" into "this goes first."
That last restraint is the design: the field is a signal to the dispatcher, and the enforcement lives in dispatch and in the review gate — not in the board.
Tier by role, not one blanket rule
Execution tier is the most visible application, but each desk role has its own rule, and they do not all point the same way.
| Role | Tier rule | Reason |
|---|---|---|
| Authoring a brief set | Strong tier, always. An economy session asked to author is instructed to stop and hand back rather than produce a draft. | Authoring errors compound downstream; a cheap draft anchors the strong reviewer instead of saving its time. |
| Front-door triage | Strong tier for classification and scoping; an economy session may do the mechanical parts only, and never classifies. | Routing decides whether something becomes work at all. The front door needs a mind, not a trigger. |
| Implementing | Effort-keyed, as above, tightened by the brief's asserted minimum. | The gates downstream are what make the cheap tier safe here — and only here. |
| Reviewing | Risk-keyed, explicitly not a blanket rule. A brief with all four risk answers clear may be reviewed at any tier; a risk-flagged one gets a strong reviewer or a human. | Defaulting every review to the top tier spends the budget where nothing is at stake and teaches nobody to read the risk fields. |
| Verifying | The local session tier — and it does not escalate to a bigger model. Where a stronger check is needed, the escalation is to a human. | See below. This is the rule that surprises people. |
| Coordinating | Top tier, spent only on judgment, arbitration, and checking other agents' output. Everything mechanical is fanned out. | The widest-context role is the one whose time is most easily consumed by work that did not need it. |
Why verification does not escalate to a bigger model
Every other rung on this ladder answers "not sure enough" with "use a stronger model." Verification does not, and the exception is deliberate.
A verification gate exists to answer a question the implementer cannot be trusted to answer about itself. Escalating that gate to a larger model of the same kind buys a more capable reader — but not an independent one in the sense that matters. The failure modes correlate: a stronger model asked whether a plausible-looking piece of work is done gives a more articulate version of the same yes. So the ladder's upper rung is not a bigger model at all. A risk-flagged brief cannot be signed off by a model; it goes to a named human, and the board's own lint enforces that floor by refusing to accept an economy-tier runner in the verified cell of a risk-flagged row.
The practical effect is that the tiering policy has a ceiling written into it. Above a certain risk, more compute is not the answer the system reaches for.
Tiers, not vendors
Nothing on this page names a model. The field's values are any and strong; the roles above speak of economy and strong tiers; the assertion a brief makes is about a class of capability, not a product.
This is not squeamishness about naming suppliers. It is that specific assignments go stale faster than the documents that carry them — a review of external material in the toolkit makes exactly this finding about tier assignments lifted from published sources, keeping the principle and discarding the names. A brief authored today naming a particular model would be asserting something that stops being true without anything changing in the brief. A brief asserting strong stays true, and the mapping from tier to product is configuration that a project revises when its options change.
The same reasoning keeps a category of number off this site entirely: no productivity multiplier, no throughput figure, and no tier-comparison result. The methodology's own operation generates such numbers, and they have no measured baseline, no quality adjustment, and no controlled design behind them. Refusing to quote a figure we cannot recompute is the position.
What is checked, and what is honour-system
What the machinery checks. That a declared tier is one of the recognised values; that a brief claiming the strong tier carries a reason; that the marker reaches the queue where a dispatcher will see it; and that a risk-flagged brief does not reach verified or done with an economy-tier runner recorded against it.
What it does not check: which model actually ran. Nothing in the toolkit observes the tier of a running session. A dispatcher is instructed to route a strong brief only to a strong implementer, and an under-tier session is instructed to hand it back — but pickup-side compliance is self-report. The enforced surfaces are dispatch and the review gate; the field itself is a signal, not a lock. A page about verifiable claims that implied otherwise would be making exactly the kind of claim it tells you to distrust.
The second tiering surface is advisory throughout. A stream may also carry free-text dispatcher guidance at the stream level. It renders into the board's notes and is never enforced and never scored — useful as a default for a whole body of work, but not a rule anything applies.
A gap we can see in our own tooling. The brief template a new adopter copies does not mention the tier-override field at all; it is documented in the specification and in the authoring instructions. An author working from the template alone would never learn the field exists, and would fall back to the effort-keyed default — which is a safe fallback, but not the intended path.
Not covered here. This page covers tier policy: which class of model does which job and how a brief asserts a floor. It does not cover the harness mechanics of dispatching to a chosen model, cost accounting or budgets, how a project maps its own tiers onto available models, or what to do about a session whose tier changes underneath it mid-run. Those are project configuration, and they vary between projects running the same methodology.
Reading
Running an agent fleet: the operator's view
From one session on one machine to several loops at once — including where the tiering policy stops paying for itself.
Article — publication pending
Why software teams need machine-checkable gates
The gates are what make cheap execution safe. This is the case for them, stated on its own.
Article — publication pending
These articles are drafted and referenced from the toolkit. Links activate on publication.