What Harness² is
Harness² is work from Google Research, published in October 2026 under the Apache-2.0 license. The central idea: the language model stays fixed, and what gets optimized is the “harness”, everything that surrounds the model in a task, such as the instructions, the guards, the memory and the guidance on tools.
The central step has three beats. An editor proposes changes on two fronts: the procedure (how to do it) and the expert knowledge (what to know). Each candidate runs on the same task. The final version is composed from the contrast between runs, with no answer key. “Keep the baseline” is a valid answer, and the final choice is made blind, judging coverage, then correctness, then fidelity.
After each task a lesson remains, in the format trigger, lesson, evidence and confidence. That record is what makes the harness improve from one round to the next.
Whoever writes the prompt
- Changes the instruction
- Runs a few cases
- Looks at the result
- Decides by eye
Every change becomes an opinion. Nobody knows whether it improved or whether the case was easy.
Workshop
- Proposes candidates on two fronts
- Runs all of them on the same task
- Composes from the contrast, blind
Doctor
- Publishes, or not
The ruler decides what is better. The person decides whether it goes to production.
How we use it at Estação Clínica
We read the paper on October 20 and, the same day, designed the Estação Clínica “agent workshop”. Only the concept came in: no code, prompt or text was copied. The first target was Escriba, the agent that writes the consultation note; then came the summary, the second reading of exams, WhatsApp and Irene, the WhatsApp nutritionist.
Two pieces of the method were refused on purpose. Automatic adoption of the winning version: at Estação, only the doctor publishes. And the “environment facts” recorded along with the version: in healthcare, that would be patient data inside a prompt.
Estação Clínica
Agent workshop- 1ProposeAn editor suggests candidates for the procedure and for the expert knowledge. It does not see the answer key.
- 2TestEach candidate runs on the same synthetic consultations, and a ruler in code measures recall of facts, of numbers and unsourced numbers.
- 3ComposeThe final version comes out of the contrast between runs. If no candidate wins by a clear margin, the answer is “keep”.
- 4PublishOnly the doctor. The workshop delivers a report; a person decides.
- Runs only in a synthetic tenant, deleted at the end.
- Every call goes through the pseudonymization funnel.
- The ruler is code, not the model grading itself.
- The report carries counts and hashes, never consultation text.
If the workshop does not convince, nothing changes: the version in production stays the same.
What happened in three rounds
In the first round, on Escriba, the baseline already had 100% recall on the test set. The workshop cost US$ 0.15 and answered “keep”. An identifier guard blocked the knowledge front on a false positive, fixed later.
In the second, we built a hard set of 30 consultations, 18 for training and 12 for validation. The workshop adopted a candidate: trap errors dropped from 2 to 0, for US$ 0.85. It was too good. We repeated the validation three times and the candidate failed: number recall dropped from 96.1% to 90.2%, and unsourced numbers rose from 2 to 4. The “2 → 0” was noise from a single round.
In the third, 126 proposals for US$ 1.28, and “keep” again. The most expensive lesson of the month was the rule that stayed: no candidate gets in without three repetitions on validation.
| Round | Set | Workshop decision | Confirmation with 3 repetitions | Cost |
|---|---|---|---|---|
| 1 · Escriba | Test set, baseline with 100% recall | Keep | Not needed | US$ 0.15 |
| 2 · Escriba | 30 hard consultations, 18 for training and 12 for validation | Adopt (trap errors 2 → 0) | Failed: number recall 96.1% → 90.2%, unsourced numbers 2 → 4 | US$ 0.85 |
| 3 · Escriba | 126 proposals | Keep | Not needed | US$ 1.28 |
When the gain was in the model, not the harness
The same ruler served a question Harness² does not ask: what if the problem is the model? We ran Escriba with Opus 5.5 and with Sonnet 5.5 on the validation set. Opus got everything right and cost 2.6 times more per note. For a consultation note, where an unsourced number is a clinical risk, Opus became the recommended model.
Recall on the validation set
Sonnet 5.5Opus 5.5
Consultation facts in the note
Sonnet 5.5: 98.3%Opus 5.5: 100%Consultation numbers in the note
Sonnet 5.5: 96.1%Opus 5.5: 100%
Cost per note
2.6×
Opus costs 2.6 times more and gets no number wrong
Unsourced numbers, in the 12 validation consultations
A number in the note that is in no line of the consultation.
See the numbers as a table
| Sonnet 5.5 | Opus 5.5 | |
|---|---|---|
| Fact recall | 98.3% | 100% |
| Number recall | 96.1% | 100% |
| Unsourced numbers | 2 | 0 |
| Cost per note | US$ 0.008 | US$ 0.021 |
The second reading of the exam
The second target was the reading of exam results, where the classic error is adding up the leukogram fractions wrong. On 12 synthetic reports with traps, Sonnet 5 with today’s instructions got 4 wrong; with a written rule about the total, it got 0 wrong and was cheaper; Haiku 4.5 got 0 wrong with today’s instructions.
The two real workshop rounds on this target answered “keep”, for US$ 0.095 and US$ 0.085. Nothing was published by the workshop. The total rule came in through the normal path, with review, and in the real tenant the share of reports read only by AI dropped from 28.9% to 17.4% between version 3 and version 4. That comparison is not clean: the reader in code also changed between the two.
| Reader | Trap errors (of 12 reports) | Round cost |
|---|---|---|
| Sonnet 5, today’s instructions | 4 | US$ 0.26 |
| Sonnet 5, with the total rule | 0 | US$ 0.19 |
| Haiku 4.5, today’s instructions | 0 | US$ 0.11 |
What we don't know yet
- 01
All synthetic
The workshop only runs on synthetic consultations and reports. That is the condition for existing in healthcare, and also the limit: the real distribution may be different.
- 02
A single round lies
The “2 → 0” candidate passed in one round and fell in three. Any number from a single run, including the ones in this note, is worth less than it seems.
- 03
The comparison in the real tenant is not clean
Between version 3 and 4 of the exam reader, both the rule and the code changed. The drop from 28.9% to 17.4% in the “AI only” share cannot be attributed to one thing alone.
- 04
Workshop cost against the gain
Three rounds cost less than US$ 2.50 and produced one process rule and one “keep”. It pays off as a decision method; it is not a machine for automatic gains.
Next steps
- In progress
1. Three repetitions as the rule
No candidate gets in without three runs on validation, on every target.
- Then
2. Workshop on WhatsApp and Irene
The targets have existed since late October; rounds with their own hard set are still missing.
- Horizon
3. Lessons that cross targets
The trigger, lesson, evidence and confidence record feeding the next target, not only the same one.
The ruler decides what is better. The person decides what gets in.
If the workshop proves its value on more targets, it becomes the door every agent change passes through: proposal, test on the same task, contrast and a report for whoever publishes. No answer key and no automatic adoption.
- Keep the baselinewhen no candidate wins by a clear margin
- Report for the doctorwhen a candidate passes three times
- Lesson recordedalways, with trigger, evidence and confidence