Consolidated judge feedback

Every written comment about a judge, turned into one coherent read they can act on, on a page only they can reach. No scores, no round names, no teams, no quoted phrases — and a person reads it all before it goes anywhere.

Open the sample How to run it

The problem it solves

At the end of a tournament, a judge has a dozen comments written about them and no way to read them. Handing over the raw comments is not an option — a distinctive turn of phrase identifies its author inside a panel of three, and half the comments are arguing with a decision rather than describing the judging. So in practice the comments sit in the tab, and the judge learns nothing.

What a judge actually wants is what a good adjudication core member would tell them over coffee: here is the pattern across everything people said, here is what came through strongly, here is the thing to work on. That is a summarising job, and it is the kind of summarising a language model is genuinely good at.

The whole design is about making it safe to do that.

What a judge sees

One judge's page. Reached only by their own private URL — there is no index, no search, and no way to get to anybody else's.

The rules, and where they came from

These are not stylistic preferences. Each one traces to a specific thing that would go wrong without it, and a draft that breaks one is thrown away rather than softened.

Rule	Why
No scores. No digits at all.	Not a feedback score, not an average, not a rank, and not a count of how many people wrote in — because “three people said” is a score with the numerals taken out. Banning every digit kills scores, round numbers and counts in one stroke.
Nothing that says which debate it came from.	Comments argue with a call and quote the motion to do it. Repeat any of that and the judge can identify the round, and from the round the panel, and from the panel the author. No round or stage names, no motion, no topic, no argument content, no country.
Nothing from or about the teams and panels.	No team, school or person named. And no attribution at all — not “a team felt”, not “your chair said”. Saying whether a comment came from a team or a co-panellist narrows it to one of three people.
Never quote. Always paraphrase.	A distinctive phrase names its author more reliably than a signature. No run of words is reused from any source comment.
A person reads it before it is sent.	This goes out under your adjudication core's name. See the review step.

What is deliberately allowed matters just as much. Generic craft vocabulary stays legal — the comparative, the bench, deliberation, the opening half — because banning it strips out the feedback worth reading and leaves platitudes. And that vocabulary is chosen from your format: British Parliamentary words would be nonsense at a two-team tournament, so which set is in play is read from your tab.

How it is enforced: mask, then gate

Two mechanisms, and the split is the whole design. The mask is the cheap fix. The gate is the proof.

The mask, before the model sees anything
Every name, team, school, code and round in the comments is replaced with a placeholder — [a person], [a team], [a round] — before the comments are sent anywhere. The model cannot repeat what it was never given.
The gate, after each draft, with a redraft
Every draft is checked against every rule. If anything fails, the specific violations are handed back as an edit instruction and it redrafts, up to four times. A leak is not something the reviewer has to catch by eye.

On the demo tournament, 18 of 37 needed at least one correction and one needed four. That is not the model being careless — it is the rules being tighter than ordinary writing, which is the intended trade.
The gate again, at build time
Because a hand-edit leaks just as easily as a draft. The build refuses to publish if anything fails.

The checks Technical

In feedback/gate.py. The sharpest is no digit anywhere. Then: no score, rank or average vocabulary; no counted people — but “one person” is allowed, because that is how you honestly report a single view, so the ban starts at two, where a count starts behaving like a score; no round or stage name; no side label; no attribution verb attached to a team or a panellist; no participant, team, school or code; no country or city; and no seven-word run shared with any source comment.

The vocabulary is built from your tab, not typed in — every name, team, school, code, round name and motion is pulled and turned into a ban list. Two things that list needs care with, both of which cost real time to learn:

A short upper-case round abbreviation must match case-sensitively. A grand final abbreviated OF, matched case-insensitively, turns every “of” in every comment into [a round].
A name that is also an ordinary English word must stay legal on its own. Someone surnamed Long is why “long-winded” must not become “[a person] winded”. Full names and adjacent name pairs always match; the lone token gets an escape hatch.

Motion words get the same treatment, and this is the part that had to be rebuilt to make the tool portable. Banning every word that appeared in a motion is safe and useless: motions are made almost entirely of ordinary English, so it bans “work”, “land”, “large” and “individual”, and you cannot write about judging without those. The test is therefore whether the word is distinctive — checked against a frequency list of common English, which is a property of the language rather than of one tournament. On the demo's motions that leaves eleven banned terms, all of them the kind of word that would identify a round.

Break-category names get an escape hatch for the same reason. A category called Open must not ban the word “open”, or no summary can say “open to persuasion”. A category called Novice or ESL is distinctive and stays banned.

Who can read what

A judge's page is addressed by a hash of their private Tabbycat URL. That one fact is the entire access model, and it is deliberately the only one:

The published site contains no private URLs, so a copy of it is not a set of credentials.
There is no index, no listing and no search, so holding the site tells you nothing about who is in it.
Names live inside the files, not in their addresses, so a directory of hashes leaks nothing even if a host were to list it.
A judge's page is reachable only by someone who already has their private URL — which is exactly the property Tabbycat already relies on for ballots.

Technical The file is sha256(url_key).json, hashed in the reader's browser, and the content security policy is connect-src 'self'. The check that earned its keep fastest: the build refuses to publish if any private URL key appears anywhere in the output — and it caught one immediately, because the first version of a code comment explaining the URL format used a real judge's key as the example.

The step you should not skip

./review exists because a person has to read this. The summaries are ordinary files in feedback/summaries/. Open some. Edit any of them by hand — the build never regenerates what you have edited. Then publish.

Everything else here is machinery for making the output safe to read. It cannot make the output right, and this is judge feedback going out under your adjudication core's name.

When almost nothing was written

A judge with two comments does not get a paragraph of pattern-finding invented out of two lines. They get a short, plainly-worded note saying little was written. On the demo tournament that is 7 judges of 37; the threshold is in tournament.json and three is a sensible floor.

Inflating thin feedback is worse than admitting it is thin, because a judge who is told a confident story about themselves from two comments will believe it.

What is deliberately absent

The “did you agree with the decision?” answers. An agreement rate is a score by another name.
Any count of how many people wrote in. Also a score.
Any way to look up somebody else. No index, no search, by design.
Any automation. This is written once, read by a person, then sent. There is no scheduled job and there should not be.

How the model is called Technical

summarise.py shells out to the claude CLI with tools disabled, slash commands disabled, dynamic system-prompt sections excluded, and its working directory in a temporary folder — a pure text transform with no tools, no skills and no project context, because anything else is a route for something unrelated to end up in a judge's feedback.

prompt.md is the whole instruction set and is meant to be edited. Three values in it are filled in from your tournament at run time: the format, the craft vocabulary that format uses, and the side labels to avoid. Everything else is prose you can rewrite to suit your own adjudication core's voice.

Next: how it all works underneath →