Internal design document, rendered for transparency. Operational sections (deployment, job recipes, credentials) have been removed. Dated 2026-09-11; may be amended by later work recorded in the changes log.

report-2-post-adjudication-residual

This report measures RESIDUAL agreement after adjudication: Krippendorff's alpha on the same eligible-1922 frame once every queued dispute is resolved to its adjudicated label on both sides. It says how much disagreement survived adjudication, not how well two coders agree independently, and must never be quoted as an inter-coder reliability figure.

Construction. queued rows resolved to (adjudicated, adjudicated); all other eligible rows keep their original (coder-1, coder-2) pair — the 2026-08-02 'observed, not ceiling' construction

Read it as. an upper-bound-shaped statistic: agreement is forced on every row an adjudicator touched, so it rises with the number of queued disputes and cannot be compared with report 1 as if both measured the same thing

Threshold. 0.70 (human-human) — point estimate against the recorded threshold; the 95% bootstrap CI is reported for context and never gates (docs/technical_blueprint.md:238)

Not this report. independent round-1 reliability is report-1-round1-independent; machine accuracy is report-3-machine-vs-gold

Frame. eligible-1922 — 1922 documents.

axis	n	alpha	95% CI	passed	reason
actor_alignment	1922	0.8773	0.8596 to 0.8938	True
function	1922	0.6980	0.6738 to 0.7211	False	below threshold: 0.6980 < 0.7
theme	1922	0.8128	0.7919 to 0.8314	True

axis	published	computed	abs diff	within 1e-3
actor_alignment	0.8773	0.877302	0.000002	True
function	0.6980	0.697956	0.000044	True
theme	0.8128	0.812750	0.000050	True

input	sha256
`FOR-PROFESSOR-2026-07-25/results/labels_coder1.csv`	`ac182d198bf122c7...`
`FOR-PROFESSOR-2026-07-25/results/labels_coder2.csv`	`ac18cbd7be1cb670...`
`01-08-requests/03-PI-ruling-3-artifact-rows/artifact-rows-78.csv`	`ca068865472bef84...`
`adjudication-2026-08-02/exclusion-table-78.csv`	`d6ece6aa6d89557f...`
`02-08-tasks/01-adjudication-actor-alignment/adjudication-queue-actor.csv`	`1e6213cc1e42aa10...`
`02-08-tasks/02-adjudication-theme/adjudication-queue-theme.csv`	`c4235bd518b62729...`
`02-08-tasks/03-adjudication-function-cell/adjudication-queue-function.csv`	`35713dfc7189f75c...`

seed reliability-frames-20260911, 2000 bootstrap resamples, generated 2026-09-11T04:58:12.268855+00:00.