Incident event study — design

1. Purpose

Pick conflict incidents by a fixed rule, isolate the material about each one by topic, wherever it was published, trace how far, how widely and how fast it travelled, then compare the incidents to see whether a pattern repeats.

Two principles from the user shape everything below:

• Impact, not location. Where a post was produced, or whether it names a place, is not the question. A channel broadcasting from Moscow that never names the village can be the most consequential carrier. Place enters only on the incident side (UCDP gives each incident a region and a date); material is linked to an incident by what it talks about.
• The result is not known in advance. The study is exploratory. Every choice that could steer the outcome — which incidents, which search terms, the inclusion threshold, the reading rule for "pattern" — is fixed and fingerprinted before any impact figure is computed (§12). Nobody sees a curve or a reach figure until stage 5.

The reader is the user first, then the PI. Success is: sixteen incident profiles whose search quality is measured and stated, a comparison that separates repeated patterns from single-case observations by a rule fixed in advance, and a cannot-claim list as plain as the findings.

Framing (who is blamed, alignment, function) is out of scope for this pass; it is a second pass on the material this study isolates (§14).

1.1 Facts that shaped the design (prod, 2026-10-15)

2. Incident selection

2.1 Types (four, stratified)

Types follow UCDP's coding, not media framing. Moura is therefore an army incident; the gap between how UCDP and the media classify it is something the results may show, not something the selection corrects.

2.2 Mechanics

1. Incident = merged events. Events with the same type, same side_a/side_b, same country and same region (gid_1, else adm_1), each starting ≤ 3 days after the cluster's first event, form one incident. Deaths are summed; the incident date (day 0) is the earliest date_start.
2. Date precision. The first event must have date_prec ≤ 2 (exact day or a range of a few days). Speed is measured from the first possible day; the incident's precision is shown beside its speed figures.
3. One per type per year, 2022–2025, deadliest first across types. Within a year, all candidates of the three violent types are queued by deaths (civilian deaths for civilian), and each type takes the first valid candidate in that queue. Coups are placed before everything.
4. No overlap within a region. A candidate whose window (§4.1, day −14..+45) overlaps an already-chosen incident in the same region is skipped. A coup blocks its whole country. Incidents in different regions of the same country may overlap; they are then a close pair (§5.4).
5. Empty year stays empty. No borrowing from other years.
6. Frozen output. incidents.yaml: the chosen incidents and the full ranked candidate log (every candidate considered, with the reason it was picked or skipped). The user reviews it; it is fingerprinted before stage 2.

2.3 Probe result (indicative; the frozen file is authoritative)

A throwaway probe of these rules on prod ucdp_event (2026-10-15) gave:

Skips logged: Hombori (russian 2022, 50) overlaps Moura in Mopti; Koulikoro (russian 2025, 65) has date precision 4.

2.4 Known consequences (accepted)

• Close pairs: Moura / Ménaka (2 days apart, Mali), BF Est / Barsalogho (16 days, Burkina). Handled by §5.4.
• Niger appears only through its coup: its violent events are never the deadliest of a year.
• Severity differs by type: russian incidents are 28–55 deaths, the others 100–400. Comparisons across types use relative measures (§7.4) and the severity view (§8.3).

3. Search inputs (stage 2)

For each incident, drafted by Claude from the UCDP record(s) and their source headlines, then corrected by the user, then frozen:

• Place terms: place names with spelling variants and transliterations in fr / en / ru / ar (e.g. Moura, Mourrah, Мура, مورة).
• Event terms: what happened and who — perpetrator and target names, distinctive details (a death toll, a named village, "coup", "massacre", "attaque", "резня", "مجزرة" …).
• Standalone terms (optional): terms specific enough to identify the incident alone inside its window (e.g. a rare village name).
• Two neutral descriptions (fr, en), two to three sentences each, for the meaning search.

Keyword match rule: (≥ 1 place term and ≥ 1 event term) or ≥ 1 standalone term; case- and diacritic-insensitive, word-boundary aware.

The descriptions are encoded with the same BGE-M3 weights, pooling and normalisation as doc_embedding (the repo's bge_m3_encoder; model revision recorded). Text and vectors are frozen together in search_inputs.yaml + search_vectors.parquet, fingerprinted.

4. Search and the incident material (stage 3a)

4.1 Window and corpus

• Window: day −14 .. day +45 around day 0.
• Corpus: every Telegram post in the window (text, plus transcript text from message_transcript where one exists) and every web article in the window. Web articles enter by keyword; by meaning only if they carry embeddings (the plan verifies which do).

4.2 Candidates and inclusion

• Keyword route: every post matching §3.
• Meaning route: per description, the posts in the window ranked by cosine similarity, top 2,000; a post's similarity = max over the two descriptions.
• Included = keyword match or similarity ≥ τ, where τ is the single threshold shared by all incidents, fixed in §5.3. Until τ is fixed, the material file carries every candidate with its route and similarity; nothing downstream reads it.
• Copies: a Telegram post that forwards an included post (fwd_from_channel, fwd_from_msg), forwards the same original as an included post, or has a NEAR_DUP edge to an included post, within the window, is included as a copy, marked with the post it copies. Copies are not separately validated and are reported separately from originals.

4.3 Coverage report

Per incident: posts in the window, share with an embedding (the meaning route cannot see the rest), keyword hits, meaning candidates, overlap of the two routes, and whether the top-2,000 cap binds (the 2,000th similarity ≥ τ once τ is fixed); a binding cap is flagged and the pool is widened for that incident before stage 4.

Output: material.parquet (one row per candidate or copy: doc key, incident, route, similarity, keyword terms hit, copy_of, channel, stratum, language, ts) + _fingerprint.json + run manifest.

5. Validation by a coder (stage 3b)

5.1 Sample (per incident, ~100 posts)

• 30 keyword-only, 20 both routes, 50 meaning-only spread evenly over five similarity bands of the top-2,000 pool (10 per band).
• Close pairs: additionally 30 posts that are candidates for both incidents of the pair.
• Total ≈ 16 × 100 + 2 × 30 ≈ 1,660 judgements (≈ 16–32 coder hours).

5.2 The coder sheet

Amendment 2026-10-15: the judgements are now given by the two AI judges plus human adjudication (§5.6); the coder text below is kept as the original design, and the same sheet layout and answer options apply.

Each row: the post text (original language, plus machine translation where available) and a neutral description of the incident. Answer: yes / no / unsure ("is this post about this incident?"). The coder does not see the route, the similarity or other incidents' rows. Close-pair rows show both incident descriptions and ask: this one / the other / both / neither.

5.3 What the returns fix

• unsure counts as not relevant for accuracy (conservative) and is reported.
• τ = the lowest similarity at which the pooled (all 16 incidents) share of yes among meaning-only posts at or above it is ≥ 0.80. Frozen in threshold.json.
• Per incident, reported with intervals: accuracy (estimated share of included posts that are about the incident, weighting the strata by their sizes) and coverage (estimated share of the relevant posts in the candidate pool that the inclusion rule keeps). Coverage relative to posts neither route found cannot be estimated (§13).
• An incident with estimated accuracy < 0.60 is flagged unreliable: shown, but excluded from pattern counts (§8.2).

5.4 Close pairs

A post that is a candidate for both incidents goes to the incident whose place or standalone terms it matches exclusively. If it matches both or neither, it is excluded from both and counted as ambiguous. The close-pair sample (answered by the AI judges plus adjudication, §5.6; originally the coder) measures how often this rule assigns correctly; the rate is reported for each pair.

5.5 Noise floor

For violent incidents (unforeseeable), included material in days −14..−1 measures what the search lets in by accident: noise floor = mean included posts per day over days −14..−1. For coups, pre-window material may be genuine early chatter; it is reported as such and not used as noise. A coup's floor is borrowed: for each violent incident take floor ÷ (all corpus posts per day in its days −14..−1); the coup's floor = the median of those twelve ratios × all corpus posts per day in the coup's days −14..−1.

5.6 AI judges (amendment 2026-10-15)

The coder is replaced by two AI agents, Codex and Gemini, which the user runs on the Mac, each in its own self-contained folder (each agent is instructed to read only its own input/ and write only its own output/, with no network, and is sandboxed where the tool allows (Codex's workspace-write sandbox limits writes and network; reads are restricted by instruction only)). The blind sample of §5.1 is unchanged, and so is what the judges see: post text and neutral incident description(s), never the route, the similarity, the stratum or the incident id.

• Both agents judge every sampled row independently (posts: yes / no / unsure; close pairs: A / B / both / neither, A/B shuffled per row as before) and give a one-sentence reason. Both translate every Russian or Arabic post into English.
• Agreement decides: where the two give the same answer, that is the answer (both unsure → unsure, which counts as not relevant as before).
• Disagreement goes to the user, blind: rows where they differ (including one unsure against a decision) are put in disagreements.xlsx with the post and both translations in random order per row — not the agents' answers, reasons or names. The user's answer is final.
• Agreement is reported: Cohen's kappa and raw agreement between the agents, for posts and pairs separately, overall and per post language (validation/agreement.json, copied into scores.json as _agent_agreement).
• Every answer records its source (agents_agree or human, validation/answer_sources.json), and each incident reports the share of its sampled posts the user decided (human_adjudicated_share).
• τ, accuracy, coverage, unreliable flags and pair agreement are computed from these final answers exactly as in §5.3–5.4. The coder workbook is still written by sample and score --answers coder still reads it, but it is not the study's path.

6. Re-fetch of view and forward counts (stage 4)

• For every included Telegram post (originals and copies), fetch the current views and forwards from Telegram, in batches of 100 per channel at a gentle rate, with the existing Telethon client code.
• Also re-fetch a baseline sample per carrying channel: up to 50 posts drawn evenly from the 90 days before the window, for the views multiple (§7.1). Stored counts are unusable as a baseline for the same reason as for the material.
• A post that no longer exists is recorded as deleted (with the fetch time), not dropped.
• Output: counts.parquet (doc key, views, forwards, deleted, fetched_at) + fingerprint. No prod table is written.
• Every post is then measured at one moment, and every post is ≥ 11 months old (the latest window, Soum 2025, ends 2025-11-23), so counts have settled.

Risks the plan must settle before building this stage:

1. Session safety. Telethon sessions sit on single-node RWO volumes, one per account; using one session from two places can get it logged out. The re-fetch runs either during a short pause of that account's ingestor or with a spare account. The plan determines which is possible.
2. Forward view counts. Whether a forwarded post's views is its own or its origin's. If the origin's, reach counts originals only and forwards contribute spread, not views.

7. Measures (stage 5)

Day 0 = the incident's first possible day. Days are UTC calendar days. All measures use the included material (originals + copies) in days 0..+45 unless stated.

7.1 Reach (Telegram only)

• Total views (re-fetched) and total forwards (Telegram's count, which includes forwards outside our corpus).
• Views multiple per post = its views ÷ the median views of its channel's baseline sample. Per incident: median multiple and share of posts with multiple > 1.
• Reach split by stratum (attributed / discovered / institutional) and by language.

7.2 Spread

• Carrier share: channels with ≥ 1 included post ÷ channels with ≥ 1 post of any kind in the window. Web: sites carrying ÷ sites active.
• Languages: count, and each language's share of posts and of views.
• Strata: each stratum's share of carriers and of views.
• Forward chains (post-level fwd_from_*, inside our corpus): number of forwards, longest chain (hops), forwards crossing a language boundary, forwards crossing a stratum boundary (by direction, e.g. attributed → discovered).
• Reused text: NEAR_DUP copies and the distinct channels they sit in.
• Variants: number of distinct narrative variants (doc_variant) the material falls into. Variant membership only; no labels.

7.3 Speed

• Days to the first post: overall, per stratum and per language.
• Peak day (most included posts).
• Half-life: first day by which ≥ 50 % of the day 0..+45 material has appeared.
• Tail: last day ≤ +45 on which the 3-day rolling mean of included posts exceeds max(2 × noise floor, 1 post/day).

7.4 Fairness across eras

Every count is shown raw and relative to the ecosystem active in that window (carrier shares, posts as a share of all window posts, views multiples). Comparisons between incidents use the relative form.

8. Comparison and the reading rule

8.1 Feature catalogue (fixed now; questions, not expected answers)

Each feature is computed mechanically per incident:

8.2 What counts as a pattern

• Repeats within a type: the same value of a feature in ≥ 3 of the type's 4 incidents.
• General: the same value in ≥ 12 of 16.
• Anything else, and anything not in the catalogue, is reported as a case observation, marked as found after looking.
• Incidents flagged unreliable (§5.3) are shown in every table but do not count toward pattern thresholds; the denominators shrink accordingly and the report says so.
• No significance tests: with four incidents per type the report gives counts ("3 of 4") and ranges.

8.3 Severity view

For the twelve violent incidents: deaths against total views, carrier share and half-life, shown as a scatter with Spearman's rho and its bootstrap interval, labelled descriptive (n = 12).

9. Outputs

• Report reports/<run_id>/: HTML + PDF (headless Chrome on the Mac, as in the news ↔ conflict study) + PNG charts + CSVs.
  • One page per incident: daily volume curve split by stratum and by language, the reach / spread / speed table, the forward-chain picture, search quality (accuracy, coverage, noise floor, ambiguous count), the incident's caveats.
  • Comparison: all curves aligned on day 0 grouped by type (relative volume); the 16-row measures table; the feature table with pattern counts per §8.2; the severity view; the cannot-claim list.
• Mac copy for the user before the PI: FOR-PI-INCIDENTS-<date>/.

13. Cannot-claim list (verbatim in the report and the run manifest)

1. No cause and effect: the study describes how material about an incident travelled, not what it caused or what caused it.
2. The material comes from the 78 Telegram channels and the web sites we collect. It is not "Sahel media" or "the Russian information space" as a whole.
3. Views are not persuasion, and Telegram view counts can be inflated; reach means "displayed", not "believed".
4. Coverage is estimated only against the candidates the two search routes produced; material neither route surfaced is unmeasured.
5. Incident types follow UCDP's coding (e.g. Moura is an army–JNIM battle in UCDP), not how the media framed the events.
6. Patterns describe these sixteen rule-chosen incidents; they are not estimates for incidents in general.
7. Russian involvement before 2024 is identified through UCDP's source headlines, so incidents whose sources did not name Russian forces are missed.
8. Speed is measured in days from the first possible day of the incident, not in hours; incidents with a date range carry that uncertainty.
9. Relevance was judged by two AI agents (Codex and Gemini) with human adjudication of their disagreements; the search being validated also relies on AI embeddings, so the check is not fully independent. Agreement between the agents is reported.
10. Views multiples compare incident posts with a baseline sample from the 90 days before the window; because the baseline is older, channel growth alone can push a multiple above 1, so a multiple above 1 is not evidence of extra attention.

14. Out of scope

• Framing: blame, alignment, function. Second pass on the frozen material once this study shows where labeling is worth spending.
• New channel or site ingestion; ACLED; dashboard tabs.
• Any write to prod tables (the re-fetch writes only to S3).
• Causal or forecasting models.