Conflict incident map — design (2026-10-21)

A new tab in the status dashboard (sahel-status, FastAPI + static pages) that shows UCDP conflict incidents on a Mapbox map with GDELT news-report context, lets the viewer pick a date range, shows three lines on hover, and opens one incident in a new tab with everything we hold about it. A local demo with real data (Mac folder GDELT-MAP-DEMO-2026-10-21/) settled the interaction; this document turns it into the cluster version.

1. Intent and why

The project manager wants to see the two conflict sources together. The earlier map in an old application failed because every fact was crammed into hover panels. The rule here: the map answers "where and when", the hover answers "what, briefly", and the incident page answers everything else. The incident page is also the future home for "what the channels said" (posts and news tied to an incident by the incident study), so it must exist as a page with a stable address, not a popup.

2. Goals, non-goals, anti-goals

Goals

1. A tab in the dashboard domain (same basic auth as the rest of the site).
2. Every UCDP incident as a dot (size = best death estimate); GDELT violent reports as region shading; a third, optional layer showing per region whether days were seen by both sources, UCDP only, or GDELT only.
3. A date range (from / to, plus 30 d / 90 d / 1 y / all presets). Dots use exact dates. Shading and coverage are summed over the exact days in the range (the demo could only do whole months; the cluster version has daily tables).
4. Hover on a dot: three lines, nothing else. Click: the incident page opens in a new browser tab.
5. Incident page: what UCDP recorded, the linked GDELT reports, the region's context for that month, and an empty, clearly labelled box reserved for posts and news.
6. Data comes from the live database; no manual export step; nothing to run by hand after deploy except creating one secret.

Non-goals

• No merged event list; the two sources stay apart (project rule since 2026-10-19). No GDELT dots (its 7,800 city-coded reports sit on 279 spots).
• No tuning of the link rules in this project (see §7, known data limits).
• No analysis or claims on the page beyond counts that come straight from the tables.
• No change to who can see the dashboard; no public map.
• No mobile layout work beyond "does not break".

Anti-goals (would make it a failure even if it works)

• A hover card with tables, charts or scrolling.
• Raw GDELT counts painted without the "reports, not incidents; capitals run high" caveat visible on the page.
• Link counts shown as if they were coverage, without the rule A / rule B split.

3. Constraints (from the codebase and the environment)

• The name incidents is already taken on this site: /api/incidents and the Reliability page mean pipeline incidents. The new feature uses the word conflict in every path and identifier (see §4.1).

4. What gets built

4.1 Names and addresses

4.2 Data path (my call: read the database directly, cache in the process)

• Refresher (run_conflict_refresher, default every 30 min, env SAHEL_STATUS_CONFLICT_INTERVAL): opens one read-only connection (same pattern as _make_fetch_pg: connect_timeout=5, default_transaction_read_only=on, statement timeout set), runs the two bulk queries below, builds the two payloads with pure functions, and swaps them into a cache object. First run at startup; until it finishes the API answers 503 "not computed yet" and the page shows a plain "loading" line. A failed refresh keeps the last good payload and records the error on the payload's meta (surfaced on the page as "data as of …, refresh failed").
• /api/conflict/incidents — every row of ucdp_event with the columns the map and the hover need, plus the per-incident link summary computed in SQL (count(*), min(date_added) over conflict_event_link joined to gdelt_event): id, date_start, date_end, date_prec, type_of_violence, side_a, side_b, best, low, high, lat, lon, where_prec, adm_1, gid_1, country, link_count, first_report_at. Sorted by date. About 5.4 k rows today; roughly 1 MB of JSON; fine as one object, served with Cache-Control: max-age=300.
• /api/conflict/region-days — per admin1 × day: GDELT violent reports (g_tensor, event_root_code in (18,19,20), summed event_count and mentions) and the coverage row (incident_coverage: coverage, ucdp_published, ucdp_event_count, gdelt_event_count). One row per region-day that has either. admin1 = 'unlocated' rows are included (the status line reports them as "n reports could not be placed in a region"; they never paint a polygon). Rows are filtered to the current pipeline_version (Settings().pipeline_version, as the completeness collector does) and, for links, to the newest link_version by built_at; both versions are reported in meta so a later link re-run cannot double the counts. About 11 k rows for the first year; grows about 11 k rows a year. Compact encoding (arrays, not objects) if the builder finds the JSON over 2 MB. The client sums these over the chosen range, so the server never sees the date range.
• /api/conflict/incident/{id} — on demand, not cached across requests beyond a small LRU (last 50 ids, 60 s): the full ucdp_event row, and its linked GDELT reports (conflict_event_link ⋈ gdelt_event: date_added, sql_date, stream, event_code, event_root_code, num_mentions, num_sources, actor1_name, actor2_name, action_geo_fullname, source_domain, source_url, rule, day_offset), ordered by date_added, capped at 2,000 rows with the true total in meta. Plus the region-month context: UCDP incidents in the same gid_1 and month, GDELT violent reports that month, and the month's both / UCDP-only / GDELT-only day counts. id must be digits only (400 otherwise); unknown id → 404.

4.3 Map page behaviour (as validated in the demo)

• Layers: UCDP incidents (dots, on by default), GDELT violent reports (region shading, on by default), coverage (off by default; when on it replaces the shading and the legend switches). "Hide country-level incidents" removes dots whose where_prec ≥ 5.
• Dots: radius from sqrt(best); incidents with where_prec ≥ 4 are drawn hollow (they sit on a region or country centroid, not a real spot).
• Date range: two date inputs bounded by the data (2021-01-01 to gdelt_max_day), presets 30 d / 90 d / 1 y / all counting back from ucdp_max_date; a line under the inputs states the range, its length in days, and the two data edges ("no GDELT shading before 19 Oct 2025", "no UCDP after 31 Aug 2026 yet").
• Shading: one blue hue, light → dark, breaks from quantiles of the non-zero region sums in the chosen range; zero = neutral. Coverage layer: three validated colours (both #0e7fb0, UCDP-only #c9731f, GDELT-only #6a4c93; dots #b5321f; palette passed the dataviz validator in light mode on 2026-10-21). The region's class is the majority of its comparable day classes in the range. A day with ucdp_published = false (after UCDP's coverage end, currently 2026-08-31) is not comparable: it is left out of the vote and counted separately as "days not yet comparable (UCDP not published)", shown as a fourth, grey line in the legend and in the region hover. A region whose range has no comparable day is painted neutral. Reason: otherwise any range reaching past the UCDP end paints every region "GDELT only" because UCDP has not published, not because UCDP missed anything (the table comment on incident_coverage says the same).
• Hover on a dot: place, country · date / side A vs side B · N deaths (UCDP) / GDELT: n reports, first +h h · open. Hover on a region, only while the coverage layer is on: region name and the three day counts. The region card never shows while the pointer is over a dot.
• Click on a dot: window.open('/conflict-incident.html?id=…', '_blank').
• Status line: incidents in range, deaths, how many have a linked report.
• Legend always carries: "GDELT counts news reports, not incidents, and reports cluster in capitals (Bamako, Niamey, Ouagadougou)."

4.4 Incident page behaviour

• Header: place, date, UCDP id, violence type, sides, deaths with range, placement precision in words.
• "What UCDP recorded": where (description, admin-2, admin-1, GADM code, precision in words, coordinates), when (with date precision in words), who, deaths split, recorded in which UCDP version (Candidate = may be revised), sources, clarity.
• "What GDELT reported about it": three figures (first report after the day began, total mentions, rule A / rule B counts), then the reports grouped by outlet (outlet, number of reports, first and last report time, event types seen), sorted by first report; each group expands to its rows (time, event type, where GDELT placed it, day offset, rule, link to the article). The 2,000-row cap and the true total are stated when they differ. The sentence "links are rule-based matches, plausible, not verified" stays on the page.
• When there are no links the page says why, in words: before the GDELT window; country-level incident (never linked); month- or year-dated incident (never linked); or simply no matching report.
• "Context around this day": the three region-month figures.
• The reserved box: "Later: what the channels said — Telegram posts and news articles that refer to this incident, with first-mention time and spread, from the incident study. Empty for now."

4.5 Dashboard integration

• app.py: four new routes behind injected callables (conflict_*_fn), so the route tests run with fakes like every other route.
• Nav: the new tab added to every page's header (seven existing pages plus the two new ones).
• No change to the stage catalog, stats catalog, health sampler, or alerts.

5. Decisions the user made (verbatim intent)

• Mapbox, not MapLibre.
• The map lives as a tab in our domain (the status dashboard).
• Dots show a super brief summary; clicking opens the incident in a new tab; that tab will later also show the news and posts related to the incident.
• A date range from X to Y, aggregating incidents between the two dates.
• Build on the cluster and deploy (2026-10-21).

6. Calls I made (check these while reading)

1. Read the database directly with a 30-minute in-process refresher instead of a nightly export Job. Reason: the dashboard already has the read-only DSN and the pattern (graph3d); an export would add a Job, an S3 prefix and a staleness tile for no gain at this data size.
2. Daily aggregation for shading and coverage (the demo used months).
3. Group linked reports by outlet on the incident page, cap the rows at 2,000, show the true total.
4. The client, not the server, sums the region-days over the chosen range.
5. Paths and identifiers use conflict, because incidents is taken on this site.
6. Unpublished UCDP days are a fourth, grey "not yet comparable" class in the coverage layer, excluded from the majority vote (reviewer's finding; my call, since the anti-goals in §2 leave no other honest option).
7. Payloads are filtered to the current pipeline version and the newest link version, both reported in meta.
8. unlocated GDELT rows are carried in the payload and shown as a count, never dropped silently.

7. Known data limits the page must state, not hide

• GDELT dates events by report, not by occurrence, and 61 % of its action rows are country-level only; 39 % reach a region.
• Capital bias: Bamako region carries 2,377 GDELT violent reports against 5 UCDP incidents in the overlap window; Niamey 991 vs 3; Ouagadougou's Centre region 295 vs 0.
• The link rules over-attach in capitals (a 0-death Bamako event on 2026-04-27 has 703 links) and miss some large incidents (a 67-death Tillabéri attack on 2026-05-14 has none). Link tuning is a separate, later piece of work; the page shows rule A / rule B counts so the reader can judge.
• UCDP ends 2026-08-31 (monthly sync on the 6th); GDELT runs daily from 2025-10-19. Links exist only where both overlap.

8. Left to the builder

Exact SQL and indexes used (read-only; no migration expected — if a query needs an index, that is a separate PR), payload encoding, cache object shape, JS file structure, quantile break computation, how the outlet groups expand, and test fixtures.

9. Questions asked and answered (2026-10-21)

• Tab label → "Conflicts on Map" (user).
• Mapbox usage telemetry → accepted, no blocking (user).
• Unpublished UCDP days in the coverage layer → fourth "not yet comparable" class (my call, §6.8).