The storm's data (WS-214)

JSON first: the incident dashboard (the incident page), Vicky's agents and the work desk read these files, not pages. Every item carries one label: fact, evidence or hypothesis (the meteorologist's analysis is a hypothesis). Times are UTC (Lisbon's own time in November).

The dashboard reads signals.json, decisions/graph.json, watch/websummit.json and weather/{warnings,stations,forecast,airport,analysis}.json. A field it uses, renamed, breaks a pane (it says which file it could not read); tell incident.dev@ in a commit (For: incident.dev@web-summit.dinis) and it follows.

signals.json

node tools/timeline.mjs incidents/2026-11-10-storm-charles

{ about, source, made_at, items: [ { at, approx, kind, label, key, text, sources: [{ label, url|null }] } ] }, oldest first.

• Reads only the table rows whose first cell looks like 10 Nov, 17:42 (or ~17:42); the paragraphs under the table are not read.
• kind: signal when the text starts "signal" (signal:, signal (unofficial):, kept as "(unofficial) …") or the source is relayed "via social.watch@"; official when the source names IPMA, portugal.gov.pt, websummit.com, ANEPC, S41 or the weather desk; team otherwise (our own record).
• label: evidence for a signal, fact otherwise. key: the row was in bold. approx: the time had a ~. Only https links are kept as URLs.

Connectors (incident.dev@web-summit.dinis)

A connector reads just the part of a source that matters, and is listed here.

watch/websummit.json

node tools/websummit.mjs incidents/2026-11-10-storm-charles

{ about, read_at, watch, sitemap: { source, count, added_this_read: [url], urls: { url: first_seen } }, blog: { source, listings: [url], posts: { url: { url, title, category, date, modified, excerpt, first_seen } } }, bluesky: { source, posts: [{ at, text, url, match }] }, alerts: [{ first_seen, channel, url, text, at?, label }], errors: [] }

• Why the blog (J6): the Day 2 decision went up at 19:09 on /blog/web-summit/important-event-update-2026/, which no sitemap lists; we learned of it 43 minutes later, from the press. Every blog listing is now read, and a category named only by a post (e.g. partners) is read too. A blog alert is new (a post not seen before), modified (its modified changed) or, on the first read only, recent (published or modified in the last 36 hours). The WordPress REST API answers 404 on this site (a Next.js front end), so the listings' embedded data is the one read.
• Every 2 minutes (the researcher's ask, 21:03): tools/websummit-poll.sh <repo checkout> <scratch dir> runs a copy of websummit.mjs pinned when the poll starts (its sha256 printed) with --quick (/blog/ only), against a scratch copy of the watch file. The loop runs no git command. On an alert it stops; incident.dev@'s session is woken, reads the alert as data and commits the watch file itself, with a fixed subject and the site's words only in the body, quoted. The poll lives as long as that session (it stops within 2 hours and restarts each wake). Redesigned at 21:30 after security.repos@'s MEDIUM (report 2026-11-10T2130Z): the first version reset to dev every loop and ran what it found there, and put site text in commit subjects. The full read (every listing, the sitemaps, Bluesky) runs on the agents' wakes.
• robots.txt is obeyed: it is read on every run; a disallowed path (today only /api/) is never fetched (robots_disallow in the file).
• Two dates per post, each named: modified is the listing's Post.modified (modified_field, read at read_at); page holds the post page's own JSON-LD date_published and date_modified with their read_at, for posts of the last 48 hours. They can differ: "Important event update" reads modified 19:58 in the listing and 19:25 in its page's JSON-LD (21:11).
• Listings disagree, and the newest wins: at 21:33 /blog/ said a post was modified at 20:06 while /blog/web-summit/ still said 19:58 (a cached copy). modified is the newest value any listing gives, modified_by_listing keeps each listing's value, and an older value never moves it back (no false "modified" alert).
• The post's own words: for posts of the last 48 hours, page.text (its __NEXT_DATA__ post.content, tags stripped, at most 4,000 characters) and page.text_sha256. A text_changed alert fires when the hash changes, whatever the dates say. (At 21:32 modified moved and we could not tell what had changed; from 21:40 we can.)
• Sitemap URLs: every URL ever seen is kept; a URL alerts once, the first time it appears, whatever its words (the set changes between reads).
• An alert is a new blog post or a changed one (above), a new sitemap URL or an own post whose text matches watch (since 20:04 also Thursday/quinta, day 2 and 3, PITCH, tickets, refunds, talks, speakers, streams, side events: the open items after the Wednesday decision). It is a fact that the organiser published it; a match is not a decision: read it.
• The first read records the URL set and the newest post and raises no alert; after that only a new URL, or a post newer than the last read's newest, can raise one. alerts keeps every alert ever raised, with first_seen.
• The dashboard also reads Bluesky live in the browser every minute (its public API allows any origin), so a post shows on the page without waiting for this file, CI or Pages' cache. The sitemap has no such header, so it needs this connector.

The weather connectors are the meteorologist's own (weather/tools/refresh.py). The researcher's parser asks (decisions/sources.json, parser_ask) are next.