PrompTugal.newsStorm CharlesUpdated 12:10 UTCNew here?v0.1.1043Repo
Not a warning service. Follow IPMA, ANEPC and your local authorities. We aggregate public sources, each with its source and the time we read it.

The chart is drawn in the browser from results.json; the same figures are in the table below.

Not a warning service: for warnings follow IPMA and ANEPC. Nothing here is a forecast or advice. Part of Other models, measured.

Extract a warning from one of IPMA's CAP messages (Common Alerting Protocol, read through MeteoAlarm's public feed). The input is the message's fields as the meteorologist keeps them in cap.json (Portuguese event and description, times with offsets); the gold answer is the same record, normalised: type and level in English, times in UTC with Z. 'sent' is in the input and is not the start.

Scoring: each field exact; an item is right when all five are. Last run 2026-11-11 02:56 UTC. Source for every figure on this page: results.json (made 2026-11-11 03:06 UTC from the runs) and ledger-summary.json.

The instruction

The same for every model (the item follows it as the user's message):

You extract one weather warning from a CAP message issued by IPMA (Portugal). Answer with one JSON object and nothing else: no explanation, no code fence.
Keys: "type" (one of: rain, wind, coastal_event, thunderstorm, fog, snow, cold, heat), "level" (yellow, orange or red), "area" (the areaDesc exactly as written), "start_utc" and "end_utc" (when the warning is in force, UTC, written YYYY-MM-DDTHH:MM:SSZ).

The table

Model, reasoningKindRuns rightItems right, run by runChanged between runstypelevelareastart_utcend_utcMalformedCut offRefusedErrorsCost per item, USDMedian latency, sp90, s
google/gemma-3-4b-it: does not reasonopen-weight, small17/1717 of 17 (1 run)171717171700000.0000163.23.6
openai/gpt-oss-20b: reasons; cannot be turned off (the provider refuses the setting)open-weight, small17/1717 of 17 (1 run)171717171700000.0000682.97.6
mistralai/ministral-8b-2512: does not reasonopen-weight, small17/1717 of 17 (1 run)171717171700000.0000450.91.1
qwen/qwen3.5-9b#reasoning-off: off (asked)open-weight, small17/1717 of 17 (1 run)171717171700000.0000411.32.2
qwen/qwen3.5-9b: on (the provider's default)open-weight, small17/1717 of 17 (1 run)171717171700000.0003018.327.5
google/gemma-4-31b-it: off (the provider's default)open-weight, large17/1717 of 17 (1 run)171717171700000.0000641.02.1
meta-llama/llama-4-maverick: does not reasonopen-weight, large17/1717 of 17 (1 run)171717171700000.000131.01.3
deepseek/deepseek-v4.1-flash: on (the provider's default)open-weight, large17/1717 of 17 (1 run)171717171700000.000121.12.1
qwen/qwen3.5-397b-a17b#reasoning-off: off (asked)open-weight, large17/1717 of 17 (1 run)171717171700000.000272.22.5
qwen/qwen3.5-397b-a17b: on (the provider's default)open-weight, large17/1717 of 17 (1 run)171717171700000.0042516.921.2
mistralai/mistral-large-4-0: reasons; cannot be turned off (the provider refuses the setting)open-weight, large17/1717 of 17 (1 run)171717171700000.002154.811.3
openai/gpt-6-luna: off (the provider's default)commercial, small17/1717 of 17 (1 run)171717171700000.0000642.62.8
google/gemini-3.5-flash-lite: reasons; cannot be turned off (the provider refuses the setting)commercial, small17/1717 of 17 (1 run)171717171700000.000270.80.9
anthropic/claude-haiku-5.5: off (the provider's default)commercial, small17/1717 of 17 (1 run)171717171700000.0000770.81.8
openai/gpt-6.1-terra: off (the provider's default)commercial, large17/1717 of 17 (1 run)171717171700000.000521.51.8

The field columns count the first run. "Changed between runs": items a row got right in one run and wrong in another, at temperature 0.

Reading (ours, a hypothesis): Every model got every item. As built, the task is a copy with a small translation (Vermelho to red, +00:00 to Z) and it separates nothing. It does say the cheapest open-weight model here can turn a CAP message into our fields for about 0.00002 US dollars an item. A harder second version is the next step: the warning as a sentence in IPMA's communiqué, several warnings in one text, and local time in summer.

Item by item

First run of each row.

ItemRightgemma-3-4b-itgpt-oss-20bministral-8b-2512qwen3.5-9b #reasoning-offqwen3.5-9bgemma-4-31b-itllama-4-maverickdeepseek-v4.1-flashqwen3.5-397b-a17b #reasoning-offqwen3.5-397b-a17bmistral-large-4-0gpt-6-lunagemini-3.5-flash-liteclaude-haiku-5.5gpt-6.1-terra
w0115/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0215/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0315/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0415/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0515/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0615/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0715/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0815/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w0915/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1015/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1115/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1215/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1315/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1415/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1515/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1615/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright
w1715/15rightrightrightrightrightrightrightrightrightrightrightrightrightrightright

w01

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6288
sent: 2026-11-10T21:23:00+00:00
msgType: Update
event: Aviso Laranja Precipitação
severity: Severe
onset: 2026-11-11T00:00:00+00:00
expires: 2026-11-11T09:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e persistente.

Scored against: {"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261110212300.PO-10-6288. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"orange","area":"Lisboa","start_utc":"2026-11-11T00:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1

w02

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6290
sent: 2026-11-10T21:23:00+00:00
msgType: Update
event: Aviso Vermelho Precipitação
severity: Extreme
onset: 2026-11-11T09:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Lisboa
description: Precipitação forte e persistente.

Scored against: {"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261110212300.PO-10-6290. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Lisboa","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1

w03

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6297
sent: 2026-11-10T21:23:00+00:00
msgType: Alert
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-10T21:24:00+00:00
expires: 2026-11-11T00:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e persistente.

Scored against: {"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261110212300.PO-10-6297. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"yellow","area":"Lisboa","start_utc":"2026-11-10T21:24:00Z","end_utc":"2026-11-11T00:00:00Z"}right1/1

w04

identifier: 2.49.0.0.620.0.PT.261110120908.PO-07-4525
sent: 2026-11-10T12:09:08+00:00
msgType: Update
event: Aviso Laranja Agitação Marítima
severity: Severe
onset: 2026-11-11T12:00:00+00:00
expires: 2026-11-11T21:00:00+00:00
areaDesc: Lisboa
description: Ondas de sudoeste com 5 a 7 metros e que poderão atingir até 13 metros de altura máxima.

Scored against: {"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261110120908.PO-07-4525. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gpt-oss-20b{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
ministral-8b-2512{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-9b{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gemma-4-31b-it{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
llama-4-maverick{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
mistral-large-4-0{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gpt-6-luna{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
claude-haiku-5.5{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gpt-6.1-terra{"type":"coastal_event","level":"orange","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1

w05

identifier: 2.49.0.0.620.0.PT.261110071840.PO-01-1915
sent: 2026-11-10T07:18:40+00:00
msgType: Update
event: Aviso Vermelho Vento
severity: Extreme
onset: 2026-11-11T12:00:00+00:00
expires: 2026-11-11T21:00:00+00:00
areaDesc: Lisboa
description: Rajadas até 120 km/h, podendo atingir 140 km/h nos locais mais expostos.

Scored against: {"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261110071840.PO-01-1915. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gpt-oss-20b{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
ministral-8b-2512{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-9b{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gemma-4-31b-it{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
llama-4-maverick{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
mistral-large-4-0{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gpt-6-luna{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
claude-haiku-5.5{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1
gpt-6.1-terra{"type":"wind","level":"red","area":"Lisboa","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T21:00:00Z"}right1/1

w06

identifier: 2.49.0.0.620.0.PT.261110071840.PO-07-4518
sent: 2026-11-10T07:18:40+00:00
msgType: Update
event: Aviso Amarelo Agitação Marítima
severity: Moderate
onset: 2026-11-11T21:00:00+00:00
expires: 2026-11-12T03:00:00+00:00
areaDesc: Lisboa
description: Ondas de sudoeste com 4 a 5 metros.

Scored against: {"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261110071840.PO-07-4518. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
gpt-oss-20b{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
ministral-8b-2512{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
qwen3.5-9b{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
gemma-4-31b-it{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
llama-4-maverick{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
mistral-large-4-0{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
gpt-6-luna{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
claude-haiku-5.5{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1
gpt-6.1-terra{"type":"coastal_event","level":"yellow","area":"Lisboa","start_utc":"2026-11-11T21:00:00Z","end_utc":"2026-11-12T03:00:00Z"}right1/1

w07

identifier: 2.49.0.0.620.0.PT.261108190331.PO-03-448
sent: 2026-11-08T19:03:31+00:00
msgType: Update
event: Aviso Amarelo Trovoada
severity: Moderate
onset: 2026-11-09T21:00:00+00:00
expires: 2026-11-10T15:00:00+00:00
areaDesc: Lisboa
description: Condições favoráveis para ocorrência de fenómenos meteorológicos localizados, como: trovoada; precipitação localmente forte, com acumulação entre 10 a 20 mm em 1 h; rajadas entre 70 e 90 km/h.

Scored against: {"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261108190331.PO-03-448. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
gpt-oss-20b{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
ministral-8b-2512{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
qwen3.5-9b{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
gemma-4-31b-it{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
llama-4-maverick{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
mistral-large-4-0{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
gpt-6-luna{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
claude-haiku-5.5{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1
gpt-6.1-terra{"type":"thunderstorm","level":"yellow","area":"Lisboa","start_utc":"2026-11-09T21:00:00Z","end_utc":"2026-11-10T15:00:00Z"}right1/1

w08

identifier: 2.49.0.0.620.0.PT.261108011310.PO-01-1895
sent: 2026-11-08T01:13:10+00:00
msgType: Update
event: Aviso Amarelo Vento
severity: Moderate
onset: 2026-11-08T18:00:00+00:00
expires: 2026-11-09T06:00:00+00:00
areaDesc: Lisboa
description: Vento do quadrante sul com rajadas até 75 km/h.

Scored against: {"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json lisbon_messages identifier 2.49.0.0.620.0.PT.261108011310.PO-01-1895. Source of the text: feeds.meteoalarm.org, read 2026-11-10 23:51 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
gpt-oss-20b{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
ministral-8b-2512{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
qwen3.5-9b{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
gemma-4-31b-it{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
llama-4-maverick{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
mistral-large-4-0{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
gpt-6-luna{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
claude-haiku-5.5{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1
gpt-6.1-terra{"type":"wind","level":"yellow","area":"Lisboa","start_utc":"2026-11-08T18:00:00Z","end_utc":"2026-11-09T06:00:00Z"}right1/1

w09

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6291
sent: 2026-11-10T21:23:00+00:00
msgType: Update
event: Aviso Vermelho Precipitação
onset: 2026-11-11T09:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Setúbal

Scored against: {"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110212300.PO-10-6291. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Setúbal","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1

w10

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6294
sent: 2026-11-10T21:23:00+00:00
msgType: Alert
event: Aviso Vermelho Precipitação
onset: 2026-11-11T09:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Beja

Scored against: {"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110212300.PO-10-6294. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Beja","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1

w11

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6295
sent: 2026-11-10T21:23:00+00:00
msgType: Alert
event: Aviso Vermelho Precipitação
onset: 2026-11-11T09:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Faro

Scored against: {"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110212300.PO-10-6295. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Faro","start_utc":"2026-11-11T09:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1

w12

identifier: 2.49.0.0.620.0.PT.261110205239.MD-10-1236
sent: 2026-11-10T20:52:39+00:00
msgType: Update
event: Aviso Laranja Precipitação
onset: 2026-11-10T21:00:00+00:00
expires: 2026-11-11T09:00:00+00:00
areaDesc: Madeira - Porto Santo

Scored against: {"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110205239.MD-10-1236. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"orange","area":"Madeira - Porto Santo","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1

w13

identifier: 2.49.0.0.620.0.PT.261110205239.MD-10-1237
sent: 2026-11-10T20:52:39+00:00
msgType: Update
event: Aviso Vermelho Precipitação
onset: 2026-11-10T21:00:00+00:00
expires: 2026-11-11T06:00:00+00:00
areaDesc: Madeira - Costa Norte

Scored against: {"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110205239.MD-10-1237. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Madeira - Costa Norte","start_utc":"2026-11-10T21:00:00Z","end_utc":"2026-11-11T06:00:00Z"}right1/1

w14

identifier: 2.49.0.0.620.0.PT.261110205239.MD-10-1238
sent: 2026-11-10T20:52:39+00:00
msgType: Alert
event: Aviso Laranja Precipitação
onset: 2026-11-11T06:00:00+00:00
expires: 2026-11-11T09:00:00+00:00
areaDesc: Madeira - Costa Norte

Scored against: {"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110205239.MD-10-1238. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"orange","area":"Madeira - Costa Norte","start_utc":"2026-11-11T06:00:00Z","end_utc":"2026-11-11T09:00:00Z"}right1/1

w15

identifier: 2.49.0.0.620.0.PT.261110163001.PO-10-6271
sent: 2026-11-10T16:30:01+00:00
msgType: Alert
event: Aviso Vermelho Precipitação
onset: 2026-11-11T12:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Castelo Branco

Scored against: {"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110163001.PO-10-6271. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Castelo Branco","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1

w16

identifier: 2.49.0.0.620.0.PT.261110163001.PO-10-6272
sent: 2026-11-10T16:30:01+00:00
msgType: Alert
event: Aviso Vermelho Precipitação
onset: 2026-11-11T12:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Leiria

Scored against: {"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110163001.PO-10-6272. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Leiria","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1

w17

identifier: 2.49.0.0.620.0.PT.261110163001.PO-10-6275
sent: 2026-11-10T16:30:01+00:00
msgType: Alert
event: Aviso Vermelho Precipitação
onset: 2026-11-11T12:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Santarém

Scored against: {"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}, from incidents/2026-11-10-storm-charles/data/weather/cap.json other_messages identifier 2.49.0.0.620.0.PT.261110163001.PO-10-6275. Source of the text: feeds.meteoalarm.org, read 2026-11-11 01:46 UTC.

ModelAnswer (first run)ResultRight in runs
gemma-3-4b-it{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-oss-20b{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
ministral-8b-2512{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b #reasoning-off{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-9b{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemma-4-31b-it{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
llama-4-maverick{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
deepseek-v4.1-flash{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b #reasoning-off{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
qwen3.5-397b-a17b{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
mistral-large-4-0{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6-luna{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gemini-3.5-flash-lite{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
claude-haiku-5.5{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1
gpt-6.1-terra{"type":"rain","level":"red","area":"Santarém","start_utc":"2026-11-11T12:00:00Z","end_utc":"2026-11-11T18:00:00Z"}right1/1