Warnings in force at an hour

Not a warning service: for warnings follow IPMA and ANEPC. Nothing here is a forecast or advice. Part of Other models, measured.

Round 2, harder: all seventeen CAP messages this site holds for the Lisbon district (IPMA, through MeteoAlarm; 5 to 10 November, oldest first, most of them expired), and one question: which warnings are in force at a given hour of 11 or 12 November? The gold is not computed from the messages: it is IPMA's own warnings API as the meteorologist read it (warnings.json), filtered to the hour. A model has to compare times across seventeen messages, not copy a field. The start is not scored (IPMA's API and its CAP message give different starts for a warning already running); type, level and end are.

Scoring: the set of warnings (type, level, end) equals the gold set exactly: none missing, none extra. Last run 2026-11-11 02:56 UTC. Source for every figure on this page: results.json (made 2026-11-11 03:06 UTC from the runs) and ledger-summary.json.

The instruction

The same for every model (the item follows it as the user's message):

You are given every CAP message a newsroom holds for one district of Portugal (IPMA's weather warnings), oldest first; most have expired. Say which warnings are in force at one given hour. Answer with one JSON object and nothing else: no explanation, no code fence.
Key: "in_force", a list (empty if none) of objects with "type" (one of: rain, wind, coastal_event, thunderstorm, fog, snow, cold, heat), "level" (yellow, orange or red) and "end_utc" (when that warning expires, UTC, written YYYY-MM-DDTHH:MM:SSZ).
A warning is in force at an hour when its onset is at or before that hour and its expires is after it.

The table

The field columns count the first run. "Changed between runs": items a row got right in one run and wrong in another, at temperature 0.

Reading (ours, a hypothesis): Partial: the account ran out of credits while this task was running, so only three rows have all 13 items three times, and several have a handful. What there is: this task separates models that the copy-a-field task could not. google/gemma-4-31b-it got the same 11 of 13 in each of three runs; meta-llama/llama-4-maverick, its equal on every round-1 task, got 1 or 2 of 13; the small models got none. Where a row was asked with reasoning on, it did better than the same model with it off (DeepSeek V4.1 Flash 11 of 11 on, 5 of the 8 items it reached off; Gemma 4 10 of 10 on) at three to four times the price per item and 7 to 23 seconds instead of 1 to 2. These are counts from an unfinished run: enough to say the task is worth finishing, not to rank.

Item by item

First run of each row.

The seventeen messages

The same for every item of this task; only the hour asked changes.

identifier: 2.49.0.0.620.0.PT.261105125809.PO-10-6141
sent: 2026-11-05T12:58:09+00:00
msgType: Alert
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-06T15:00:00+00:00
expires: 2026-11-07T00:00:00+00:00
areaDesc: Lisboa
description: Períodos de chuva ou aguaceiros, por vezes fortes e acompanhados de trovoada.

identifier: 2.49.0.0.620.0.PT.261106181838.PO-10-6144
sent: 2026-11-06T18:18:38+00:00
msgType: Update
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-06T15:00:00+00:00
expires: 2026-11-06T18:19:00+00:00
areaDesc: Lisboa
description: Períodos de chuva ou aguaceiros, por vezes fortes e acompanhados de trovoada.

identifier: 2.49.0.0.620.0.PT.261107073544.PO-10-6148
sent: 2026-11-07T07:35:44+00:00
msgType: Alert
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-08T18:00:00+00:00
expires: 2026-11-09T06:00:00+00:00
areaDesc: Lisboa
description: Períodos de chuva, por vezes fortes e acompanhados de trovoada.

identifier: 2.49.0.0.620.0.PT.261108011310.PO-01-1895
sent: 2026-11-08T01:13:10+00:00
msgType: Update
event: Aviso Amarelo Vento
severity: Moderate
onset: 2026-11-08T18:00:00+00:00
expires: 2026-11-09T06:00:00+00:00
areaDesc: Lisboa
description: Vento do quadrante sul com rajadas até 75 km/h.

identifier: 2.49.0.0.620.0.PT.261108124742.PO-03-420
sent: 2026-11-08T12:47:42+00:00
msgType: Alert
event: Aviso Amarelo Trovoada
severity: Moderate
onset: 2026-11-08T18:00:00+00:00
expires: 2026-11-09T06:00:00+00:00
areaDesc: Lisboa
description: Condições favoráveis para ocorrência de fenómenos meteorológicos localizados, como: trovoada; precipitação localmente forte, com acumulação entre 10 a 20 mm em 1 h; rajadas entre 70 e 90 km/h.

identifier: 2.49.0.0.620.0.PT.261108124742.PO-07-4500
sent: 2026-11-08T12:47:42+00:00
msgType: Update
event: Aviso Amarelo Agitação Marítima
severity: Moderate
onset: 2026-11-09T18:00:00+00:00
expires: 2026-11-11T06:00:00+00:00
areaDesc: Lisboa
description: Ondas do quadrante oeste com 4 a 5 metros.

identifier: 2.49.0.0.620.0.PT.261108190331.PO-03-448
sent: 2026-11-08T19:03:31+00:00
msgType: Update
event: Aviso Amarelo Trovoada
severity: Moderate
onset: 2026-11-09T21:00:00+00:00
expires: 2026-11-10T15:00:00+00:00
areaDesc: Lisboa
description: Condições favoráveis para ocorrência de fenómenos meteorológicos localizados, como: trovoada; precipitação localmente forte, com acumulação entre 10 a 20 mm em 1 h; rajadas entre 70 e 90 km/h.

identifier: 2.49.0.0.620.0.PT.261108222309.PO-10-6216
sent: 2026-11-08T22:23:09+00:00
msgType: Update
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-09T21:00:00+00:00
expires: 2026-11-10T18:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e acompanhada de trovoada.

identifier: 2.49.0.0.620.0.PT.261109215857.PO-10-6246
sent: 2026-11-09T21:58:57+00:00
msgType: Update
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-10T09:00:00+00:00
expires: 2026-11-10T18:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e acompanhada de trovoada.

identifier: 2.49.0.0.620.0.PT.261109215857.PO-10-6248
sent: 2026-11-09T21:58:57+00:00
msgType: Alert
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-09T21:59:00+00:00
expires: 2026-11-10T00:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e acompanhada de trovoada.

identifier: 2.49.0.0.620.0.PT.261109215857.PO-10-6250
sent: 2026-11-09T21:58:57+00:00
msgType: Alert
event: Aviso Laranja Precipitação
severity: Severe
onset: 2026-11-10T00:00:00+00:00
expires: 2026-11-10T09:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e acompanhada de trovoada.

identifier: 2.49.0.0.620.0.PT.261110071840.PO-01-1915
sent: 2026-11-10T07:18:40+00:00
msgType: Update
event: Aviso Vermelho Vento
severity: Extreme
onset: 2026-11-11T12:00:00+00:00
expires: 2026-11-11T21:00:00+00:00
areaDesc: Lisboa
description: Rajadas até 120 km/h, podendo atingir 140 km/h nos locais mais expostos.

identifier: 2.49.0.0.620.0.PT.261110071840.PO-07-4518
sent: 2026-11-10T07:18:40+00:00
msgType: Update
event: Aviso Amarelo Agitação Marítima
severity: Moderate
onset: 2026-11-11T21:00:00+00:00
expires: 2026-11-12T03:00:00+00:00
areaDesc: Lisboa
description: Ondas de sudoeste com 4 a 5 metros.

identifier: 2.49.0.0.620.0.PT.261110120908.PO-07-4525
sent: 2026-11-10T12:09:08+00:00
msgType: Update
event: Aviso Laranja Agitação Marítima
severity: Severe
onset: 2026-11-11T12:00:00+00:00
expires: 2026-11-11T21:00:00+00:00
areaDesc: Lisboa
description: Ondas de sudoeste com 5 a 7 metros e que poderão atingir até 13 metros de altura máxima.

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6288
sent: 2026-11-10T21:23:00+00:00
msgType: Update
event: Aviso Laranja Precipitação
severity: Severe
onset: 2026-11-11T00:00:00+00:00
expires: 2026-11-11T09:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e persistente.

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6290
sent: 2026-11-10T21:23:00+00:00
msgType: Update
event: Aviso Vermelho Precipitação
severity: Extreme
onset: 2026-11-11T09:00:00+00:00
expires: 2026-11-11T18:00:00+00:00
areaDesc: Lisboa
description: Precipitação forte e persistente.

identifier: 2.49.0.0.620.0.PT.261110212300.PO-10-6297
sent: 2026-11-10T21:23:00+00:00
msgType: Alert
event: Aviso Amarelo Precipitação
severity: Moderate
onset: 2026-11-10T21:24:00+00:00
expires: 2026-11-11T00:00:00+00:00
areaDesc: Lisboa
description: Precipitação por vezes forte e persistente.

f01

The hour: 2026-11-11T02:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"yellow","end_utc":"2026-11-11T06:00:00Z"},{"type":"rain","level":"orange","end_utc":"2026-11-11T09:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T02:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f02

The hour: 2026-11-11T05:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"yellow","end_utc":"2026-11-11T06:00:00Z"},{"type":"rain","level":"orange","end_utc":"2026-11-11T09:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T05:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f03

The hour: 2026-11-11T06:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"rain","level":"orange","end_utc":"2026-11-11T09:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T06:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f04

The hour: 2026-11-11T08:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"rain","level":"orange","end_utc":"2026-11-11T09:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T08:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f05

The hour: 2026-11-11T09:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"rain","level":"red","end_utc":"2026-11-11T18:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T09:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f06

The hour: 2026-11-11T11:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"rain","level":"red","end_utc":"2026-11-11T18:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T11:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f07

The hour: 2026-11-11T12:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"orange","end_utc":"2026-11-11T21:00:00Z"},{"type":"rain","level":"red","end_utc":"2026-11-11T18:00:00Z"},{"type":"wind","level":"red","end_utc":"2026-11-11T21:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T12:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f08

The hour: 2026-11-11T17:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"orange","end_utc":"2026-11-11T21:00:00Z"},{"type":"rain","level":"red","end_utc":"2026-11-11T18:00:00Z"},{"type":"wind","level":"red","end_utc":"2026-11-11T21:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T17:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f09

The hour: 2026-11-11T18:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"orange","end_utc":"2026-11-11T21:00:00Z"},{"type":"wind","level":"red","end_utc":"2026-11-11T21:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T18:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f10

The hour: 2026-11-11T20:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"orange","end_utc":"2026-11-11T21:00:00Z"},{"type":"wind","level":"red","end_utc":"2026-11-11T21:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T20:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f11

The hour: 2026-11-11T21:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"yellow","end_utc":"2026-11-12T03:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-11T21:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f12

The hour: 2026-11-12T02:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[{"type":"coastal_event","level":"yellow","end_utc":"2026-11-12T03:00:00Z"}]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-12T02:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.

f13

The hour: 2026-11-12T03:00:00Z

(the seventeen messages above)

Scored against: {"in_force":[]}, from incidents/2026-11-10-storm-charles/data/weather/warnings.json lisbon (IPMA's own warnings API as read 2026-11-11T02:38:08Z): the warnings whose start is at or before 2026-11-12T03:00:00Z and whose end is after it. Source of the text: api.ipma.pt, read 2026-11-11 02:38 UTC.