The record
Kept by incident.archivist@web-summit.dinis, 11 Nov 2026, 04:27 UTC
Evidence (a page reported by its own publisher, kept as we read it). This is our dated copy, not the live page; the live page may have changed since.
| Address | https://www.theguardian.com/robots.txt |
| Source | theguardian.com (press) |
| Read | 11 Nov 2026, 04:27 UTC (2026-11-11T04:27:03Z) |
| Answer | HTTP 200; Last-Modified: Mon, 09 Nov 2026 10:35:05 GMT; served from a cache, age 0 s |
| Title | (none) |
| Against the capture before | the first capture of this address |
| sha256 of the HTML as served | 3387cd8d393a0c257e687ce63fcf30558ada1eb1fc167bfc51cb216e29005db2 |
| sha256 of the visible text | 53a88da688c7cb1f0d2dc3653e7f78c972ee71a56e52d43ce094c0f3d4954e25 |
| sha256 of the screenshot | 1fff39e401fd273f1296d01df168243e9ff2f51708cb74d8d1e485ac7afe928e |
| Shown here | no: the capture is kept privately; only the hash, title, time and a short excerpt are shown |
| Licence | Press: all rights reserved by the publisher. The capture is kept privately; shown here are only the hash, the title, the time read and a short excerpt, with the link. |
| Robots | robots.txt has no rule for this address; read as Mozilla/5.0 (compatible; page-capture/1.0), not logged in |
The first words on the page (a short excerpt): "# This is the robots.txt file for theguardian.com # Guardian content is made available under our terms and conditions of use. # Any other uses are not permitted, incl. but not limited to: for large language # models (LLMs), machine learning and/or artificial intelligence-related…"
Every capture of this address
| Capture | Read | Answer | Against the one before | Text hash |
|---|---|---|---|---|
| this one | 11 Nov 2026, 04:27 UTC | 200 | first | 53a88da688c7cb1f |
The outside address
This link leaves this site. What you find there is the live page, which may no longer match the capture above: https://www.theguardian.com/robots.txt
The record as data: 0427-robots-txt.json. All captures: evidence.