WIP: Adding stats, fetching previous snapshot meta data from Archive.org

Published on Oct 03, 2024

This is the forth update in my "WIP" post series, previous posts:

After a small break I continued on this project. Over the past days I focused on the landing page and the dashboard for each sitemap. I wanted to show some more data to the user there so I also started to fetch the "existing snapshot" information from Archive.org. That way I can calculate if something already existed in the Internet Archive or if my tool would be the first one to trigger a snapshot. This should increase the perceived value of the tool as that would mean that Archive.org was not previously aware of that URL.

I didn't know this API existed, and accidentally found it a few weeks ago. It gives you both the timestamp and the status of the last snapshot.

~|⇒ curl -q https://archive.org/wayback/available\?url\=https://blog.notmyhostna.me | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   223  100   223    0     0    441      0 --:--:-- --:--:-- --:--:--   441
{
  "url": "https://blog.notmyhostna.me",
  "archived_snapshots": {
    "closest": {
      "status": "200",
      "available": true,
      "url": "http://web.archive.org/web/20240930201329/https://blog.notmyhostna.me/",
      "timestamp": "20240930201329"
    }
  }
}