Check Domain History in Bulk with Wayback Machine

Summary

Pixel’s announcement explains how to check expired-domain history in bulk using the Wayback Machine and an Apify Actor. It covers submitting domain lists, fetching archived page titles, and reviewing summary rows to spot useful site histories or risky past use before buying.

DEV Community

Check a domain's history before you buy it: bulk Wayback Machine lookups for expired-domain due diligence

Disclosure: I'm Lindenbyte, the small studio that built the Apify Actor used below, and this post was drafted with AI assistance. The Wayback Machine itself is free and you can do all of this by hand; this is a bulk shortcut. An expired or auctioned domain comes with a past. Sometimes that past is a good one (years of a real site, backlinks, trust). Sometimes the domain dropped, sat dark, got picked up by a casino affiliate or a replica-watch shop, and dropped again. Registrars and auction lists rarely tell you which. The Internet Archive usually can. For one domain, the free route is fine: open https://web.archive.org/web//example.com, scroll the calendar, click a few years. For the 50 names on today's auction list it is an afternoon. This post does the same check in bulk and gives you one summary row per domain. 1. Put your candidate domains in one run Open the Wayback Domain History checker and paste your list. To show what the output looks like I used a deliberately mixed set: a long-lived personal site, a famous dead service, a sold brand, a social network that went dark, and a name that was never registered. {
"domains": [
"paulgraham.com",
"geocities.com",
"mtgox.com",
"friendster.com",
"pets.com",
"lindenbyte-never-registered-48213.com"
],
"fetchTitles": true,
"maxTitleFetches": 10
}
fetchTitles opens up to 10 archived copies of each homepage (spread over the years) and reads the , which is where most of the useful signal comes from. Leave the archived URL list (includeUrls) off for due diligence: it is slow on big domains and you don't need it to judge a name. The run took 4 min 24 s for the six domains. The Actor goes slowly on purpose (the Wayback APIs are free and rate limited), so for a long list, start it and come back, or schedule it. 2. Read the summary row Each domain gets one row. The fields that matter most for a buying decision: Domain First seen Capture days Longest gap Flags paulgraham.com 2000-09-18 4,282 6.5 months (2001) none geocities.com 1996-10-22 5,401 6.7 months (1997) redirected-elsewhere mtgox.com 2007-05-13 1,607 22.8 months (2007-09 to 2009-08) none friendster.com 2002-05-25 5,084 15.9 months (2020-11 to 2022-03) redirected-elsewhere pets.com 1996-12-21 1,369 19.8 months (1997-98) redirected-elsewhere lindenbyte-never-registered-48213.com - 0 - never-archived What each part tells you: firstCapture / archiveAgeYears: when the domain first had something on it. paulgraham.com has 26 years of history; that is a different asset from a name first seen last spring. capturesPerYear and missingYears: continuity. "Capture days" means days with at least one homepage capture. Friendster shows 308 capture days in 2020, zero in 2021, 2 in 2022, then 186 in 2023. That hole is exactly the pattern you look for on an expired domain: the old site stopped, something new started later. titleSamples: what was actually there. Friendster's titles go from "Friendster - Home" (2006-2009) to "Friendster.com - Living the Game" (2013-2018, the gaming era) to "Friendster" again in 2024 and "Friendster — real friends, real life" in 2026: three different sites on one name. pets.com starts as "Coming Soon!" (1996-97), becomes the famous store, then a community site ("The Online Community for Dog, Cat, Pet Lovers", 2010). redirects / externalRedirectDomains: where the homepage pointed. pets.com redirected to petsmart.com (2003, 2021, 2026) and mypetsmart.com (2015); geocities.com to three Yahoo hosts; Friendster in 2020 to molglobal.mol.com/investor-relations/ (its owner at the time). A redirect to another site usually means a sale, a merger, or, on random expired names, a link network. statusCodes: paulgraham.com served 200 on 3,947 of 4,282 capture days. geocities.com: 3,317 of its days were 3xx redirects, so the "30-year-old domain" has mostly been a pointer to Yahoo since 1999. Never archived is an answer, not an error: you get a row with archived: false and zero captures. 3. Fl...
Check Domain History in Bulk with Wayback Machine
Powered by Readybot.io*

The latest from Pixel

Incident Response: Look Beyond the Red Alert

Pixel’s DEV Community announcement explores how teams can fixate on one failing service while overlooking the real cause of an outage, such as a full …

Sach: AI Fact-Checks WhatsApp Forwards in India

Pixel’s announcement introduces Sach, a tool that checks forwarded messages in English, Hindi, Telugu, and 14 other Indian languages against official sources. It gives cited …

Java 25 Removes SecurityManager: What to Fix

Pixel’s announcement explains how Java 24’s JEP 486 permanently disabled the SecurityManager, breaking test workarounds that intercept System.exit. It recommends returning exit codes or throwing …