Cloudflare will use the Internet Archive to boost redundancy

(Image credit: Shutterstock)

Cloudflare is now working with the Internet Archive to make the web more reliable by displaying cached copies of webpages from the nonprofit's Wayback Machine when sites go down.

According to the director of the WayBack Machine Mark Graham, the Wayback Machine will now store snapshots of websites enrolled in the CDN provider's Always Online service in order to provide access to those sites in the event they go offline.

In a blog post on the Internet Archive's site, co-founder and CEO of Cloudflare Matthew Prince explained how the new partnership can help make the internet more resilient, saying:

“The Internet Archive’s Wayback Machine has an impressive infrastructure that can archive the web at scale. By working together, we can take another step toward making the Internet more resilient by stopping server issues for our customers and in turn from interrupting businesses and users online.”

Boosting redundancy

While large customers have the necessary resources to run the hosting infrastructure of their websites in a reliable way, smaller ones often struggle when their web hosting provider goes offline. If Cloudflare is unable to access a site's content, it won't be able to serve it up across the network which is why its new partnership with the Internet Archive makes so much sense.

For those unfamiliar, the nonprofit's Wayback Machine is a digital archive of the internet and the pages that make it up. Since the service's launch in 2001, over 463bn pages have been added to the archive so that users can go back and see how popular sites used to look in the past.

In order to take advantage of Cloudflare's updated Always Online service, its customers will need to provide the Internet Archive with some information from their websites, such as a hostname and popular URLs, for web crawling. 

If their sites then go down, Cloudflare will first try to provide visitors with a stale or expired version of their content cached from an edge data center. However, if this data can't be found, the company will then ask the Internet Archive for its most recent site capture. Cloudflare will then serve up the cached copy of a site from the Internet Archive with a banner at the top of the page indicating that the original website is currently inaccessible.

Cloudflare's Always Online site availability service is offered at no charge to the company's customers and registering with the Internet Archive certainly seems like a good idea for site owners that want to avoid the possibility of visitors being unable to access their content.

Via The Register

Anthony Spadafora

After working with the TechRadar Pro team for the last several years, Anthony is now the security and networking editor at Tom’s Guide where he covers everything from data breaches and ransomware gangs to the best way to cover your whole home or business with Wi-Fi. When not writing, you can find him tinkering with PCs and game consoles, managing cables and upgrading his smart home.