Internet Archive faces soaring storage costs as AI boom strains archiving
Updated
Updated · 404 Media · May 5
Internet Archive faces soaring storage costs as AI boom strains archiving
12 articles · Updated · 404 Media · May 5
Founder Brewster Kahle said preferred 28-30TB drives are scarce or costly as the Archive adds 100TB daily and maintains more than 210PB.
Wikimedia said rising memory and hard-drive prices and longer server lead times are forcing budget trade-offs and hardware life extensions, while Western Digital has effectively sold out 2026 enterprise inventory.
Micron is leaving the consumer market to prioritise data-centre customers, and archivists say AI scraping has also prompted website blocks that hinder preservation alongside storage shortages.
As AI consumes the world's data storage, is the internet's memory being permanently erased?
With memory shortages lasting until 2030, what happens when the digital world simply runs out of space?
Beyond high prices, how is the AI boom straining our power grids and raising your electricity bill?
The Shrinking Web Memory: Publisher Blocks and AI Storage Shortages Endanger 210PB of Archived News in 2026
Overview
In 2026, major news publishers blocked the Internet Archive's Wayback Machine crawlers, driven by fears that AI companies were scraping copyrighted news content without permission. Simultaneously, AI firms overwhelmed the Archive with massive data requests, threatening its mission to preserve digital history. This led to significant gaps in the historical record, harming journalists, researchers, and the legal system that rely on archived web pages. The AI boom also caused a severe shortage and price surge in storage hardware, forcing the Archive to make difficult choices about maintaining its vast data infrastructure. Despite ongoing talks with publishers, legal battles and resource challenges continue, prompting archivists to adopt AI tools responsibly while advocates call for policies to protect digital heritage.