The book database ISBNdb has suspended a controversial service that appeared to facilitate the destruction of rare books for artificial intelligence training. The company stated that a marketing landing page describing the service was merely a "test of market interest" and that no such service was ever brought to life, vowing to remove the page in response to widespread concern.
Reports emerged this week detailing a disturbing trend where book suppliers were receiving unusually large orders. These orders were suspected to be from AI firms seeking source material, potentially to bypass copyright laws and acquire cleaner data. At the heart of the ensuing controversy was ISBNdb, a database that not only seemed to endorse this practice but also offered a way to connect AI companies with book depositories. A now-deleted landing page on their site suggested that books, with their "curated, peer-reviewed, domain-specific human knowledge," offered a superior training dataset compared to web crawls.
The outrage, while peaking this week, follows earlier revelations. In January, court documents exposed "Project Panama," an initiative by AI firm Anthropic aimed at scanning and destroying books. Anthropic reached a $1.5 billion settlement with authors over the practice, but unsealed filings indicated that the method, while generating negative publicity, was legally permissible.
Suppliers have reported mixed feelings about the bulk orders. One anonymous seller noted the financial benefit and the clearing of old inventory but expressed discomfort with the end-use and the destruction of uncommon books. This practice has become more scrutinized amid concerns about AI companies struggling to find high-quality, "clean" training data. Issues such as low-quality web content and AI-generated material creating negative feedback loops have driven some companies to seek more authoritative sources. For example, A24 recently partnered with Google to aid in training DeepMind, aiming to improve its media generation capabilities.
The physical destruction of books is likely motivated by cost-saving measures. While scanning and archiving do not inherently require destroying the source material, performing these tasks quickly and cheaply could lead to the shredding of books after data extraction. The specific reasons for the destruction in "Project Panama" were not detailed, but the method suggests a focus on rapid processing over preservation.