Story thread · 2 reports / 2 sources

The bots already won the front door

fastcompany.com · 1hShare

The bots are winning. That’s my clearest takeaway after reading the first section of the latest State of the Bots report from TollBit, which builds payment rails between publishers and AI crawlers . In it, People Inc.’s Chief Innovation Officer, Jonathan Roberts , outlines the company’s approach to AI bots scraping content from its many media properties: aggressively block unauthorized bots while allowing access to legitimate crawlers , which typically means some kind of licensing agreement. However, Roberts concedes that the unauthorized crawlers have become harder and harder to identify and block. The report shows evidence that “bad” bots sometimes try to masquerade as legit bots like Google’s, they often rotate IP address if their first scrape is blocked, and some industrial-scale scraping companies have resorted to using huge networks of devices in people’s homes to make it look like their traffic is coming from real people. Even People Inc. hasn’t been entirely successful at block

First report: Is Tech Dismantling Democracy? — slate.com, 2d