3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 2 days agoCloudflare blocking AI crawlerslemmy.relayeasy.comimagemessage-square33linkfedilinkarrow-up1334arrow-down12file-text
arrow-up1332arrow-down1imageCloudflare blocking AI crawlerslemmy.relayeasy.com3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 2 days agomessage-square33linkfedilinkfile-text
Cloudflare trying to stop AI crawling somehow! https://arstechnica.com/tech-policy/2025/07/pay-up-or-stop-scraping-cloudflare-program-charges-bots-for-each-crawl/
minus-squareProntomomo@lemmy.worldlinkfedilinkEnglisharrow-up5·1 day agoI don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling
I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling