1.3.6.1.4.1.61513@xssfox@cloudisland.nz
So I see often any Ai bots scraping websites often at high rates. What's the actual go here, are they training without storing the scrapped data? Or is this after training when people are using them (requests to supplement a query, eg tool/mcp usage). Or are people inflating scraping the site 7 times a second to mean the whole site when it's just one page
I'm not doubting the extra load, I'm just curious about how it's not a scrape once and then they are done kind of deal
networking wizard catgirl@pearl@fedi.rrr.sh
@xssfox@cloudisland.nz a lot of them will scrape a page, then scrape it a few hours later in case it changed in the last few hours, for every page on a website, including ones that are expensive for the server to handle
