Drunk & Root to Selfhosted@lemmy.worldEnglish • 5 months agoHow to combat large amounts of Ai scrapersmessage-square54fedilinkarrow-up1125arrow-down15file-text
arrow-up1120arrow-down1message-squareHow to combat large amounts of Ai scrapersDrunk & Root to Selfhosted@lemmy.worldEnglish • 5 months agomessage-square54fedilinkfile-text
everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions
minus-square@HelloRoot@lemy.lollinkfedilinkEnglish4•5 months agocrowdsec is arguably not completely open source, but I’m very satisfied with it.
minus-square@lemonuri@lemmy.mllinkfedilinkEnglish3•5 months agoThe scraper blocklist on crowdsec requires a paid subscription, though, or did you find another workaround?
minus-square@HelloRoot@lemy.lollinkfedilinkEnglish1•5 months agoI don’t remember how I set it up a long time ago. But when I look at my server logs I only see myself. Afaik I just added the biggests lists. But I don’t remember.
crowdsec is arguably not completely open source, but I’m very satisfied with it.
The scraper blocklist on crowdsec requires a paid subscription, though, or did you find another workaround?
I don’t remember how I set it up a long time ago. But when I look at my server logs I only see myself.
Afaik I just added the biggests lists. But I don’t remember.