Bad Bots

03 April 2026

Updated: 03 April 2026

I have a website that kept randomly going down every few weeks - turned out that I hadn’t enabled a healthcheck so the auto-restart didn’t know to that it had to restart. So after some annoyance I added a basic check and I think it’s fine now - probably …

Anyways, not the point - I looked into why it kept going down and seemed like it would randomly get thousands of simultaneous requests from bots and I decided that I don’t want that anymore?

I remembered reading a post a while back about blocking AI crawlers which upon some searching turned out to be by Matthais Ott. And so I did some reading and here’s a list of some related content on how to stop the LLMs:

I particularly resonate with the last paragraph of the last link in that list:

“If you personally work on developing LLMs et al, know this: I will never work with you again, and I will remember which side you picked when the bubble bursts.” - Please stop externalizing your costs directly into my face - Drew DeVault

None of these are Traefik so I’ll probably put together a bit of an implementation once I settle on something and put in in my Traefik notes