Dear readers and customers,

in 2022 the latest the AI bot craze and LLM takeover of the web began. Four years in a row, we were mostly directly unaffected. This year however, we and our web projects are flooded by AI bot spam requests on a daily basis. It directly affects how we count visitors and pageviews in a very negative way. To this date, this hasn’t stopped.

How did we identify?

Mainly because the requests happen for hours in intervals of a few seconds. Plus, the country changes per request, we have visitors from countries, or so it seems, which never visited us the decades before. The visited URL however, stays the same sometimes.

Looking over the years and by project, we also have an increase of 300-500%, depending on the project. This is neither realistic nor possible. We also don’t advertise for years, so there is absolutely no logical reason the “popularity” is this high suddenly. If this prevails over 2026, we actually could call us “mainstream media”, which is not only unrealistic but complete nonsense.

What to do now?

We tried hard, first we introduced GeoIPEnable On via the file .htaccess in all our projects. This was not really successful but we expected this of course. It should have stopped when you put China and USA in there; the two countries which develop and copy all successful AI bot systems out there at the moment.

We then looked into our tracking tool, Matomo, as it’s self-hosted. And we decided to just try to not track AI bots and crawlers at all. Since then it became a little better but it hasn’t stopped fully. Here is how we did it:

1) in your settings, go to “Global User Agent Exclusions” and add the following regex:
/GPTBot|ChatGPT-User|Google-Extended|ClaudeBot|Claude-Web|Anthropic-AI|PerplexityBot|cohere-ai|Omgilibot|FacebookBot|Diffbot|ImagesiftBot|Bytespider|Doubaobot|YisouSpider|Sogou|Baiduspider|KimiBot|DeepSeekBot/i
Save the settings. This prevents the known bots from USA and China completely but not the unknowns or rogue ones of course

2) install plugin “Tracking Spam Prevention”. It’s available via the “Marketplace”. Activate it, then had to its settings and block all requests from cloud systems. Also block all headless browsers; this is where the bots live.

3) more complicated because of the server-level configuration but if you can do it, do this:

For Nginx Users:

Add a map directive to your configuration file (usually /etc/nginx/nginx.conf or inside your server block) to reject bad user agents:

map $http_user_agent $ai_bot {
default 0;
~*gptbot|chatgpt-user|google-extended|claudebot|perplexitybot|bytespider 1;
}

server {
...
if ($ai_bot) {
return 403;
}
}

For Apache Users:
Add this snippet to your .htaccess file at the root of your website:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (GPTBot|ChatGPT-User|Google-Extended|ClaudeBot|PerplexityBot|Bytespider) [NC]
RewriteRule .* - [F,L]

We recommend you activate as much as you can yourself, even if only one or two points from the recommendations list here. It should get better, however this won’t block everything. This was expected but for the time being this is probably the only way.

In the meantime, for the first time in the history of the WWW, the traffic by bots has exceeded 60% according to Cloudflare. We are actually witnessing the transformation of this medium away from human interaction to artificial. And we guess it’s not for the better of us but it is what it is.

Nonetheless, we still believe in this technology and as time passes, we might have the tools to mitigate this problem. For the time being, we feel like Sysiphos but as we love Greece we not yet have a big problem with it. 🙂

Stay vigilant,
the aethyx staff