Last edit
Summary: I'm surprised – bots only amount to 9% of my traffic? Great! :)
Changed:
< I'm surprised -- bots only amount to 9% of my traffic? Great! :)
to
> I'm surprised -- bots are responsible for 14% of all my hits. That's better than
> [[Comments on 2009-10-15 Network Traffic|what I saw in 2009]].
> The way I have set up my web pages, bots should not crawl "actions" (URLS containing the ##action## parameter) -- and yet 9% of them do it, and superfeedr does it most of all. I should investigate, I guess.
Back in 2009 I wondered about network traffic. Decided to take another look. Yesterday’s log file via bot-analyze:
aschroeder@thinkmo:~$ bot-analyze < logs/access.log.1 | head
----------------------------Bandwidth-------Hits-------Actions
Everybody 2177M 113317
All Bots 274M 16154 100% 9%
--------------------------------------------------------------
www.google.com 92390K 5542 34% 2%
yandex.com 63545K 3954 24% 9%
bot 33803K 2309 14% 1%
superfeedr.com 31740K 1091 6% 63%
www.bing.com 14161K 525 3% 4%
ahrefs.com 9251K 465 2% 2%I’m surprised – bots are responsible for 14% of all my hits. That’s better than what I saw in 2009.
The way I have set up my web pages, bots should not crawl “actions” (URLS containing the action parameter) – and yet 9% of them do it, and superfeedr does it most of all. I should investigate, I guess.