2020-08-05 Abusing Soweli Lukin
A long time ago, I wrote Soweli Lukin, a web proxy for Gopher and Gemini. That allowed me to do some munging of the Gopher texts and render it as HTML, which meant reflowing long lines. That’s the main reason Gopher clients don’t work for me on mobile phones.
Now, if somebody wants to crawl all of Gopher space, they can crawl Gopher… or they can use a web proxy. That’d be pretty aggravating, of course, but not inconceivable. There are, after all, lots of people not wearing masks and endangering others, I bet they’re also willing to write such crawlers.
Now look at that, for the logfiles of today and the day before:
root@sibirocobombus:~# grep soweli-lukin /var/log/apache2/access.log /var/log/apache2/access.log.1 | /home/alex/bin/leech-detector ip hits bandw. hits% interv. status code distrib. 173.231.59.198 14956 5K 91% -0.0s 200 (99%), 500 (0%), 502 (0%) XX.XX.XXX.XXX 629 2K 3% -5.3s 200 (97%), 502 (1%), 500 (0%) XXX.XXX.XX.XXX 433 4K 2% -1.4s 200 (99%), 502 (0%) XX.XX.X.XXX 221 7K 1% 92.9s 502 (68%), 200 (31%) …XXXX:XXXX:XXXX:XXXX 16 5K 0% 25.2s 200 (56%), 500 (25%), 304 (18%) …XX:XXX:XXX:XX:XXX:X 12 4K 0% 1756.0s 200 (100%)
Fifteen thousand hits.
Reminds me of 70.113.100.216 who was slamming Gemini space a few days ago.
Well… I used ipset to drop their packets at the firewall, for now. Let’s see whether they change it, trying to make a comeback.
While I was at it, I also checked the IP numbers with requests in the hundreds. One of them was my monitor that checks whether the site is up, the other two were spiders with such promising names as “spider@seocompany.store” and “SEOkicks”. Ugh. Think of all the CO₂ wasted because of this SEO-💩.
Hmm it feels wrong to me that you’re basically do two parsings. Without having seen any code, wouldn’t it be cleaner to extend the markdown parser to do the steps you describe?
– Andreas Gohr 2019-06-25 12:10 UTC
Yeah, that would be better, I’m sure. At the same time, I like using it as a black box. It would make it easy to exchange one Markdown parser for another.
– Alex Schroeder 2019-06-25 15:38 UTC
On my own homegrown gopher client, I have a keyboard command (or two) to reflow the text if needed. That way, I don’t go crazy trying to reflow those crazy baudbaby pages.
– Sean Conner 2019-06-26 01:13 UTC
Yeah, VF-1 uses
fold -w 70 -s
if you use thefold
command. So what you’re saying is:I’m still operation under the assumption that we can do better, but it’s not a bad fallback. 🙂
– Alex Schroeder 2019-06-26 06:20 UTC
Add Comment