Welcome! :-)

This is both a wiki (a website editable by all) and a blog (an online diary about the stuff Alex Schroeder reads and does). If you’re a friend or relative, you might be interested in reading Life instead of this page. If you’ve come here from an RPG blog, you might want to head over to RPG. There are other similar categories to be found on the SiteMap.

FΓΌr Rollenspieler gibt es ebenfalls eine eigene RSP Kategorie.

2017-02-17 Leisure

+David McGrogan writes on his blog about creativity requiring time off to be creative because you need to balance both free roaming thoughts induced by boredom with focus and attention induced by discipline. If you’re constantly distracted at work by meetings, or at home by social media, it’s hard to get anything done.

Luckily, I only work 60% so I get to have Fridays off and a four month break every year. People sometimes ask me whether this isn’t boring to the extreme. To which I say that most people simply adapt to their current way of life and that’s great. It allows us to be happy under a wide range of circumstances. It also means that we’re never bored, as in spending idle hours contemplating life, the kind of leisure the rich Greek philosophers must have felt without the need to work, when all you had to do was hang out, drink, talk, and some calisthenics to defend your city’s interests on the battlefield every now and then. Anyway, leisure and occasional bouts of boredom are essential, I think. As we’re busy at work and stressed out by family, squeezing in a few hours of playing games with friends, I think it’s easy to forget that something atrophies at the same time. That very source of novelty.


Add Comment

2017-02-15 Load

What the hell happened around 18:00 yesterday?

Processes going up Threads making a big jump Munin taking minutes to run CPU 100% Load over 30

Was it an unusual amount of requests? I don’t think so:

alex@sibirocobombus:~$ sudo cat /var/log/apache2/access.log.1 | bin/time-grouping
      15/Feb/2017:06        426     1%
      15/Feb/2017:05        949     3%
      15/Feb/2017:04       1050     3%
      15/Feb/2017:03       1320     4%
      15/Feb/2017:02       1292     4%
      15/Feb/2017:01       1272     4%
      15/Feb/2017:00       1253     4%
      14/Feb/2017:23       1205     4%
      14/Feb/2017:22       1181     4%
      14/Feb/2017:21       1195     4%
      14/Feb/2017:20       1161     4%
      14/Feb/2017:19       1015     3%
      14/Feb/2017:18        965     3%
      14/Feb/2017:17       1121     4%
      14/Feb/2017:16       1129     4%
      14/Feb/2017:15       1022     3%
      14/Feb/2017:14        852     3%
      14/Feb/2017:13       1085     4%
      14/Feb/2017:12       1094     4%
      14/Feb/2017:11       1154     4%
      14/Feb/2017:10       1115     4%
      14/Feb/2017:09       1198     4%
      14/Feb/2017:08       1408     5%
      14/Feb/2017:07        973     3%
      14/Feb/2017:06        531     1%

Nothing unusual in the Toadfarm logs.

I noticed something weird in my Apache logs, though:

alex@sibirocobombus:~$ sudo cat /var/log/apache2/access.log.1 | bin/leech-detector | head
           IP Number       hits bandw. hits% interv. status code distrib.
        1590     4K   5%   54.3s  200 (99%), 301 (0%), 404 (0%)       1255     2K   4%   68.7s  301 (33%), 200 (33%), 302 (32%), 503 (0%)

Who is and what are they doing?

communitywiki.org:80 - - [15/Feb/2017:06:38:14 +0100] "GET /de?action=rss;rcidonly=OpenWLAN HTTP/1.1" 301 487 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"
communitywiki.org:443 - - [15/Feb/2017:06:38:14 +0100] "GET /de?action=rss;rcidonly=OpenWLAN HTTP/1.1" 302 4026 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"
communitywiki.org:443 - - [15/Feb/2017:06:38:14 +0100] "GET /wiki?action=rss;rcidonly=OpenWLAN HTTP/1.1" 200 4687 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"
communitywiki.org:80 - - [15/Feb/2017:06:39:45 +0100] "GET /en?action=rss;rcidonly=ConservationOfRationality HTTP/1.1" 301 521 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"
communitywiki.org:443 - - [15/Feb/2017:06:39:45 +0100] "GET /en?action=rss;rcidonly=ConservationOfRationality HTTP/1.1" 302 4060 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"
communitywiki.org:443 - - [15/Feb/2017:06:39:45 +0100] "GET /wiki?action=rss;rcidonly=ConservationOfRationality HTTP/1.1" 200 4698 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"

What we see here is that somebody is requesting various pages via HTTP (port 80) and getting redirected permanently (status 301) to HTTPS (port 443) and getting redirected temporarily (status 302) to a different URL (/wiki instead of /en). We also see that they are getting the URL in various languages.

Just to make sure, I added the following to my .htaccess file:

# idiots downloading every single RSS file
order allow,deny
deny from
allow from all

And now they’re getting something else:

communitywiki.org:80 - - [15/Feb/2017:08:51:57 +0100] "GET /cw-fr?action=rss;rcidonly=LackOfReworking HTTP/1.1" 301 507 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"
communitywiki.org:443 - - [15/Feb/2017:08:51:57 +0100] "GET /cw-fr?action=rss;rcidonly=LackOfReworking HTTP/1.1" 403 3930 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.66 Safari/537.36"

Their HTTP request gets redirected to HTTPS and then they’re getting an access denied (status 403).

But wait, are they perhaps the cause for all these problems?

alex@sibirocobombus:~$ sudo cat /var/log/apache2/access.log.1 | grep | bin/time-grouping
      15/Feb/2017:06         24     1%
      15/Feb/2017:05         60     4%
      15/Feb/2017:04         54     4%
      15/Feb/2017:03         44     3%
      15/Feb/2017:02         48     3%
      15/Feb/2017:01         48     3%
      15/Feb/2017:00         57     4%
      14/Feb/2017:23         78     6%
      14/Feb/2017:22         38     3%
      14/Feb/2017:21         41     3%
      14/Feb/2017:20         45     3%
      14/Feb/2017:19         42     3%
      14/Feb/2017:18         33     2%
      14/Feb/2017:17         65     5%
      14/Feb/2017:16         46     3%
      14/Feb/2017:15         69     5%
      14/Feb/2017:14         24     1%
      14/Feb/2017:13         45     3%
      14/Feb/2017:12         68     5%
      14/Feb/2017:11         62     4%
      14/Feb/2017:10         69     5%
      14/Feb/2017:09         63     5%
      14/Feb/2017:08         45     3%
      14/Feb/2017:07         48     3%
      14/Feb/2017:06         39     3%

Nope, they didn’t start yesterday at around 18:00.

Oh well. :(

Restarted Toadfarm. Manually killed all the farm processes I found via top. Restarted Apache for good measure. I think we’re good.

I also checked /var/log/syslog.1 and found nothing surprising around 18:00. Spamd is checking mails, Munin is running, more spam filtering, and that basically covers the time from 17:00 to 19:00.

Well, things are still not good.

top - 10:39:14 up 29 days, 23:18,  1 user,  load average: 5.26, 5.60, 5.69
Tasks:  66 total,   6 running,  57 sleeping,   0 stopped,   3 zombie
%Cpu(s): 98.7 us,  0.7 sy,  0.0 ni,  0.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   4194304 total,  2157836 used,  2036468 free,        0 buffers
KiB Swap:  1048576 total,   480444 used,   568132 free.  1456564 cached Mem

18835 alex      20   0  217104  71452   2284 R  19.0  1.7  20:59.77 /home/alex/farm
17130 alex      20   0  216768  71140   2268 R  18.6  1.7  37:40.49 /home/alex/farm
18286 alex      20   0  217932  72364   2308 R  18.6  1.7  23:40.29 /home/alex/farm
19937 alex      20   0  218128  72652   2332 R  18.6  1.7  16:56.54 /home/alex/farm
22862 alex      20   0  218780  73208   2332 R  18.6  1.7   7:35.41 /home/alex/farm
16345 alex      20   0  209860  68980   3432 S   1.7  1.6   0:52.01 /home/alex/farm
16344 alex      20   0  212272  69720   2180 S   0.7  1.7   0:52.34 /home/alex/farm
16346 alex      20   0  208112  65936   2012 S   0.7  1.6   0:55.12 /home/alex/farm
16347 alex      20   0  210904  68176   2120 S   0.7  1.6   0:52.76 /home/alex/farm

And they keep coming back. I’m slowly taking sites offline to where the trouble lies.


Comments on 2017-02-15 Load

OK, I started disabling sites in my Toadfarm configuration until I only had two sites left: campaignwiki.org and oddmuse.org. A few hours I checked back and I had ten farm processes running using 10% CPU each. Still a problem!

Then I disabled campaignwiki.org and tried calling it from the command line. Mojolicious allows me to do that, just like the CGI module does, too. It turns out that I can run a few requests, but when I load RecentChanges, I get a timeout.

Here’s me getting a regular page:

alex@sibirocobombus:~/farm$ perl campaignwiki.pl get '/?action=browse&id=Hauptseite'
[Wed Feb 15 22:39:27 2017] [debug] GET "/"
[Wed Feb 15 22:39:27 2017] [debug] Routing to a callback
[Wed Feb 15 22:39:27 2017] [debug] [CGI:CODE(0x1506388):11783] START /home/alex/farm/wiki.pl
[Wed Feb 15 22:39:28 2017] [debug] 200 OK (0.32674s, 3.061/s)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

Here’s me getting RecentChanges:

alex@sibirocobombus:~/farm$ perl campaignwiki.pl get '/RecentChanges'
[Wed Feb 15 22:39:42 2017] [debug] GET "/RecentChanges"
[Wed Feb 15 22:39:42 2017] [debug] Routing to a callback
[Wed Feb 15 22:39:43 2017] [debug] [CGI:CODE(0x1552388):11786] START /home/alex/farm/wiki.pl
[Wed Feb 15 22:39:43 2017] [debug] 200 OK (0.350602s, 2.852/s)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<div class="rc"><hr />[Wed Feb 15 22:39:58 2017] [debug] Inactivity timeout
[Wed Feb 15 22:40:03 2017] campaignwiki.pl: Problem loading URL "": Inactivity timeout

When I check top, I see a perl process running eating all available CPU for every time I try to view RecentChanges and got an inactivity timeout. That’s suspiciously like the problem I’ve been experiencing.

So here’s the current status: All the web apps are back online except for campaignwiki.org.

– Alex Schroeder 2017-02-15 21:50 UTC

I added some debug warning messages and discovered that the infinite loop was somehow caused by StripRollbacks for the Gridmapper namespace. I looked at the rc.log file and found a single rollback entry. I removed the entire section and everything seems to work. So campaignwiki.org is up again and the bug remains a mystery.

It took me a while to get load back down

– Alex Schroeder 2017-02-15 22:16 UTC

Add Comment

2017-02-14 Handwritten Digits for Download

If you’re interested in “deep learning” or neural networks and the identification of handwriting, sooner or later you will run into the MNIST database. (“The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.”)

Strangely enough, Americans don’t write their numbers like the people around here do. If I want to train a neural network to identify digits, I’m forced to train and test it on data that is subtly different from what I’d expect from local (Swiss) data.

Unless I collect my own data, that is. Some coworkers and I have started doing just that.

How to help:

  1. print the random numbers
  2. print the empty page
  3. copy the numbers onto the empty sheet
  4. scan it
  5. mail it → kensanata@gmail.com


Every page gets us 600 digits. Our goal is 10,000 digits. We’re already half way there!


Add Comment

2017-02-13 To Linear B

To Linear B:

  • “man loves woman” → “𐂀 𐀫𐀸 𐂁”
  • “dis is a tekusutu fo linear be” → “𐀇 𐀂 𐀀 𐀳𐀓𐀱𐀢 𐀑 π€ͺπ€šπ€€ 𐀀”

Because I added a bunch of whole words form the Wikipedia page, as well as some more rules (such as v is w and -s disappearing).ο»Ώ


Add Comment

2017-02-12 Grottos

Gridmapper now does caves (“grottos”). Click the image to get transported into dungeon mapping heavy.



Comments on 2017-02-12 Grottos

And I’ve prepared the ground for end-to-end tests using Nightwatch.js.

– Alex Schroeder 2017-02-12 12:45 UTC

Add Comment

2017-02-07 Load

I’m usually quite happy with our old MacBook Pro. We bought it many years ago and it has served us well. I don’t know when it started to downhill. After some upgrade or other, I guess? Starting up takes forever. It’s not anything in particular that takes forever. It’s just that everything is slow.

Everything is slow. Just now, maybe half an hour ago, the laptop was sleeping on the table and I wanted to get started with some web development. I opened it up and noticed that Firefox was not loading the pages I had open. I just got the spinning rainbow. Killed Firefox, started Emacs, started Safari, and then I noticed that it wasn’t just Firefox that wasn’t actually doing anything. Everything is slow.

Eventually I managed to open a terminal and run top. Here’s what it says. I’ve been watching it all this time and apparently the CPU is idle around 80% of the time and yet the load was higher than 7. No wonder it’s slow. But how is that possible?

Processes: 261 total, 2 running, 18 stuck, 241 sleeping, 1128 threads  21:35:37
Load Avg: 1.39, 2.19, 7.34  CPU usage: 10.9% user, 12.50% sys, 77.40% idle
SharedLibs: 121M resident, 33M data, 47M linkedit.
MemRegions: 49527 total, 790M resident, 68M private, 408M shared.
PhysMem: 4130M used (1380M wired), 220M unused.
VM: 2289G vsize, 623M framework vsize, 790349(0) swapins, 981169(0) swapouts.
Networks: packets: 4506731/4815M in, 3142372/731M out.
Disks: 17051508/261G read, 2698121/80G written.

I tried to quit it all again. Then I was looking at sudo top, ordered by mem, noticed a firefox, sent it a TERM signal, noticed some more like Keybase and two Keybase help processes still running, killed those, too. This still feels like a totally lousy approach. How can I get the old startup speed back?


Comments on 2017-02-07 Load

Well, right now I’m using Chrome and the system is reasonably fast. Is it Firefox? Is it the extensions I have installed? Is it because I just rebooted the laptop?

– Alex Schroeder 2017-02-07 21:20 UTC

There are a number of processes after reboot that clean-up caches of various things, perhaps those grinding concurrently are what’s running the load up?

– Kris Browne 2017-02-09 19:49 UTC

PhysMem: 4130M used (1380M wired), 220M unused. There is a lot of the memory being used…

Funny thing is, I had persistent-Firefox-although-terminated problems on Windows 7, too, but it’s a thing of the past. Could this be a firefox problem cross-platform if you use an older version?

– Rorschachhamster 2017-02-09 21:45 UTC

@Kris: Possibly. Sadly I can’t open a terminal window and run top while it grinding at the beginning. I’d need to see something in the logs but I’m not sure what to look for, or how to monitor that.

@Rorschachhamster: I think I’m always upgrading my browsers when new versions come out, so I don’t think that’s it. But then again, fixing the problem on Windows doesn’t mean it’s necessarily gone on the Mac.

Haven’t been running Firefox in a while and the system feels better. Installed uBlock Origin and uBlock Origin Extra. Some of my favorite extensions are still missing (recovery of text areas, Privacy Badger), but I’ll get there, eventually.

– Alex Schroeder 2017-02-10 06:58 UTC

Turn off auto-login temporarily, and SSH in from another device to run top (or htop if you build/brew it)? That’s how I found the above mentioned processes.

– Kris Browne 2017-02-10 20:04 UTC

Ah, good point!

– AlexSchroeder 2017-02-11 10:43 UTC

I also ordered top by vsize and killed Dropbox because it was first or second and using 470MB.

– AlexSchroeder 2017-02-11 10:45 UTC

I’ve been using the laptop for a bit now with Chrome instead of Firefox and it has never been sluggish and fast enough when starting up. Not sure yet how it will fare once we switched users back and forth a few times but for the moment I’m happy and I didn’t even have to watch processes using a remote.

– Alex Schroeder 2017-02-12 12:48 UTC

Add Comment

2017-02-05 Running

The temperature rises above 0Β°C and I feel like running again. So I went running yesterday and the day before. And now my calves hurt as if somebody had kicked them. Yay for forefoot running in my running sandals. :)


Add Comment

2017-02-05 History Podcasts

I’m listening to another podcast while I wait for my wife to get up…

As I’ve listened to nearly all the episodes of Robin Pierson’s History of Byzantium podcast (after listening to all the episodes of Mike Duncan’s History of Rome podcast, naturally) (and all the while listening to all the episodes of Scott C.’s The Ancient World podcast) I’m starting to look for the next podcast with exciting history material to listen to. Currently I’m enjoying the first few episodes of Sharyn Eastaugh’s History of the Crusades. We’re living in a great time for history lectures when we’re walking around town and doing our chores. Yay!

20’000 peasants including women and children walking from Germany to Turkey? Who needs a zombie apocalypse if you can have a starving families apocalypse.

Today, though, I’m listening to Dan Carlin’s Hardcore History. I listened to many of Dan’s episodes, but I didn’t go through his entire archive.

I guess I’m fascinated by podcasts that, taken together, form an ongoing narrative from the ancient past to the present.

Mike Duncan’s Revolutions podcast does just that. The English, American, French and Spanish American revolutions all belong to a continuity. It’s great.

As for another history podcast that doesn’t fit this linear model, I recommend Laszlo Montgomery’s Chinese History Podcast.

(Based on a G+ post.)


Add Comment

2017-02-04 Blogging

On G+, somebody recently asked about blogging advice and I wrote the following:

Plan to blog for the long run – like working without a burnout, like living without a midlife crisis. Only write when you do not depend on well wishes of others. Only write about the issues you care about. Only write when you feel like it.

Will you stop if nobody reads your blog for a week? Will you stop if you get a comment saying you’re an idiot? Will you stop if you don’t get a single +1 or Like? If you are, then I predict that you will stop blogging within a year because there will always be weeks when this happens.

I use my blog as a public notepad. Write things you want to remember. Write things you want to link back to in the future.

Others may occasionally find something on my site by searching the web even if they don’t follow my blog. Regular readers are a happy bonus.

Don’t think about other people’s interests because they’re hard to guess and even if you knew, if they’re not your own, then it should be called work.


Add Comment

2017-02-03 Map Collaboration

I don’t think anybody ever used the ability to host a Gridmapper map, live. Well, the least used feature just got better! I know, I know. But hey, it’s free, and I liked working on it, so there’s that. The new feature: collaborate on a map. That is, where as before would one of you would be the mapper and the others could get a read-only view of what the mapper was doing, now you can all draw at the same time. That’s what I hope, anyway.

A word of caution: In the spirit of all things Internet, there is currently no protection at all. Anybody can join a map. I’ll probably add optional passwords, soon. Please leave a comment if you were collaborating with friends and vandals joined your map!

Please note that Gridmapper has no chat functionality. You must meet people online, elsewhere. One way to do this is to join the DIY DND chatroom on Discord. It’s like G+, but one ongoing conversation.

And when you do collaborate on a map, please let me know. My current estimate of Gridmapper users is one or two active users.


Add Comment



Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. See Info for text formatting rules. You can edit this page if you need to fix typos. You can subscribe to updates by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.

Referrers: Dreams of Mythic Fantasy Bloglines Gothridge Manor Diary campaignwiki.org/gridmapper.svg? gggg g g g g... ggg g rsp-blogs.de