Soweli Lukin

Soweli Lukin is a web app acting as a Gopher and Gemini client.

The name is Toki Pona and means “lovely mammal looking” (a gopher client). It started as as 2019-06-19 A Simple Text Client Simple Text Client.

This is the client that’s part of Nimi Mute.

2020-08-05 Abusing Soweli Lukin

A long time ago, I wrote Soweli Lukin, a web proxy for Gopher and Gemini. That allowed me to do some munging of the Gopher texts and render it as HTML, which meant reflowing long lines. That’s the main reason Gopher clients don’t work for me on mobile phones.

Now, if somebody wants to crawl all of Gopher space, they can crawl Gopher… or they can use a web proxy. That’d be pretty aggravating, of course, but not inconceivable. There are, after all, lots of people not wearing masks and endangering others, I bet they’re also willing to write such crawlers.

Now look at that, for the logfiles of today and the day before:

root@sibirocobombus:~# grep soweli-lukin /var/log/apache2/access.log /var/log/apache2/access.log.1 | /home/alex/bin/leech-detector
                  ip       hits bandw. hits% interv. status code distrib.      14956     5K  91%   -0.0s  200 (99%), 500 (0%), 502 (0%)
       XX.XX.XXX.XXX        629     2K   3%   -5.3s  200 (97%), 502 (1%), 500 (0%)
      XXX.XXX.XX.XXX        433     4K   2%   -1.4s  200 (99%), 502 (0%)
         XX.XX.X.XXX        221     7K   1%   92.9s  502 (68%), 200 (31%)
…XXXX:XXXX:XXXX:XXXX         16     5K   0%   25.2s  200 (56%), 500 (25%), 304 (18%)
…XX:XXX:XXX:XX:XXX:X         12     4K   0% 1756.0s  200 (100%)

Fifteen thousand hits.

Reminds me of who was slamming Gemini space a few days ago.

Well… I used ipset to drop their packets at the firewall, for now. Let’s see whether they change it, trying to make a comeback.

While I was at it, I also checked the IP numbers with requests in the hundreds. One of them was my monitor that checks whether the site is up, the other two were spiders with such promising names as “” and “SEOkicks”. Ugh. Think of all the CO₂ wasted because of this SEO-💩.

Add Comment

2019-07-01 Soweli Lukin does TLS

Soweli Lukin now does TLS!

I’ve decided to use a kind of “mode.”

  1. if you’re following a gophers link (note the trailing s), you are upgraded to TLS and the TLS checkbox is checked
  2. if you’re following a gopher link (no trailing s), and the TLS checkbox is checked, to app asks you to uncheck the TLS checkbox first

That is, you can’t simply follow a link to an unencrypted site if you’ve been browsing an encrypted site, but you can always follow a link to an encrypted site.

Now, when looking at a Gopher map, we can’t tell whether a link is encrypted or not. It probably isn’t. In this case, if the TLS checkbox is checked and the link points to the same host and the same port, then the link is going to use gophers (with trailing s), and you’ll be fine. Otherwise, it’s going to use gopher (no trailing s), and clicking on it will again ask you to uncheck the TLS checkbox first. The app will also show you a helpful (?) unlocked padlock next to the link... 🔓

If you know that the host and port in question are in fact encrypted, you should change the gopher scheme to gophers (add the trailing s).


Add Comment

2019-06-25 Reflowing arbitrary text

So I’ve been struggling at reflowing arbitrary text written with Gopher in mind. This is a culture that is steeped in ASCII art. My Gopher-compatible, simple text client Soweli Lukin isn’t doing too badly, but the code is horrible. 😀

Let’s take a look at the preprocessing I do before handing it off to the Markdown library which turns Markdown into HTML:

  1. split the text into paragraphs
  2. look at each paragraph individually
  3. if it has two lines and the second line is nothing but - or = it’s a Markdown header, so no changes required
  4. if the number of punctuation and whitespace characters is higher than the number of alphanumeric characters, then it’s some sort of ASCII art
  5. if every line of this paragraph looks like a list item (it starts with an asterisk followed by a non-asterisk), then each line is wrapped in code tags and ends in a br tag, because I’m assuming this is a bunch of Gopher items from a Gopher map, and each is actually a link so I cannot use a pre tag for the entire paragraph since that would prevent the Markdown library from processing the links
  6. in all other cases the ASCII art paragraphs are are wrapped in pre tags
  7. in addition to that, check if the paragraph consists of list items with each containing at least three spaces; in this case we’re assuming it’s a file listing with dates and sizes and so we wrap the list item in code tags
  8. furthermore, if the entire paragraph looks like a definition list, warp it in pre tags as well; but allow for some slack: a definition is a line starting with a letter, contains a colon followed by some whitespace, and some more characters; if the paragraph is at least 5 lines long, one of the lines need not be a definition; if the paragraph is at least 9 lines long, two of the lines need not be a definition

Are you crying, yet? I know I am! Some examples I’ve added to comments all over the code:

Examples of text that isn’t handled well:

It’s hard!


Comments on 2019-06-25 Reflowing arbitrary text

Hmm it feels wrong to me that you’re basically do two parsings. Without having seen any code, wouldn’t it be cleaner to extend the markdown parser to do the steps you describe?

Andreas Gohr 2019-06-25 12:10 UTC

Yeah, that would be better, I’m sure. At the same time, I like using it as a black box. It would make it easy to exchange one Markdown parser for another.

– Alex Schroeder 2019-06-25 15:38 UTC

On my own homegrown gopher client, I have a keyboard command (or two) to reflow the text if needed. That way, I don’t go crazy trying to reflow those crazy baudbaby pages.

Sean Conner 2019-06-26 01:13 UTC

Yeah, VF-1 uses fold -w 70 -s if you use the fold command. So what you’re saying is:

  1. a human has to decide whether reflowing is possible
  2. a tool is used to reflow the entire page

I’m still operation under the assumption that we can do better, but it’s not a bad fallback. 🙂

– Alex Schroeder 2019-06-26 06:20 UTC

Add Comment

2019-06-24 Writing to the Simple Text Server

A few days ago I wrote about the Simple Text Server I had written, Nimi Mute. At the time I really wanted to use those ASCII control characters. Now that I’ve thought about it some more, I decided to drop it and just use what ed uses: a period on a line by itself. And as for indicating that the user intends to upload a file, I decided to go with what shells use: a simple > to overwrite the file, or >> to append to the file.

This makes the documentation simpler to write, and I take that as a sign that I have been overthinking it.

I also discovered that Gopher maps are text files and so Nimi Mute can act as a Gopher server, no problem. I took advantage of this when writing my test cases. These now use both Nimi Mute and Soweli Lukin.


Add Comment

2019-06-19 A Simple Text Client

OK, this one goes together with the Simple Text Server I wrote about yesterday. The server is basically a weird Gopher/Finger server that can only serve text files. It doesn’t know anything about Gopher menus, of FTP commands, or directory listings. Everything else would be part of the client.

As @solderpunk said, one vision could be this:

  1. All connections are TLS secured
  2. Requests are a UTF-8 encoded selector and a CLRF, nothing more
  3. Responses are guaranteed UTF-8 encoded text
  4. No further structure of response is guaranteed
  5. Clients should be able to extract any URLs from anywhere in the text and construct a menu, with interface of their own choosing.

In my particular case, Nimi Mute (the server I wrote) sadly doesn’t qualify:

  1. TLS is optional. You can run it without encryption.
  2. Requests are not decoded. It’s raw bytes. These encode filenames and since I use UTF-8 encoded filenames, this might work. But it might also not work: remember that Unicode can be in NFC or in NFD, both equivalent encodings but using a different sequence of bytes. See Unicode equivalence for more.
  3. Responses are not guaranteed to be UTF-8 encoded. As I prefer using UTF-8, this usually works. However, yesterday I showed how you can download binary files from the text server. That alone shows you that arbitrary bytes are allowed.
  4. No further structure of response is guaranteed. This I support, haha!
  5. Clients should be able to extract any URLs from anywhere in the text and construct a menu, with interface of their own choosing. And that brings us to the topic of today: I need a proof-of-concept client.

My main problem is that I want a better Gopher client for my iOS phone. Since I have no idea how to write a native application, and since I don’t want to develop on my Mac, that leaves me with a web app. I can’t write a client-side web app because they all run in the browser sandbox which doesn’t have access to TCP sockets. All you have is web sockets. They are not compatible. For a while I tried to use Fennel, compile it to Lua, and use Fengari to host it. Then I thought about WebPerl, which is a Perl compiled with Emscripten to WebAssembly. But it’s all the same: no escaping from the sandbox.

So I decided to write a server-side client: Soweli Lukin. Here’s a link to yesterday’s blog post using it: check it out. Visit it using your browser. The browser access the web app I wrote, passes it a Gopher URL, which it fetches from my Gopher server, and renders as Markdown, with some additional translation. It works!

You can follow Gopher links to external sources: if they are links to text files, these URLs are accessed via Soweli Lukin. Everything else is up to your browser.

Ideally, this would be a native app on my phone, or a command line program on my laptop. But as it is, this was the simplest proof of concept I could think of.


Add Comment


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. Note: in order to facilitate peer review and fight vandalism, we will store your IP number for a number of days. See Privacy Policy for more information. See Info for text formatting rules. You can edit the comment page if you need to fix typos. You can subscribe to new comments by email without leaving a comment.

To save this page you must answer this question:

Just say HELLO