2017-12-30 Gopher Wiki

The problem with the Gopher protocol as defined in RFC 1436 is that it doesn’t allow you to send anything to the server except for a query string. That’s not good enough for wiki editing. At the same time, we don’t want to make the interaction between Gopher server and Gopher client harder than it needs to be. This page offers a proposal:

First, the server needs to know about a new item type which says: item is a file you can write to. Below is a Gopher menu for a wiki page. The selector for this menu is Alex/menu. The selector for writing a new page is Alex/write/text.

$ echo Alex/menu | nc localhost 7070
iThe text of this page:		localhost	7070
0Alex	Alex	localhost	7070
hAlex	Alex/html	localhost	7070
wReplace Alex	Alex/write/text	localhost	7070
1Page History	Alex/history	localhost	7070
i		localhost	7070
iThere are no links leaving this page.		localhost	7070
.

Let’s replace the text with something! We need to end our text with a period to let the server know that we don’t intend to send any more.

$ nc localhost 7070 << 'EOF'
Alex/write/text
test
.
EOF
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Or post a text file. You need to add the selector at the top and the period at the end:

$ ( echo Alex/write/text; cat haiku.txt; echo . ) | nc localhost 7070
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Verify the page content:

$ echo Alex | nc localhost 7070
Quiet disk rattling
Keyboard clicking, then it stops.
Rain falls and I think.
.

The wiki also allows a different kind of upload: binary files. When uploading a file, the server has two problems:

  1. When has the end of the file been uploaded? If the client hangs up after sending the file, it can’t read the response. If the client doesn’t hang up but does a shutdown on the socket, letting the server know it’s done writing, it would work – but you can’t do it using telnet or nc. That’s why we need to tell the server how long the file is going to be.
  2. What’s the content-type of the file? Sure, we could depend on file name extensions. But that doesn’t always work. My wiki restricts upload to certain MIME types, for example. Other Gopher servers might not care and that’s why this part is simply a solution I’m going to use for my wiki. I think it might just as well not be part of the new item type. It’s an local experiment, so to say.

So anyway, how would it work? It did not make an appearance in the menu above, but the selector would be Alex/write/file. The content-length is appended to this selector, separated by a tab character. wc -c counts the number of bytes and when it does that for standard input, no filename is printed.

$ echo -e "Alex/write/file\t"`wc -c < test.png` \
  | cat - test.png \
  | nc localhost 7070
3Page was not saved: Files of type application/octet-stream are not allowed.	localhost	7070
.

When writing a file using this selector, a content-type of application/octet is assumed.

The wiki will extract the MIME type from the selector, if possible: Alex/image/png/write/file, for example.

$ echo -e "Alex/image/png/write/file\t"`wc -c < test.png` \
  | cat - test.png \
  | nc localhost 7070
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Test it:

$ echo Alex | nc localhost 7070 > test2.png
$ diff test.png test2.png

Note that for my particular wiki, Alex/text/plain/write/file is not quite the same as Alex/write/text. The first will store a binary blob with MIME type text/plain which will be served like any other binary file: no parsing for links and all that. The second will edit a wiki page, which can be read using a Gopher client.

I understand that allowing file uploads can be tricky. It’s up to the Gopher server to handle this with care: store files in write-only upload directories, or provide all the peer review facilities like a wiki does, it’s up to you to write a good Gopher server. To paraphrase the description of item type 9: “Item is a binary file! Server will read until the TCP connection closes. Beware.”

Let’s get back to uploading text files and the problem of peer review facilities: Ideally, authors would provide some metadata as they upload their document. A summary of changes made, a name. My wiki will interpret a fenced off block at the very beginning of the document as metadata.

The following example sends the selector for writing a text file called Alex, a metadata block, and finally the new text.

$ nc localhost 7070 <<'EOF'
Alex/write/text
```
username: Alex
summary: a different Haiku
```
Cold and dark outside
Remember all the sick beds
Good riddance old year
.
EOF
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Check it out:

$ echo Alex | nc localhost 7070
Cold and dark outside
Remember all the sick beds
Good riddance old year
.

Let’s check Recent Changes, too:

$ echo do/rc | nc localhost 7070
iRecent Changes		localhost	7070
1Show minor edits	do/rc/showedits	localhost	7070
i		localhost	7070
i2018-01-06		localhost	7070
i		localhost	7070
1Alex	Alex/menu	localhost	7070
i20:55 UTC by Alex from 127.0.0.1: a different Haiku		localhost	7070
i		localhost	7070
i2018-01-05		localhost	7070
i		localhost	7070
1test	test/menu	localhost	7070
i21:53 UTC by 127.0.0.1		localhost	7070
.

Yay! 👍

Sources:

Try browsing the wiki:

Note that you will not be able to edit pages because spam protection requires you to answer a question before you are allowed to save. The question is simply that you need to say hello. Luckily, we can use the metadata for text files to do this:

$ nc alexschroeder.ch 70 << 'EOF'
SandBox/write/text
```
username: Alex
answer: hello
```
This is a test.
.
EOF
iPage was saved.		alexschroeder.ch	70
1SandBox	SandBox/menu	alexschroeder.ch	70
.

Anyway, here’s a quick setup for local experiments. You need to get Oddmuse, of course. And then:

Run perl stuff/server.pl starts an Oddmuse Wiki on port 8080. You can edit pages on http://localhost:8080/. Data is stored in /tmp/oddmuse.

Run perl stuff/gopher-server.pl --port=localhost:7070 --wiki=./wiki.pl --wiki_dir=/tmp/oddmuse to start the Gopher server on port 7070.

Run lynx gopher://localhost:7070 to browse the same wiki.

Tags:

Comments

“Phlogs” are Gopher Blogs. I think the SDF has the biggest collection of them (link goes via a proxy, the real deal is gopher://phlogosphere.org).

People don't like comments.

– Alex 2018-01-07 12:49 UTC


There are still problems with the code, though. I finally decided to document it as issue 15 on GitHub. For now, the interface will stay as it is. I just have issues getting this Mojo::IOLoop code to work. [This was later fixed by switching back to Net::Server.]

– Alex Schroeder 2018-01-08 14:24 UTC


Stuff I need to check:

– Alex 2018-01-13 17:20 UTC


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. Note: in order to facilitate peer review and fight vandalism, we will store your IP number for a number of days. See Privacy Policy for more information. See Info for text formatting rules. You can edit the comment page if you need to fix typos. You can subscribe to new comments by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.