Oddmuse

Oddmuse is the wiki engine running all the wikis at emacswiki.org – including EmacsWiki itself. My main interests are:

See OddmuseRoadmap for my thoughts about the design of Oddmuse.

2018-01-03 Gopher Mode

Yeah, I’ve been working on Gopher stuff over the holidays.

  1. a Gopher server wrapper around Oddmuse wiki (and this site is running it, see gopher://alexschroeder.ch)
  2. a proposal of a new item type to write to a Gopher server with examples based on netcat, i.e. nc
  3. improvements to the Emacs Gopher client with support for HTML and the new item type (see this branch on GitHub)

Isn’t that amazing.

Tags:

Add Comment

2017-12-27 Markdown for Campaign Wiki

I’m experimenting with letting users switch between the default (Wiki Creole) and Markdown on Campaign Wiki. You start a page with “#MARKDOWN” on a line of its own and that switches some stuff around. Most of the text formatting rules still apply – you can still use bbCode, for example. But it switches off Creole and it switches on Markdown.

Here’s an example page, Halberds & Helmets: Treasure.

What do you think? Do you foresee any problems?

Tags:

Add Comment

2017-12-27 Gopher Server

The wiki is available as a Gopher server. This is weird, but it works.

Try it. lynx is a good Gopher client:

lynx gopher://alexschroeder.ch

or:

lynx gopher://alexschroeder.ch/02017-12-27_Gopher_Server

Does anybody know why I need to start the pagename with the gopher type character? (or any character at all)?

As for monitoring, I use monit and have set up the following:

check process gopher-server with pidfile /home/alex/alexschroeder-gopher-server.pid
      start program = "/home/alex/perl5/perlbrew/perls/perl-5.24.0/bin/perl /home/alex/farm/gopher-server.pl --setsid --user=alex --group=alex --port=70 --log_level=2 --log_file=/home/alex/farm/gopher-server.log --pid_file=/home/alex/farm/gopher-server.pid --wiki=/home/alex/farm/wiki.pl --wiki_dir=/home/alex/alexschroeder --wiki_pages=SiteMap --wiki_pages=About"
    stop program = "/bin/bash -c 'kill -s SIGTERM `cat /home/alex/alexschroeder-gopher-server.pid`'"
    if failed
	host alexschroeder.ch
	port 7070
	type tcp
        send "Alex\r\n"
	expect "#REDIRECT.*"
	for 5 cycles
	then restart
    if totalmem > 100 MB for 5 cycles then restart
    if 3 restarts within 15 cycles then timeout

The Net::Server is where you find stuff regarding all the command line options. --wiki, --wiki_pages and --wiki_dir are the ones I added.

Why gopher? Well...

“It’s actually rather nice to have a small ecosystem because no one’s running annoying ads in Gopherspace or trying to track your browsing habits,” he says. “The protocol makes the former hard and the latter almost impossible.” – Cameron Kaiser being quoted in The rise and fall of the Gopher protocol.
«The modern web is an ugly, massive, broken mess. This isn’t a secret or a new take or a controversial opinion. We stuff our webpages so full of tracking cookies and bitcoin mining advertisements and javascript apps that monitor your every move that it’s no wonder that modern web browsers are less effecient than ever.» — Gopher: Remembering the web that wasn't

This is nothing we can’t fix using HTML and HTTPS, but Gopher forces us to go without JavaScript and CSS and all that.

Tags:

Comments on 2017-12-27 Gopher Server

RFC 1436 remains relevant, I guess.

– Alex 2017-12-28 16:44 UTC


Work continues. I’ve added menus for page filtering, for search, for recent changes both major changes only and including minor changes.

Next up: page history.

Is there much else? I think soon enough the lingering question will be: how do we edit pages via gopher?

– Alex 2017-12-29 14:31 UTC


Writing unit tests makes me feel so much better.

– Alex 2017-12-30 04:56 UTC


On the Gopher web (what’s the name to use here, do we like the -verse suffix?), all the encoding problems of the early web are coming back. Latin 1 instead of UTF 8? No content negotiation? Not even encoding labeling? What are we going to do? Surely limiting ourselves to Latin 1 for menu texts is not good enough.

From RFC 1436: “It is highly recommended that the User_Name field contain only printable characters, since many different clients will be using it. However if eight bit characters are used, the characters should conform with the ISO Latin1 Character Set.”

It’s a different world out there. We need to talk about indicating encoding to clients.

Menu type m for metadata? e.g.

mEncoding\tUTF-8

I’ve been using a Gopher client on iOS for a bit and being German speaker, I soon stumbled over encoding issues. If I search Gopherpedia for „Zürich“, the client crashes. If I search for „Zurich“ and click on the first link, „Zürich“, it also crashes.

– Alex 2017-12-30 09:51 UTC

Add Comment

2017-12-15 Including Mastodon Comments

Steve Ivy suggested pulling in replies to a Mastodon status referring to a blog post, and suggested using the regular API for it, because displaying a status and its context doesn’t require authorization. Interesting idea!

But then again: Am I no longer going to expire these toots?

shower thought: wondering if I could automate posting links to my blog posts on mastodon, then pull in replies and boosts via *javascript* (like masto-discus)

@sivy yes you could! It’d be amazing, maybe? I think the Mastodon.py library already has everything. Perhaps we just need a sub command for the archive tool? I see it unfolding before my coding third eye 👁 …

@sivy Now I’m thinking crossposting would violate copyright and go against Mastodon user expectations. :(

@kensanata you mean pulling public replies out of mastodon for display on a website? possibly, not sure.

I still think something "realtime" in javascript (like discus) would be better... though not sure that solves the copyright problem?

@sivy Hm, that’s true. You could render the context Atom feed—if it exists. That would also make sure that Mastodon users retain control o er their toots. Deleting their toots would still have the intended effect.

@kensanata better, use the /api/v1/statuses/:id/context endpoint. It's unauthenticated and provides threaded (ish) responses:

toot.cafe/api/v1/statuses/9917

@kensanata of course, that means *writing javascript& which is the downside of all of this. >_<

Tags:

Add Comment

2017-11-14 CSS Naked Day

I should write an Oddmuse module for CSS Naked Day (April 9).

Try this site without CSS!

Get dressed again...

I wrote about this back in 2007 and never thought of it again. If I had a module, I’d get reminded of it, every April.

Today I was reminded of it because my browser at the office has 11 tabs with Wikipedia pages. I plugged in the laptop after yesterday’s meeting elsewhere, opened the browser, and the 11 tabs came back—but without styling! I had forgotten to plug in the networking cable and apparently Chrome caches the Wikipedia HTML but not the Wikipedia CSS.

Anyway, the code would be something like this:

$ModulesDescription .= '<p><a href="https://alexschroeder.ch/wiki/2017-11-14_CSS_Naked_Day">No CSS Day</a></p>';
*NoCssOldGetCss = \&GetCss;
*GetCss = \&NoCssNewGetCss;
sub NoCssNewGetCss {
  my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday) = gmtime($Now);
  # $mon: April is 3
  if ($mday == 9 and $mon == 3) {
    return '';
  } else {
    return NoCssOldGetCss(@_);
  }
}

Untested!

Also, I had never heard of the term “international day” before. The code on the CSS Naked day website has a funkier test than mine above and here’s why:

CSS Naked Day lasts for one international day. Technically speaking, it will be “April 9th” somewhere in the world for 48 hours. This is to ensure that everyone’s website will be publicly nude for the entire world to see at any given time during April 9. [1]

Tags:

Add Comment

2017-07-07 Markdown

I’ve been working on Markdown support for Oddmuse and added nested list, and mixed list support.

There are still issues, unfortunately.

1. item

-- signature

The signature looks like a second list item containing “- signature” because the space after the dash is not mandatory. Should it be? Or should two dashes simply be the exception? Or should the character after the dash be a word character based on Unicode properties?

Tags:

Add Comment

2017-01-25 Finding Encoding Bugs

When I look at one of my log files, I see the following:

alex@sibirocobombus:~/communitywiki$ head -n 3 ~/farm/communitywiki.log
[Sun Jan 22 08:59:11 2017] hypnotoad: utf8 "\xFD" does not map to Unicode at /home/alex/farm/wiki.pl line 3455, <$FILE> line 1.
[Sun Jan 22 08:59:11 2017] hypnotoad: utf8 "\xFD" does not map to Unicode at /home/alex/farm/wiki.pl line 3455, <$FILE> line 1.
[Sun Jan 22 08:59:11 2017] hypnotoad: utf8 "\xFD" does not map to Unicode at /home/alex/farm/wiki.pl line 3455, <$FILE> line 1.

What could be the problem? Let’s find the pages containing this byte.

alex@sibirocobombus:~/communitywiki$ grep -l -P '\xFD' page/*.pg
page/MultilingualExperiment.pg
page/ProjectSpaceMultiLingual.pg

Can we reproduce the problem? Apparently, simply opening the file is not a problem. Those sequences appear to be valid UTF-8.

alex@sibirocobombus:~/communitywiki$ WikiDataDir=/home/alex/communitywiki perl -e 'package OddMuse; $RunCGI=0; do "/home/alex/src/oddmuse/wiki.pl"; Init(); ReadIndex(); OpenPage("MultilingualExperiment"); print "$Page{text}\n";'>/dev/null
alex@sibirocobombus:~/communitywiki$ WikiDataDir=/home/alex/communitywiki perl -e 'package OddMuse; $RunCGI=0; do "/home/alex/src/oddmuse/wiki.pl"; Init(); ReadIndex(); OpenPage("ProjectSpaceMultiLingual"); print "$Page{text}\n";'>/dev/null

How about simply opening all the files?

alex@sibirocobombus:~/communitywiki$ WikiDataDir=/home/alex/communitywiki perl -e 'package OddMuse; $RunCGI=0; do "/home/alex/src/oddmuse/wiki.pl"; Init(); ReadIndex(); OpenPage("ProjectSpaceMultiLingual"); for (@IndexList) { OpenPage($_); print "$OpenPageName\n" };'
...
2007-01-06
2007-01-09_NewsCwb
[Wed Jan 25 14:49:21 2017] -e: utf8 "\xFD" does not map to Unicode at /home/alex/src/oddmuse/wiki.pl line 2845.
[Wed Jan 25 14:49:21 2017] -e: utf8 "\xFD" does not map to Unicode at /home/alex/src/oddmuse/wiki.pl line 2845.
[Wed Jan 25 14:49:21 2017] -e: utf8 "\xFD" does not map to Unicode at /home/alex/src/oddmuse/wiki.pl line 2845.
2007-01-13
2007-01-27_HwoToDo
...
RSS_3.0
rssanchor
RssExclude
[Wed Jan 25 14:49:26 2017] -e: utf8 "\xE8" does not map to Unicode at /home/alex/src/oddmuse/wiki.pl line 2845.
[Wed Jan 25 14:49:26 2017] -e: utf8 "\xE8" does not map to Unicode at /home/alex/src/oddmuse/wiki.pl line 2845.
[Wed Jan 25 14:49:26 2017] -e: utf8 "\xE8" does not map to Unicode at /home/alex/src/oddmuse/wiki.pl line 2845.
RssInterwikiTranslate
RuleOfOrder
...

Interesting! And less finds these:

alex@sibirocobombus:~/communitywiki$ less page/2007-01-13.pg
...
diff-minor: <p><strong>Changed:</strong></p>
        <div class="old"><p>&lt; Well, I'll be donating another $75 this year, and hoping that my sites get moved to the  new machine they'll be buying soon. ;) My previous experience with commercial hosting has been less than satisfactory (at $20 per month), so I'd only make such a move if a group of people convinced me. There's shell access, cron jobs, Perl modules <strong class="changes"><FD><FD><FD></strong> a lengthy list of requirements I have personally...</p></div><p><strong>to</strong></p>
...

Thus, I edited 2007-01-13 and RssInterwikiTranslate, removing anything that looked weird in a major edit and from now on I hope to no longer see these warnings.

Alternatively, consider this little script written by CapnDan on the #oddmuse channel, Freenode:

#!/usr/bin/env perl
use utf8 ;
foreach (@ARGV) {
  my $file = $_ ;
  print "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n--- $file ---\n";
  utf8::encode($file); # filenames are bytes!
  if (open(my $IN, '<:encoding(UTF-8)', $file)) {
    local $/ = undef; # Read complete files
    my $data=<$IN>;
    close $IN;
  };
};

Tags:

Add Comment

2016-06-15 Oddmuse and Windows

Today I ran the Oddmuse test suite on Windows and it worked! I am amazed.

All tests successful.
Files=136, Tests=2907, 1028 wallclock secs ( 0.41 usr  0.45 sys + 211.30 cusr 219.98 csys = 432.14 CPU)
Result: PASS

I had already installed Cygwin and had used it to install Perl v5.22.2. I don’t think I ever ran cpan on this system because I had to set it up. It created $USERPROFILE/AppData/Roaming/perl5/ for me and set up local::lib.

And then I started installing.

I needed some libraries and a compiler which I installed via Cygwin. I already had the diff utils installed, but perhaps you never needed them. Oddmuse needs them.

diffutils
gcc-g++
libcrypt-devel
libexpat-devel
libxml2-devel
libgd-devel
libdb-devel

Then I started by installing cpanm.

cpan App::cpanminus

I had to add /usr/local/bin to my PATH.

cpanm CGI XML::Atom IO::Pipely Mojolicious Mojolicious::Plugin::CGI Toadfarm HTTP::Server::Simple Pod::Strip Crypt::Rijndael Crypt::Random::Seed RPC::XML HTML::Template Capture::Tiny MLDBM Captcha::reCAPTCHA DB_File XML::LibXML GD Time::ParseDate

And with that, I could finally run prove t. I’ll note, however, that make test caused t/big-brother.t to fail, perhaps because it runs with 4 jobs which overwhelmed my machine.

Tags:

Comments on 2016-06-15 Oddmuse and Windows

Perhaps add that info here: https://oddmuse.org/wiki/Unit_Testing ?

– AlexDaniel 2016-06-17 20:18 UTC


Good idea.

– Alex Schroeder 2016-06-17 21:00 UTC

Add Comment

2015-10-09 Oddmuse and Mojolicious

I’m trying to run Oddmuse within a Perl web framework: Mojolicious using Mojolicious::Plugin::CGI. It won’t be as perfect as a true Mojolicious app, but it will still be much faster than a simple CGI script. When running as a CGI script, every request loads Perl and compiles all the modules – including the CGI module itself – and Oddmuse and the config files. When running under Mojolicious, we no longer load Perl and we compile Oddmuse just once. Oddmuse itself will keep loading the config file and all that, but it’s still much better than before. The Mojolicious app itself is then started by Toadfarm. And in order to force myself to test it, I’ve switched this wiki over to the new setup!

First, tell Apache to act as a reverse proxy and pass all /wiki requests to the new server. The two ProxyPass instructions at the bottom are the important bits:

<VirtualHost *:80>
    ServerName alexschroeder.ch
    ServerAlias www.alexschroeder.ch
    Redirect permanent / https://alexschroeder.ch/
</VirtualHost>
<VirtualHost *:443>
    ServerAdmin alex@alexschroeder.ch
    <Directory />
        Options None
        AllowOverride None
        Order Deny,Allow
        Deny from all
    </Directory>
    ServerName alexschroeder.ch
    ServerAlias www.alexschroeder.ch
    ServerAlias rpg.alexschroeder.ch
    DocumentRoot /home/alex/alexschroeder.ch
    <Directory /home/alex/alexschroeder.ch>
        Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
        AddHandler cgi-script .pl
        AllowOverride All
        Order Allow,Deny
        Allow from all
    </Directory>

    SSLEngine on
    SSLCertificateFile      /home/alex/ssl/alexschroeder.crt
    SSLCertificateKeyFile   /home/alex/ssl/alexschroeder.key
    SSLCertificateChainFile /home/alex/ssl/GandiStandardSSLCA2.pem
    SSLVerifyClient None

    ProxyPass /wiki             https://alexschroeder.ch:8080/wiki
    ProxyPass /mojo             https://alexschroeder.ch:8080/mojo

</VirtualHost>

Toadfarm setup in ~/farm/farm:

#!/usr/bin/env perl
use Toadfarm -init;

logging {
  combined => 1,
  file     => "farm.log",
  level    => "info",
};

1. other apps go here...

mount "$farm/alexschroeder.pl" => {
  "Host" => qr{^(www.)?alexschroeder\.ch$},
  mount_point => '/wiki',
};

plugin "Toadfarm::Plugin::AccessLog";

start; # needs to be at the last line

The above means that all requests to htpp://alexschroeder.ch:8080/wiki and htpp://www.alexschroeder.ch:8080/wiki will be handled by Oddmuse.

Mojolicious wrapper in ~/farm/alexschroeder.pl:

#! /usr/bin/env perl

use Mojolicious::Lite;

plugin CGI => {
  support_semicolon_in_query_string => 1,
};

plugin CGI => {
  route => '/',
  script => 'wiki.pl', # ~/farm/wiki.pl
  env => {WikiDataDir => '/home/alex/alexschroeder'},
  errlog => 'alexschroeder.log', # path to where STDERR from cgi script goes
};

app->start;

In this case, wiki.pl is simply the original Oddmuse script, unchanged. :)

Tags:

Comments on 2015-10-09 Oddmuse and Mojolicious

Cool! That’s interesting.

What about side effects? Like, if you forget to initialize some variable somewhere, what is going to happen? (Thinking about mod_perl and all stuff related to it)

Also, I miss one important thing from your report... What about page load time? How much faster is it right now, comparing to regular cgi script?

– AlexDaniel 2015-10-09 11:21 UTC


I did not measure page load time. I also think that it keeps forking to run the script, so globals start empty every time? Not sure about the details. I haven’t seen any side effects, yet. Let me know if you see anything!

– Alex Schroeder 2015-10-09 11:39 UTC


Could you be so kind to measure the load time?

– AlexDaniel 2015-10-09 11:50 UTC


I think I was under the illusion that it was working. Now it is working. Hah! I had to rename my old wrapper script to realize that my ProxyPass instructions had been wrong.

Things to remember:

Check the config file for weird stuff.

1. if ($ENV{SERVER_PORT} == 8080) {
1.   $ScriptName  = 'https://alexschroeder.ch:8080/wiki';
1.   $FullUrl     = 'https://alexschroeder.ch:8080/wiki';
1. }

Make sure the user running toadfarm can write alle the files in your data directory. In my case, the data directory belongs to www-data.alex but there were temp files that alex could not write.

– Alex Schroeder 2015-10-09 13:27 UTC


Hm. Current setup, I get more or less the same time when I look at network analysis with my browser when I’m loading /wiki/About and when I’m loading /wiki2/About with wiki2.pl being my old wiki.pl wrapper script. I get numbers between 200ms and 700ms. I do note that /wiki/About keeps getting me a 200 OK response instead of a 304 NOT MODIFIED. I’m not quite sure what the problem is.

– Alex Schroeder 2015-10-09 13:40 UTC


Oh, and the code shows that our script is being exec’d. I’m not sure about the headers. I think if our script prints HTTP headers, we should be fine.

– Alex Schroeder 2015-10-09 13:47 UTC


Well, at least it seems to work right now. I think I got confused about the various restarts required.

The missing 304 NOT MODIFIED remains a mistery, however. I’ve tried $q = new CGI; $q->nph(1) in the config file, $CGI::NPH = 1 in the config file, CGI::nph(1) in the config file, use CGI qw/-utf8 -nph/ in the core script, and print $q->header(-nph=>1, -status=>'304 NOT MODIFIED') and return if PageFresh(); in the core script, reloading my toadfarm between edits and found no change.

When I tried to export MOJO_PLUGIN_CGI_DEBUG=1 before starting the toadfarm, I got a 503 error when visiting the site.

This is very annoying.

– AlexSchroeder 2015-10-09 14:32 UTC


OK, so the question is: How often is 304 NOT MODIFIED actually used?

alex@kallobombus:~/farm$ sudo wc -l /var/log/apache2/access.log.1
259852 /var/log/apache2/access.log.1
alex@kallobombus:~/farm$ sudo grep ^oddmuse /var/log/apache2/access.log.1 | wc -l
32867
alex@kallobombus:~/farm$ sudo grep "^oddmuse.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l
270

Filtering out feed request we’re down to 8‰.

For my own site:

alex@kallobombus:~/farm$ sudo grep ^alexschroeder /var/log/apache2/access.log.1 | wc -l
79286
alex@kallobombus:~/farm$ sudo grep "^alexschroeder.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l
702

This reduces the percentage to 9‰.

So, if we redirect feed requests to files and use a cron job to produce some of them, the 304 NOT MODIFIED response seems to be something we can do without, even if I think it ought to work.

– AlexSchroeder 2015-10-09 14:54 UTC


In other words, no noticeable performance boost? Do I read it right? But then why bother?

– AlexDaniel 2015-10-09 21:26 UTC


Hehe, indeed. The point is, however, that I want some sort of plan that will allow me to incrementally develop a solution. Mojolicious::Plugin::CGI does a lot of what I want. It integrates into Mojolicious and thus it can do logging, forking, and so on. It handles input and output. The only thing that’s missing is that it exec’s the script – loading Perl and compiling all the modules on every request.

I think I’ll try and modify the Plugin for my purposes. Then it’ll require Oddmuse once and just run DoWikiRequest on every request, mod_perl style.

– AlexSchroeder 2015-10-10 07:40 UTC


These are the changes I had to make to Mojolicious::Plugin::CGI: Add run option for a code reference and Make sure the status code is actually used and Add HTTP_IF_NONE_MATCH to environment.

This is the new server script I used:

#! /usr/bin/env perl

use lib '/Users/alex/src/mojolicious-plugin-cgi/lib';

use Mojolicious::Lite;

plugin CGI => {
  support_semicolon_in_query_string => 1,
};
 
plugin CGI => {
  route => '/wiki',
  script => 'wiki.pl',
  run => \&OddMuse::DoWikiRequest,
  before => sub {
    $OddMuse::RunCGI = 0;
    $OddMuse::DataDir = '/tmp/oddmuse';
    require 'wiki.pl' unless defined &OddMuse::DoWikiRequest;
  },
  env => {},
  errlog => 'wiki.log', # path to where STDERR from cgi script goes
};

get '/' => sub {
  my $self = shift;
  $self->redirect_to('/wiki');
};
 
app->start;

– AlexSchroeder 2015-10-11 06:51 UTC


With the above setup now installed on alexschroeder.ch, I’m getting response times of 150–230ms when looking at /wiki/About (and a 304 Not Modified response). The old CGI setup under /wiki2/About is giving response times of 250–720ms.

So, much better?

I’ve also started noticing the first signs of trouble: Edit page, save, look at it (looks good), edit page again and notice that it’s showing the old text, not the new text.

– AlexSchroeder 2015-10-11 10:28 UTC


When I tried it with this page, I saw my browser requesting the page using the following:

GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1
...
If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com

Oddmuse replied with a 304 Not Modified and the content was wrong. As I checked, however, the timestamp of the index file does in fact match:

alex@kallobombus:~/alexschroeder$ date -r pageidx +%s
1444395217

Looking at the log of recent changes, however:

alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log
1444559479Comments_on_2015-10-09_Oddmuse_and_Mojolicious1With the above setup now installed on alexschroeder.ch, I'm getting response times of 150–230ms when looking at wikiAbout (and a 304 Not Modified response). So, much better? 127.0.0.1AlexSchroeder23en

I guess something about TouchIndexFile isn’t working.

sub TouchIndexFile {
  my $ts = time;
  utime $ts, $ts, $IndexFile;
  $LastUpdate = $Now = $ts;
}

I’ve made sure that all the files belong to alex.alex instead of www-data.alex. And that seems to help. After saving:

alex@kallobombus:~/alexschroeder$ date -r pageidx +%s
1444560771
alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log
1444560771Comments_on_2015-10-09_Oddmuse_and_MojoliciousWhen I tried it with this page, I saw my browser requesting the page using the following: {{{ GET wiki?action=edit;id=Commentson2015-10-09OddmuseandMojolicious HTTP1.1 … If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com }}} Oddmuse replied with a 304 Not Modified and the content…127.0.0.1AlexSchroeder24en

And the next request:

GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1
...
If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com

HTTP/1.1 200 OK
...
Etag: 1444560771%1e1%1eAlexSchroeder%1ekensanata%40gmail.com
...

Yay?

– AlexSchroeder 2015-10-11 10:52 UTC


OK, added one final patch to make it work without having to use the -nph option for the CGI library and submitted a pull request. And I installed it for this site as well. “Eat your own dog food,” and all that.

– AlexSchroeder 2015-10-11 12:41 UTC


Wow, many more commits after talking to the maintainer.

– AlexSchroeder 2015-10-11 16:14 UTC

Add Comment

2015-06-14 Go Oddmuse

Sometimes I wonder whether I should move Oddmuse away from Perl 5 to something else. Something I would like to learn. An opportunity to redo everything, from scratch. Not many people are using Oddmuse, so I might as well write something else, for myself.

I recently read Writing Web Applications for Go, where they develop a simple wiki. I wondered about storing the data in a git repository. I found git2go. Hm...

Tags:

Comments on 2015-06-14 Go Oddmuse

Perl 6 is an obvious candidate. And yes, they did not just change print ’x’ to print(’x’) like they did in Python, it is a completely new language that you have to learn.

Speaking of Go, there is a great video called Perl 6 for Mere Mortals, and here is a direct link to a part of it which is somehow :) related to Go. That’s not a meaningful comparison, but it should give you the idea. You can also watch that video from the beginning, it is very interesting if you have not investigated into Perl 6 yet.

Now, if we think about all of the languages out there, what are the actual features that are required for Oddmuse?

Well, since Oddmuse is all about parsing the wiki text, you need some tools for parsing. Perl 5 has regexes (um, okay), Perl 6 has grammars (exactly what we need!), and Go has something as well (like this, which is, again, verbose as hell), but I’m not an expert. You can also use libs, but having a built-in support is probably advantageous.

Somehow I can’t really think of any other criteria, everything else probably exists in every other language out there. You might also say “performance” but the only website suffering from that is probably emacswiki, and it all depends on your actual goals – for example, better execution time can be achieved by using concurrency, which does not really lead to better CPU time. Concurrency in Perl 6 – yes (but maybe just a bit flaky at the moment), performance in Perl 6 – well, not now, but the potential is there (i.e. gradual typing).

In other words: what is the motivation? “Just to learn” is not going to get you anywhere, not even make you learn something (learning to find reasons for doing stuff (i.e. causation) is part of the learning, doing stuff purposelessly is harmful to the thinking process).

AlexDaniel 2015-06-14 14:05 UTC


Perhaps Go just makes me want to delve into the low level stuff. I like the post about Handwritten Parsers & Lexers in Go. :)

I keep thinking about all the design decisions. Filenames are pagenames. Namespaces are subdirectories. Can we switch to “git first”? No more log files. We no longer need keep files. How would modules work? Would you simply recompile the wiki? Dynamic linking? Would I want to use a templating architecture? People seem to like that. Was “printing as we go” a good decision? Perhaps it no longer matters much.

I need to think about extensibility of Cajun, the Wiki Creole lexer and parser.

– Alex Schroeder


These are very good questions. Maybe you should write down these ideas on Oddmuse: Revolutionary Changes.

  • “git first” – Yea, but... This would effectively break the goal of having one script file that just works (and I think that this is more important than having no recent changes and keep files logic in the core).
  • “How would modules work? Would you simply recompile the wiki?” – this would effectively throw away all of the nice things like Module Updater or Module Bisect (or at least make them much harder to implement). It will also break the potential of a few other ideas that keep floating in my head. In other words, “simply recompile the wiki” is not that simple.
  • “Would I want to use a templating architecture? People seem to like that.” Yea, but... It seems like if we switch to Perl6 then the core will collapse into a small piece of code with all of the irrelevant stuff separated into grammars or other classes, or maybe just tiny subroutines. Which brings us to the next point...
  • “Was “printing as we go” a good decision? Perhaps it no longer matters much.” – Good decision, but it really depends on your understanding of “as we go”. For example, what if we parse the wiki text first, before printing anything except the header? Then, once we start processing the parsed text, we can print stuff as we go. That’s exactly how it will work in perl6 (at least, that’s the easiest way) – first you slam the wiki text into your grammar and wait for it to be processed (oh by the way, while this is happening you can actually do other stuff asynchronously, like start reading the tag index or whatever). Then, when your wikitext is parsed, you can start deciding what to print now. And this is so much better than what we have now. Want to print table of contents? Just take the required information from the match object, no need for dirty hacks. In other words, in perl6 it is very easy to have parsing and actions separately, which is beneficial in many ways. But at the same time, once you start processing the actions you can start printing as you go (which makes sense, what if there is some complex search stuff included on the page, you’d probably want to see at least half of the page while this is going).
  • - AlexDaniel 2015-06-16 00:22 UTC

And yeah, if you are seriously thinking about Go, what if we make a comparison table with Perl6 and Go, and the relevant language features? Both of the languages have some interesting stuff to offer, for example Perl6 to JS and Go to JS. And there are definitely some winning points for Go, especially associated with execution time and maturity.

AlexDaniel 2015-06-16 00:31 UTC


Hehe, I think you and Oddmuse: Revolutionary Changes is what started this all! I looked at Oddmuse as a “mission accomplished” project. With the migration of Emacs Wiki I’ve realized that a big site might want to serve its own pages without running a web server. I tried to understand PSGI/Plack in order to have Perl run Oddmuse like a web application. When I saw the wiki tutorial for Go, and how I got a web server up and running in no time, and how easy it was to include some basic markup, I was thrilled! Sure, the code doesn’t have a lot of features. But it beats the SmallestWiki candidates. :)

You arguments in favor of some of the core architecture decisions made for Oddmuse are good – and your hopes for Perl 6 and improved parsing are infectious! I’m very much looking forward to Perl 6.

– Alex Schroeder 2015-06-16 06:02 UTC


It’s also cool to look at the TinyWiki source code again and rediscover why I rewrote the Usemod Wiki style search and replace code with the extensible state machine code we have right now.

– Alex Schroeder 2015-06-16 08:00 UTC

Add Comment

More...

Comments


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. See Info for text formatting rules. You can edit the comment page if you need to fix typos. You can subscribe to new comments by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.