Oddmuse 6

Oddmuse 6 is my first application written in Perl 6. It’s a rewrite of “regular” Oddmuse. Eventually, I’m hoping to replace regular Oddmuse with Oddmuse 6, but I think this will be a long time in the making. For now, it’s just my little side project.

2018-10-08 User Interface

Thinking about adding extra functionality for administrators to Oddmuse 6. But how to do that without cluttering the interface? I really like how right now the bottom of the page is just two links:

Edit History

Compared with the above, adding a link for something most people will never click is a bit jarring.

Edit History Administration

Now what?

Or perhaps we can “hide” this on Changes and History: at the top there could be a link to lock the site, or the page. That will require you to log in if you aren’t already, and so on.

This might actually work.

Tags:

Add Comment

2018-10-08 Setting up Oddmuse 6

Here’s what I needed to do in order to get the Oddmuse 6 wiki up an running.

I downloaded and compiled the latest Rakudo Star sources (2018.06).

Install cro and Oddmuse 6:

zef install --/test cro
zef install Oddmuse6

🔥 When installing dependencies using zef as shown, you could be running into an OpenSSL issue even if you have the correct development libraries installed. On Debian, you need libssl-dev but apparently versions 1.1.0f and 1.1.0g won’t work. See issue #34. You could decide to ignore SSL support and opt to have a web server act as a proxy which provides SSL. That’s what I intend to do. In which case there is a terrible workaround available: run zef install --force-test IO::Socket::Async::SSL before you zef install Oddmuse6.

Create a new application. Remember that installing cro printed a message telling you where the binary got installed. I’m assuming you added $HOME/rakudo/share/perl6/site/bin to your PATH.

Start with a stub and accept all the defaults:

cro stub http oddmuse6 oddmuse6

Now edit oddmuse6/services.p6 and replace use Routes with use Oddmuse::Routes.

You can delete the oddmuse6/lib/Routes.pm6 which cro stub generated for you.

Your default wiki directory is oddmuse6/wiki, so we need to tell cro to ignore it. If you don’t, you’ll confuse cro to no end as soon as you start editing files! Add the following section section to your in oddmuse6/.cro.yml file:

ignore:
  - wiki/

Run it:

cd oddmuse6
cro run

Check it out by visiting http://localhost:20000. Your wiki is ready! 🙃

But this is not enough. I needed to get my own domain. My name service provider for oddmuse.org is Gandi so I logged in, found the entry and added a new subdomain called next pointing at the same IP, basically a copy of the www subdomain.

Next, I tell Apache about it. I’m using Dehydrated to get my oddmuse.org certificates from Let’s Encrypt and I’m just going to reuse them.

In the file /etc/dehydrated/domains.txt I added the new next.oddmuse.org domain:

oddmuse.org www.oddmuse.org next.oddmuse.org

Then I ran /usr/bin/dehydrated -c as root to check and possibly renew all my certificates.

I’m using Apache as my webserver. I edited /etc/apache2/sites-enabled/500-oddmuse.org.conf and added two new sections for next.oddmuse.org.

The first redirects HTTP traffic to HTTPS:

<VirtualHost *:80>
    ServerName next.oddmuse.org
    Redirect permanent / https://next.oddmuse.org/
</VirtualHost>

The second redirects all HTTPS traffic it receives back to HTTP on port 20000, which is where Oddmuse 6 is listening.

<VirtualHost *:443>
    ServerAdmin alex@oddmuse.org
    ServerName next.oddmuse.org

    RewriteEngine on
    RewriteCond "%{HTTP_USER_AGENT}" "Mastodon"
    RewriteRule ".*" "-" [redirect=403,last]

    SSLEngine on
    SSLCertificateFile      /var/lib/dehydrated/certs/oddmuse.org/cert.pem
    SSLCertificateKeyFile   /var/lib/dehydrated/certs/oddmuse.org/privkey.pem
    SSLCertificateChainFile /var/lib/dehydrated/certs/oddmuse.org/chain.pem
    SSLVerifyClient None

    ProxyPass /             http://next.oddmuse.org:20000/

</VirtualHost>

Thus, all connections to the wiki are encrypted. What about people connecting to port 20000 directly? Well, I have a file called /etc/apache2/conf-enabled/security.conf which has a few settings including this one:

# Setting this header will prevent access via HTTP.
# Requires mod_headers to be enabled.
# See https://tools.ietf.org/html/draft-ietf-websec-strict-transport-sec-14#section-6.1.2
# The number of seconds is the equivalent of one year.
#
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"

The rewrite rule for Mastodon is a simple defence against the spikes I get when I share links to my sites. In a second, all the servers seeing the link attempt to fetch a preview image and in order to avoid overloads, I just disallow these.

So now, finally, requests end up as simple HTTP on 20000. Unfortunately, the wiki is listening on localhost, not next.oddmuse.org.

So I wrote a little wrapper script to set all the options, write a PID file and all that. I called it ~/bin/oddmuse6.

#!/bin/bash
pidfile=$HOME/oddmuse6/nohup.pid
if test -f $pidfile; then
    echo Killing $(cat $pidfile)
    kill -SIGTERM $(cat $pidfile)
fi

export ODDMUSE_MENU="Home, Changes, About"
export ODDMUSE_QUESTION="Name a colour of the rainbow."
export ODDMUSE_ANSWER="red, orange, yellow, green, blue, indigo, violet"
export ODDMUSE_SECRET="rainbow-unicorn"
export ODDMUSE_HOST="next.oddmuse.org"
export ODDMUSE_PORT=20000

cd $HOME/oddmuse6
rm nohup.out
nohup /home/alex/rakudo/bin/perl6 service.p6 &
tail nohup.out
echo $! > $pidfile

But that still isn’t enough because ODDMUSE_HOST and ODDMUSE_PORT actually don’t match the environment variables used in oddmuse6/service.p6. Each cro service gets to environment variables that determine its host and its port. Their names depend on the name you provided when you called cro stub. If you called it oddmuse6 like I did in the example above, the two environment variables you need are called ODDMUSE6_HOST and ODDMUSE6_PORT. I just changed them to ODDMUSE_HOST and ODDMUSE_PORT in service.p6.

As you can see from the shell script, I’m not even using cro run to run my wiki. I’m using perl6 directly. I’ll have to think about that. I suspect there are benefits of using cro run when you use multiple applications. At the moment I don’t, so perl6 it is.

Since I’m not using cro run you could delete all these other files, actually. You really just need service.p6 and the wiki subdirectory.

The content of service.p6 is simple:

use Cro::HTTP::Log::File;
use Cro::HTTP::Server;
use Oddmuse::Routes;

my Cro::Service $http = Cro::HTTP::Server.new(
    http => <1.1>,
    host => %*ENV<ODDMUSE_HOST> ||
        die("Missing ODDMUSE_HOST in environment"),
    port => %*ENV<ODDMUSE_PORT> ||
        die("Missing ODDMUSE_PORT in environment"),
    application => routes(),
    after => [
        Cro::HTTP::Log::File.new(logs => $*OUT, errors => $*ERR)
    ]
);
$http.start;
say "Listening at http://%*ENV<ODDMUSE_HOST>:%*ENV<ODDMUSE_PORT>";
react {
    whenever signal(SIGINT) {
        say "Shutting down...";
        $http.stop;
        done;
    }
}

Check it out: Oddmuse 6.

Tags:

Add Comment

2018-10-06 Next Steps for Oddmuse 6

I recently added question-asking and rollbacks to Oddmuse 6. This takes it closer to being usable on the wild Internet.

Here’s what it still needs before it can deployed on the Intranet:

  • a link to an administration page
  • a way to enter your administrator password
  • a way for administrators to lock the site for non-administrators
  • a way for administrators to lock a page for non-administrators
  • a way to list all the locked pages
  • a protected page where administrators can put banned phrases.
  • an error when people try to save pages containing banned phrases
  • a way to list all the pages containing banned phrases

Here’s what I’d like to see:

  • preview button
  • cancel button
  • full text search
  • page title match
  • page tagging
  • tag search
  • tag cloud

Interesting next steps:

  • namespaces
  • comments
  • file uploads

Nice to have:

  • a SQLite backend, once the storage API is stable
  • an extension to provide an example of how this could be achieved

Anyway, I’ve been following the instructions on distributing modules and I’ve received my PAUSE account info, and so uploaded what I have to CPAN!

And I have a public install: Next Oddmuse. Write-up to follow.

Tags:

Comments on 2018-10-06 Next Steps for Oddmuse 6

The website doesn’t look good on my phone: checkboxes and buttons are minuscule.

– Alex Schroeder 2018-10-07 21:35 UTC

Add Comment

2018-09-20 Perl 6 Peer Review?

Does anybody feel like looking at my code and giving me some feedback? My main problem as a lone programmer is that I never know whether I’m doing it idiomatically. There’s no peer review when I’m working on my private projects away from the office.

Sometimes there’s no peer review when I’m working at the office, too. :)

So, I’m guessing there are quite a few areas where I could use some help:

  • Better ways to write things using Perl 6
  • Better ways to write things using Cro
  • Better ways to write tests
  • Better ways to organize code
  • Better ways to document code
  • Better installation instructions
  • How to make this distributable via CPAN

Code:

git clone https://alexschroeder.ch/cgit/oddmuse6

Tags:

Comments on 2018-09-20 Perl 6 Peer Review?

I made another reorganisation of the source code, trying to follow the instructions about distributing modules and thinking that the ultimate test would be to make a local installation. I somehow had the idea that this should work using zef:

zef install ./Oddmuse

But sadly, it doesn’t:

zef install ./Oddmuse
===> Testing: Oddmuse6:ver<0.0.1>:auth<kensanata>
===> Testing [OK] for Oddmuse6:ver<0.0.1>:auth<kensanata>
===> Installing: Oddmuse6:ver<0.0.1>:auth<kensanata>
===> Install [FAIL] for Oddmuse6:ver<0.0.1>:auth<kensanata>: ===SORRY!===
Could not find Oddmuse::View at line 18 in:
    /home/alex/rakudo/install/share/perl6/site
    /home/alex/rakudo/install/share/perl6/vendor
    /home/alex/rakudo/install/share/perl6
    CompUnit::Repository::AbsolutePath<94588950434432>
    CompUnit::Repository::NQP<94588953675624>
    CompUnit::Repository::Perl5<94588953675664>

===SORRY!===
Could not find Oddmuse::View at line 18 in:
    /home/alex/rakudo/install/share/perl6/site
    /home/alex/rakudo/install/share/perl6/vendor
    /home/alex/rakudo/install/share/perl6
    CompUnit::Repository::AbsolutePath<94588950434432>
    CompUnit::Repository::NQP<94588953675624>
    CompUnit::Repository::Perl5<94588953675664>

Now what?

– Alex Schroeder 2018-09-20 15:02 UTC


On #perl6 I was given the solution: my META6.json was wrong.

  1. I had a typo in the one file I had in the provides section
  2. I was missing all the other files in the provides section

I definitely will need a tool to check this!

– Alex Schroeder 2018-09-20 18:23 UTC


I kept working on that README file and I think it’s looking good! Now I’ll wait for the confirmation mail from PAUSE so that I can upload it to CPAN. At least that’s the plan. And adding more features, of course. ☺

– Alex Schroeder 2018-09-20 19:21 UTC


Heh. And I found the offline Cro documentation in the form of Markdown files deep down in the ~/.zef directory. That’s not as integrated as I would have liked but it works for me. And who knows, perhaps I’ll find the tool that allows me to browse it all.

– Alex Schroeder 2018-09-20 21:51 UTC


All these days have gone by and I didn’t get a reply from CPAN. No account created, I guess.

– Alex Schroeder 2018-09-26 21:46 UTC

Add Comment

2018-09-18 Thoughts on using Perl 6

So, I’ve used Perl 6 for a few days. I’m trying to write a wiki, because that’s what I like to do, and that also what I use every day. I’ve been maintaining and running Oddmuse since 2003. I know what I want. I’ve been down every dead end, I sometimes feel. Now I want to learn the language using a real project and that’s what it is.

The current state, from a programming perspective:

  • It uses Cro as the web framework, meaning it can run stand-alone, without Apache or nginx. It’s no longer a CGI script. I expect that most people will still use a regular web server as the front end for all their applications because of caching, logging, encryption, access control and all that.
  • It uses Markdown to generate the HTML. I’ve started using Markdown more and more. I no longer recommend Wiki Creole.
  • It uses Mustache templates to generate the layout and everything around the actual wiki page “content.” No more printing as we go in order to appear faster.
  • The storage backend is a class that delegates everything to the actual storage class. Right now I’ve only implemented file-based storage, but if I keep all access going through this storage class then that will make replacing it with git or a SQLite a breeze.
  • For the moment, configuration happens via environment variables: which storage backend to use, which data directory to use if you use files as your backend, what links to show at the top, it’s all environment variables.
  • This means that translating the interface means setting a bunch of environment variables and editing the templates.
  • There are tests!
  • File locking when writing files (for the file storage backend).
  • Keeping (numbered) backups of page files.
  • Random test directories for parallel testing

The software, from a user perspective:

  • View pages written in Markdown and rendered as HTML.
  • Edit pages as Markdown.
  • Fields available: text, summary, author, and a checkbox for minor changes
  • List recent changes.
  • A rudimentary filter for recent is available as a proof of concept: it allows you to specify how many items to see.
  • IP addresses are transformed into four octets and shown as little flag icons if no author was provided. This makes it likely that you can detect who edited a bunch of pages without revealing their IP number of host name.
  • Old page revisions can be examined.
  • Showing the latest change for every page on Recent Changes
  • Diff with old page revisions

Missing features:

  • unified layout
  • merge in case of conflicts
  • expiration of old page revisions
  • admin passwords
  • banning regular expressions
  • locking the site
  • locking pages
  • question asking for first time posters
  • rolling back changes
  • RSS feed
  • tags
  • search
  • comments

Missing big ideas:

  • a plugin system
  • markup rules which users can add, ideally by merging grammars (this enables us to have rules that protect text from further processing, such as code blocks; this requires the notion of rule precedence)
  • a backend implementation using SQLite

More to come! I’m running out of time and I have to go. Here’s an idea: draft pages. They are like real pages but they don’t show up in search or feeds. Hah!

Anyway, I was talking to @tyil and thought that I should write this stuff down. Not sure who the intended audience is for these things. I feel like I’m too confused right now to understand where the problem is to be found, how can I create an issue, and where, and with what suggestion? Perhaps my future self is the audience for this blog post. One day I’ll wonder how newbies get into Perl 6 and what their issues are and I’ll be able to come back to this blog post and see myself struggling.

So, for a start, these are the things that immediately jumped out at me:

  • missing integration of Perl 6 docs into the man system
  • missing a good Cro manual for offline use (I brought the laptop along on a trip)
  • missing good search for when I know what I want but don’t know the signal, twiggils and operators

I bought Perl 6 Deep Dive by Andrew Shitov and I’m surprised at how quickly I’m running out of information in the book. It was a good read to get started and to motivate myself. Good enough to actually pull me into starting this project. That’s cool. But the next point is the sort of documentation I’ve had trouble finding.

For example:

I’m trying to have a class called Storage which delegates all the important method calls to another class which I want to load at run time. In this case, it’s going to be Storage::File. How to do it? I searched for “delegation” and the like and ended up on documents that were deprecated. I figured it would have to involve require somewhere, and then something like that “synopsis” or whatever it was. I don’t really care about the “synopsis” and “exegesis” and all the other funny ways the language was developed now that I’m actually trying to get stuff done. I’m a different kind of user, I know – but this is the kind of user that we will have to pull in. In the end I asked on the #perl6 channel on IRC and found something that works:

class Storage {
    my $class = %*ENV<storage> || 'Storage::File';
    require ::($class);
    has $!delegate handles <get-page put-page get-template put-change get-changes> = ::($class).new;
}

This illustrates two points: I might have found out about “handles” but I didn’t find it in the book. The index doesn’t mention “delegation”. Apparently this is called a “trait”. The index doesn’t mention “traits”. But even knowing all these keywords, I can’t find out about these things by starting with p6doc objects.

And that brings me to another topic: p6doc.

  • I didn’t realize that was a separate thing that I would have to install.
  • When you run p6doc --help, it tells you stuff about p6doc build and p6doc list. When you do that, you see a lot of topics. When you then try p6doc slurp or p6doc "sub slurp" you’ll get a No such type error. How does it work? What do I need to do?
  • When I run p6doc Str because I want to learn about join I find information about the vector flag. Searching for concat also doesn’t help. It must be some operator? Try p6doc operators, search for join, and then it dawns on me that it works on lists. Python: echo 'print(",".join(["1", "2"]))' | python3. Perl: perl6 -e 'say [1, 2].join(",")'. Oh well.
  • Are there native file locks? I’m looking at p6doc IO, IO, IO::Path, and I don’t see “lock” mentioned anywhere. On the #perl6 channel somebody tells me about IO::Handle. Yay!

And then there are all those little things that bite.

  • ternary operator is x ?? y !! z instead of x ? y : z
  • regular expressions are very different now
  • map requires a comma after the block
  • here docs are different

And many other things besides. As I’m too new to the language I don’t understand why things have to be “same same but different”. I hope there’s is a grandiose pay-off, I guess?

Here’s an example of how access to the hash elements no longer uses braces but angled brackets, the regular expression is different, the braces aren’t used when creating the hash, and map takes a comma after the block:

    my $menu = %*ENV<menu> || "Home, Changes";
    my @pages = $menu.split(/ ',' \s* /);
    my %params =
	id => $id,
	pages => [ map { id => $_ }, @pages ];

The Pod is also subtly different and the documentation confused me a bit. There are many options. What the suggested way of doing it? Is this good? Should I be using #| instead?

    =head3 put-page
    =begin pod
    Pages are saved in the C<page> subdirectory with the <md> extension.
    =end pod

    method put-page (Page $page!) is export {
	my $dir = %*ENV<dir> || '.';
	my $path = "$dir/page/$($page.name).md";
	spurt $path, $page.text, :enc('UTF-8');
    }

Anyway, that’s the kind of stuff I’m thinking about when I write Perl 6 code.

Oh, and recently my application has this super aggravating tendency: Often, when making changes (and this only started showing up recently!), I get an error.

♻ Restarting edit (edit)
⚠ edit ===SORRY!===
⚠ edit This type (NQPMu) does not support associative operations

The strange thing is that no matter what change I make, that fixes it. So now I have a second shell open where I just do the following:

echo >> lib/Storage/File.pm6

And then it works!

♻ Restarting edit (edit)
📓 edit Listening at http://localhost:20000

Apparently, all I have to do is trigger a recompilation.

When I’m testing, same thing.

dir=. storage=Storage::File prove6 -l -v t/edit.t

This causes an error:

===SORRY!===
This type (NQPMu) does not support associative operations
t/edit.t .. Dubious, test returned 1
No subtests run
All tests successful.

Test Summary Report
-------------------
t/edit.t (Wstat: 256 Tests: 0 Failed: 0)
Non-zero exit status: 1
  Parse errors: No plan found in TAP output
Files=1, Tests=0,  4 wallclock secs
Result: FAILED

The fix is simple:

echo >> lib/Storage/File.pm6

The result:

    ok 1 - Status is acceptable
    ok 2 - Content type is acceptable
    ok 3 - Body is acceptable
    1..3
ok 1 - GET /edit/About
    ok 1 - Status is acceptable
    ok 2 - Content type is acceptable
    ok 3 - Body is acceptable
    1..3
ok 2 - POST /save
ok 3 - page name correct
ok 4 - page content saved
ok 5 - changes correct
ok 6 - year
ok 7 - time
ok 8 - major change
ok 9 - page name
ok 10 - code
ok 11 - summary
    ok 1 - Status is acceptable
    ok 2 - Content type is acceptable
    ok 3 - Body is acceptable
    1..3
ok 12 - POST /save
ok 13 - year
ok 14 - time
ok 15 - minor change
ok 16 - page name
ok 17 - author
ok 18 - summary
1..18
t/edit.t .. ok
All tests successful.
Files=1, Tests=18,  6 wallclock secs
Result: PASS

How weird is that? I asked on the #perl6 channel and one of the people there was curious and was interested in fixing this. Sadly, I was unable to produce a small working example. I tried for quite a while. I have no idea what causes this.

Tags:

Comments on 2018-09-18 Thoughts on using Perl 6

Sadly, I was unable to produce a small working example. I tried for quite a while. I have no idea what causes this.

I can try something without a small example, as long as you can provide any example that reproduces the issue in at least a semi-reliable fashion. Basically we have a bunch of tools that can run stuff on any given rakudo commit, and bisect if necessary. Just ping me on #perl6 if you have that (a zip file or something like that will do) and I’ll try to find out if it’s a new issue or if it existed before, and we’ll see what caused it if it’s new.

– AlexDaniel 2018-09-18 22:28 UTC


I’ll keep it in mind. It’s been working for today, with no issues except for once. Who knows.

– Alex Schroeder 2018-09-19 14:41 UTC


Here’s another strange thing: locks don’t seem to work as I’d expect them to.

use File::Temp;
my $fh = open('test.p6', :w);
say "File locked: " ~ $fh.lock;
say "Deadlock: " ~ $fh.lock;

Output:

File locked: True
Deadlock: True

I sort of expected there to be a deadlock as the second call to lock never returns. But apparently not...

So I wrote my code using empty lock directories, like I did for Perl 5.

– Alex Schroeder 2018-09-19 14:44 UTC


One of the goals of the design of Perl 6 is to be more consistent.

This is why map has a comma after the block, it is just a subroutine and the block is just a lambda.

This consistency is even more evident when you look at the pointy block.

for @a -> $i { say $i }
if $a.method() -> $result { say $result }
my &lambda = -> $a { say $a }

Note that pointy blocks works with all operators of the same form as for and if.

The reason for ?? !! is that

  1. The : is used everywhere for many things, so it is very confusable.
  2. The ? is used for a variety of features itself.
  3. There is no need for !!$a because you can just use ?$a

Frankly, it is a very odd operator because it is in a category all of its own. If it wasn’t already so widespread amongst other languages it probably wouldn’t have been included at all. (similar reason for the inclusion of loop (;;){})

The Regex sub-language got a complete overhaul as it was getting very kludgy. It was also better integrated such that regular Perl 6 language features can be used rather than create regex exclusive features. (A regex in Perl 6 is just a type of method with a different base syntax)

p6doc copies perldoc in that you use the -f flag to read up on functions.

– Brad Gilbert (b2gills) 2018-09-20 23:28 UTC


Thanks!

– Alex Schroeder 2018-09-21 21:31 UTC


Just to note, whenever you would normally reach for a parent class, it’s probably more idiomatic for Perl 6 to use a Role instead.

Then you can have your absolute base class as a role with the barest essentials, including stubbed methods that “role-ing” classes are required to implement.

The advantage is that you don’t end up with the kind of hierarchical entanglement associated with multiple inheritance and you get to keep all the goodness of distinct type-ness, Smartmatching against a role vs a class is the same for the end user, for example.

– ab5tract 2018-09-24 20:01 UTC


Well, right now I’m basically using classes as structs: just data, no methods. Roles seem like a very cool addition to objects, but my love for Lisp leads my code to much less object oriented than what people might be used to.

– Alex Schroeder 2018-09-24 21:48 UTC

Add Comment

2018-09-08 What would a new Oddmuse be?

I was at the Swiss Perl Workshop and bought Perl 6 Deep Dive by Andrew Shitov. I’m again enthused by the idea of writing an application in Perl 6.

What would a new Oddmuse be like? If I’d write a new wiki using Perl 6 instead of Perl 5, I’d like to keep the name. It’d be Oddmuse 6 for a while, and then it’d simply be Oddmuse again.

Here’s how I would start:

  • install rakudo-star
  • install cro
  • create a stub ✓
  • implement a view action ✓
  • use Markdown::Text to render text ✓
  • text is loaded from text files in a page directory ✓
  • implement an edit action ✓
  • text is saved in text files ✓

I guess this is how far I got with Oddµ: A Small Wiki Written in Go, more or less. At the time I still believed in WikiCreole. But there’s more.

  • old revisions are kept in a a keep directory, like today ✓
  • write an OpenAPI Specification as we go along
  • a log file for recent changes is still used, don’t use git or the like because I want data to be ephemeral and old revisions to fade away into oblivion ✓
  • meta data is kept in a separate file, not mixed with the text file
  • implement a changes action ✓
  • implement a feed action
  • implement a history action ✓
  • implement a rollback action ✓
  • implement a diff action including the highlighting of changed words ✓
  • implement merging of files
  • implement error handing for merge conflicts, unlike now where the conflict is saved
  • add HTTPS
  • add passwords
  • implement banning of URL fragments and regular expressions matching text for admins
  • implement page locking and site locking for admins

This would result in a small, functional wiki. The user interface wouldn’t be template based, just like it isn’t template based right now. I think this is necessary to make it easy to mix and match extensions.

I think that, like before, we should move as much as possible into modules. So once we’re at this stage, we need to think about ways people can “compose grammars” for their text rendering.

Tags:

Comments on 2018-09-08 What would a new Oddmuse be?

Hey! :)

So it has been a while since I touched Oddmuse, but here are some of my (previous) random realizations.

  • The user interface can be wiki-based. That is, a top page can be used to render everything, as if that page was a template by itself. It can be markdown or whatever, as long as you can extend it enough to be able to call internal-ish functions from it. This is similar to sidebar / gotobar oddmuse extensions, just one tiny step ahead.
  • Another potentially crazy idea is to make any editable thing a page, e.g. settings and what not. That page does not have to be editable, e.g. it can be hardcoded internally to always be the same, but that can open a lot of potential for other awesome things.
  • Cro is the right choice, yes.
  • Grammar-based parsing would be ideal, but parsing wiki markup with grammars is a pain (one challenge is that any input is valid)… I actually experimented with it for quite a bit and even managed to do multiple grammars (e.g. basic Markdown and basic Creole) which can then be combined to achieve Markdown + Creole parsing. It does work, it can be done, but it needs quite some manpower which I currently can’t provide.
  • As of today, avoid DBIish and keep file-based db. There are multiple stability issues with DBIish, so even if you wanted to use a proper DBMS you’d regret it…
  • Maybe JSON for storing pages? I don’t know… I’m slightly worried about attacks focusing on edge cases in json parsing. But that’d be an easy and good start I guess.
  • Make page store backend as a separate module/plugin (can be in the same file though, that doesn’t matter). Just so that those who need Git backend can easily plug it in when, or so that eventually people can use a DBMS if they want to.

FWIW, I’m quite happy with most of the things in oddmuse5. So if it was the same thing, but with biggest pain points (html5, CSP, etc.) resolved, I’d be a happy user again :)

Ah, also you’d probably need caching just like in oddmuse5 (because Rakudo is currently slow-ish). It’s a useful mechanism for other things too.

– AlexDaniel 2018-09-09 16:59 UTC


Thanks for the feedback! Now all I need to do is find some time. :)

– Alex Schroeder 2018-09-10 10:25 UTC


Oddmuse 6, first commit! :)

– Alex Schroeder 2018-09-12 16:22 UTC


Rewriting it to be based on Template::Mustache. Still 100% Text::Markown. No revisions, no recent changes, no CSS. Not even page editing works. But there’s already and edit link and it shows the raw text if you click on it.

Screenshot

– Alex Schroeder 2018-09-13 15:59 UTC


Hi Alex,

I am proud I sold the book to you at the Swiss Perl Workshop 2018. Interesting blog, interesting plan. I hope you succeed and will give a good presentation about it next year at the Swiss Perl Workshop 2019.

I hope you don’t mind, but my happiness lead me to writing about you on Facebook, and I posted in the Perl6 group (which is open, no need for an account to read; an account is needed to write though).

– Wendy van Dijk 2018-09-13 18:50 UTC


Thanks! :)

– Alex Schroeder 2018-09-13 21:54 UTC


Very interesting! A small, tractable wiki in the spirit of oddmuse would be fab!

I checked out the repo, and noticed that the first paragraph of the README is quite confusing: “...based on Perl 5 ... instead of Perl 5...”

– Robert Lemmen 2018-09-14 12:03 UTC


Hah! Thanks for the heads up. Fixed!

– Alex Schroeder 2018-09-14 17:05 UTC


Editing pages works. CSS added. Tests added.

– Alex Schroeder 2018-09-17 09:02 UTC


Sadly, Git::PurePerl only does checkout and clone, Git::Simple is archived by the author and all the forks are lagging behind instead of forging ahead, Git::Wrapper leads to a 404, wow, the situation is bad for Perl 6 Modules about git. Maybe LibGit2 will do it?

sudo apt install libgit2-dev
zef install LibGit2

This fails with an error... but this works:

zef install https://github.com/CurtTilmes/perl6-libgit2.git

– Alex Schroeder 2018-09-17 11:06 UTC


I ran out of steam on LibGit2 a while back. It has the basics, but LibGit2 is really, really big (and complicated), so not everything is in there yet. – if it is missing anything you need, file an issue, and I may find some time to implement some more. It actually has a lot more functionality than is completely documented, if you can’t figure something out, file an issue and I’ll try to write some more docs. The tests can be helpful to figure out how to use it too.

I’m also trying to add more Perl 6-ish ways to do things conveniently, so if something seems harder than it should be, file an issue..

I’ll try to figure out why the install failed for you.

If you do want a database, I agree DBIish has issues, but I’m using DB::Pg with PostgreSQL pretty heavily and it has been solid for me. If anything breaks with it for you, please file an issue. I recently fixed a bug with embedded NULLs in arrays I haven’t yet released, so there will be a new release of it soon.

– Curt Tilmes 2018-09-18 01:19 UTC


Thanks!

– Alex Schroeder 2018-09-18 12:49 UTC


Anyway, I am happy to report that I have a form of recent changes:

Screenshot of Recent Changes

As you can see, the IP numbers are pseudo-anonymized and colour coded.

– Alex Schroeder 2018-09-18 13:20 UTC

Add Comment

Comments


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. Note: in order to facilitate peer review and fight vandalism, we will store your IP number for a number of days. See Privacy Policy for more information. See Info for text formatting rules. You can edit the comment page if you need to fix typos. You can subscribe to new comments by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.