Comments on 2015-10-09 Oddmuse and Mojolicious

Cool! That’s interesting.

What about side effects? Like, if you forget to initialize some variable somewhere, what is going to happen? (Thinking about mod_perl and all stuff related to it)

Also, I miss one important thing from your report... What about page load time? How much faster is it right now, comparing to regular cgi script?

– AlexDaniel 2015-10-09 11:21 UTC


I did not measure page load time. I also think that it keeps forking to run the script, so globals start empty every time? Not sure about the details. I haven’t seen any side effects, yet. Let me know if you see anything!

– Alex Schroeder 2015-10-09 11:39 UTC


Could you be so kind to measure the load time?

– AlexDaniel 2015-10-09 11:50 UTC


I think I was under the illusion that it was working. Now it is working. Hah! I had to rename my old wrapper script to realize that my ProxyPass instructions had been wrong.

Things to remember:

Check the config file for weird stuff.

1. if ($ENV{SERVER_PORT} == 8080) {
1.   $ScriptName  = 'https://alexschroeder.ch:8080/wiki';
1.   $FullUrl     = 'https://alexschroeder.ch:8080/wiki';
1. }

Make sure the user running toadfarm can write alle the files in your data directory. In my case, the data directory belongs to www-data.alex but there were temp files that alex could not write.

– Alex Schroeder 2015-10-09 13:27 UTC


Hm. Current setup, I get more or less the same time when I look at network analysis with my browser when I’m loading /wiki/About and when I’m loading /wiki2/About with wiki2.pl being my old wiki.pl wrapper script. I get numbers between 200ms and 700ms. I do note that /wiki/About keeps getting me a 200 OK response instead of a 304 NOT MODIFIED. I’m not quite sure what the problem is.

– Alex Schroeder 2015-10-09 13:40 UTC


Oh, and the code shows that our script is being exec’d. I’m not sure about the headers. I think if our script prints HTTP headers, we should be fine.

– Alex Schroeder 2015-10-09 13:47 UTC


Well, at least it seems to work right now. I think I got confused about the various restarts required.

The missing 304 NOT MODIFIED remains a mistery, however. I’ve tried $q = new CGI; $q->nph(1) in the config file, $CGI::NPH = 1 in the config file, CGI::nph(1) in the config file, use CGI qw/-utf8 -nph/ in the core script, and print $q->header(-nph=>1, -status=>'304 NOT MODIFIED') and return if PageFresh(); in the core script, reloading my toadfarm between edits and found no change.

When I tried to export MOJO_PLUGIN_CGI_DEBUG=1 before starting the toadfarm, I got a 503 error when visiting the site.

This is very annoying.

– AlexSchroeder 2015-10-09 14:32 UTC


OK, so the question is: How often is 304 NOT MODIFIED actually used?

alex@kallobombus:~/farm$ sudo wc -l /var/log/apache2/access.log.1
259852 /var/log/apache2/access.log.1
alex@kallobombus:~/farm$ sudo grep ^oddmuse /var/log/apache2/access.log.1 | wc -l
32867
alex@kallobombus:~/farm$ sudo grep "^oddmuse.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l
270

Filtering out feed request we’re down to 8‰.

For my own site:

alex@kallobombus:~/farm$ sudo grep ^alexschroeder /var/log/apache2/access.log.1 | wc -l
79286
alex@kallobombus:~/farm$ sudo grep "^alexschroeder.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l
702

This reduces the percentage to 9‰.

So, if we redirect feed requests to files and use a cron job to produce some of them, the 304 NOT MODIFIED response seems to be something we can do without, even if I think it ought to work.

– AlexSchroeder 2015-10-09 14:54 UTC


In other words, no noticeable performance boost? Do I read it right? But then why bother?

– AlexDaniel 2015-10-09 21:26 UTC


Hehe, indeed. The point is, however, that I want some sort of plan that will allow me to incrementally develop a solution. Mojolicious::Plugin::CGI does a lot of what I want. It integrates into Mojolicious and thus it can do logging, forking, and so on. It handles input and output. The only thing that’s missing is that it exec’s the script – loading Perl and compiling all the modules on every request.

I think I’ll try and modify the Plugin for my purposes. Then it’ll require Oddmuse once and just run DoWikiRequest on every request, mod_perl style.

– AlexSchroeder 2015-10-10 07:40 UTC


These are the changes I had to make to Mojolicious::Plugin::CGI: Add run option for a code reference and Make sure the status code is actually used and Add HTTP_IF_NONE_MATCH to environment.

This is the new server script I used:

#! /usr/bin/env perl

use lib '/Users/alex/src/mojolicious-plugin-cgi/lib';

use Mojolicious::Lite;

plugin CGI => {
  support_semicolon_in_query_string => 1,
};
 
plugin CGI => {
  route => '/wiki',
  script => 'wiki.pl',
  run => \&OddMuse::DoWikiRequest,
  before => sub {
    $OddMuse::RunCGI = 0;
    $OddMuse::DataDir = '/tmp/oddmuse';
    require 'wiki.pl' unless defined &OddMuse::DoWikiRequest;
  },
  env => {},
  errlog => 'wiki.log', # path to where STDERR from cgi script goes
};

get '/' => sub {
  my $self = shift;
  $self->redirect_to('/wiki');
};
 
app->start;

– AlexSchroeder 2015-10-11 06:51 UTC


With the above setup now installed on alexschroeder.ch, I’m getting response times of 150–230ms when looking at /wiki/About (and a 304 Not Modified response). The old CGI setup under /wiki2/About is giving response times of 250–720ms.

So, much better?

I’ve also started noticing the first signs of trouble: Edit page, save, look at it (looks good), edit page again and notice that it’s showing the old text, not the new text.

– AlexSchroeder 2015-10-11 10:28 UTC


When I tried it with this page, I saw my browser requesting the page using the following:

GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1
...
If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com

Oddmuse replied with a 304 Not Modified and the content was wrong. As I checked, however, the timestamp of the index file does in fact match:

alex@kallobombus:~/alexschroeder$ date -r pageidx +%s
1444395217

Looking at the log of recent changes, however:

alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log
1444559479Comments_on_2015-10-09_Oddmuse_and_Mojolicious1With the above setup now installed on alexschroeder.ch, I'm getting response times of 150–230ms when looking at wikiAbout (and a 304 Not Modified response). So, much better? 127.0.0.1AlexSchroeder23en

I guess something about TouchIndexFile isn’t working.

sub TouchIndexFile {
  my $ts = time;
  utime $ts, $ts, $IndexFile;
  $LastUpdate = $Now = $ts;
}

I’ve made sure that all the files belong to alex.alex instead of www-data.alex. And that seems to help. After saving:

alex@kallobombus:~/alexschroeder$ date -r pageidx +%s
1444560771
alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log
1444560771Comments_on_2015-10-09_Oddmuse_and_MojoliciousWhen I tried it with this page, I saw my browser requesting the page using the following: {{{ GET wiki?action=edit;id=Commentson2015-10-09OddmuseandMojolicious HTTP1.1 … If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com }}} Oddmuse replied with a 304 Not Modified and the content…127.0.0.1AlexSchroeder24en

And the next request:

GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1
...
If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com

HTTP/1.1 200 OK
...
Etag: 1444560771%1e1%1eAlexSchroeder%1ekensanata%40gmail.com
...

Yay?

– AlexSchroeder 2015-10-11 10:52 UTC


OK, added one final patch to make it work without having to use the -nph option for the CGI library and submitted a pull request. And I installed it for this site as well. “Eat your own dog food,” and all that.

– AlexSchroeder 2015-10-11 12:41 UTC


Wow, many more commits after talking to the maintainer.

– AlexSchroeder 2015-10-11 16:14 UTC


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. Note: in order to facilitate peer review and fight vandalism, we will store your IP number for a number of days. See Privacy Policy for more information. See Info for text formatting rules. You can edit this page if you need to fix typos. You can subscribe to updates by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.