Software

RSS Feed This page lists only the software pages recently written.

2016-01-23 Testing Encrypted Mail Setup

I’ve updated my guide, the Gmail Gnus GPG Guide (GGGG) to incorporate the latest addition to the FSF’s Email Self-Defense guide to free software-powered encryption. They even have a guide on how to help your friends and family, including talking points if want to make your case.

Anyway, the thing I added to the guide is how to initiate communication with the bot:

  1. send him your Public key
  2. get his public key from a keyserver
  3. how to set up a keyserver in order to achieve step #2
  4. send him an encrypted mail

Tags: RSS RSS RSS RSS

Comments on 2016-01-23 Testing Encrypted Mail Setup

The public (not private) key should be sent in point 1. Best regards, A.

– A. 2016-01-23 16:01 UTC


Aaaaargh!

– Alex Schroeder 2016-01-23 16:36 UTC


Thanks!

By the way, what’s the reason to use gpg instead of gpg2?

– AlexDaniel 2016-01-23 22:44 UTC


Also, it looks like it has to be ~/.gnus.el instead of ~/.gnus.

– AlexDaniel 2016-01-24 00:07 UTC


Hm, I think ~/.gnus is fine, according to the manual: “… and gnus-init-file (‘~/.gnus’ by default)”

As for the GPG version, I am using gpg. Perhaps on Debian version 2 is only available as gpg2?

alex@Megabombus:~$ gpg --version
gpg (GnuPG) 2.0.29
libgcrypt 1.6.4
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: ~/.gnupg
Supported algorithms:
Pubkey: RSA, RSA, RSA, ELG, DSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

– Alex Schroeder 2016-01-24 00:15 UTC


Interesting. Also it would be great to know how to make gnus sort things in the right order by default…

– AlexDaniel 2016-01-24 00:35 UTC


Oh well. It seems like the idea is to have only unread messages visible by default. If you keep it this way then there’s no reason to change the order. Hmmmm.

– AlexDaniel 2016-01-24 01:37 UTC


I’m not sure if this approach is really useful. Gmail sorts messages into three tabs – Primary, Social and Promotions. Obviously I never read Social and Promotions, and I even ignore some messages in Primary. That’s the result:

https://files.progarm.org/2016-01-24-033741_154x97_scrot.png

In such case default settings in gnus are so alien. But I thought that it is how all people read their emails? Is there any way to get this Primary/Social/Promotions distinction in gnus?

– AlexDaniel 2016-01-24 01:40 UTC


Not unless Gmail makes the tabs available via IMAP, i.e. via folders (like it does for labels). I don’t think this is happening. Apparently you can search for them but I don’t understand how you would use this from Gnus. [1]

– Alex Schroeder 2016-01-27 20:26 UTC

Add Comment

2015-10-09 Oddmuse and Mojolicious

I’m trying to run Oddmuse within a Perl web framework: Mojolicious using Mojolicious::Plugin::CGI. It won’t be as perfect as a true Mojolicious app, but it will still be much faster than a simple CGI script. When running as a CGI script, every request loads Perl and compiles all the modules – including the CGI module itself – and Oddmuse and the config files. When running under Mojolicious, we no longer load Perl and we compile Oddmuse just once. Oddmuse itself will keep loading the config file and all that, but it’s still much better than before. The Mojolicious app itself is then started by Toadfarm. And in order to force myself to test it, I’ve switched this wiki over to the new setup!

First, tell Apache to act as a reverse proxy and pass all /wiki requests to the new server. The two ProxyPass instructions at the bottom are the important bits:

<VirtualHost *:80>
    ServerName alexschroeder.ch
    ServerAlias www.alexschroeder.ch
    Redirect permanent / https://alexschroeder.ch/
</VirtualHost>
<VirtualHost *:443>
    ServerAdmin alex@alexschroeder.ch
    <Directory />
        Options None
        AllowOverride None
        Order Deny,Allow
        Deny from all
    </Directory>
    ServerName alexschroeder.ch
    ServerAlias www.alexschroeder.ch
    ServerAlias rpg.alexschroeder.ch
    DocumentRoot /home/alex/alexschroeder.ch
    <Directory /home/alex/alexschroeder.ch>
        Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
        AddHandler cgi-script .pl
        AllowOverride All
        Order Allow,Deny
        Allow from all
    </Directory>

    SSLEngine on
    SSLCertificateFile      /home/alex/ssl/alexschroeder.crt
    SSLCertificateKeyFile   /home/alex/ssl/alexschroeder.key
    SSLCertificateChainFile /home/alex/ssl/GandiStandardSSLCA2.pem
    SSLVerifyClient None

    ProxyPass /wiki             https://alexschroeder.ch:8080/wiki
    ProxyPass /mojo             https://alexschroeder.ch:8080/mojo

</VirtualHost>

Toadfarm setup in ~/farm/farm:

#!/usr/bin/env perl
use Toadfarm -init;

logging {
  combined => 1,
  file     => "farm.log",
  level    => "info",
};

# other apps go here...

mount "$farm/alexschroeder.pl" => {
  "Host" => qr{^(www.)?alexschroeder\.ch$},
  mount_point => '/wiki',
};

plugin "Toadfarm::Plugin::AccessLog";

start; # needs to be at the last line

The above means that all requests to htpp://alexschroeder.ch:8080/wiki and htpp://www.alexschroeder.ch:8080/wiki will be handled by Oddmuse.

Mojolicious wrapper in ~/farm/alexschroeder.pl:

#! /usr/bin/env perl

use Mojolicious::Lite;

plugin CGI => {
  support_semicolon_in_query_string => 1,
};

plugin CGI => {
  route => '/',
  script => 'wiki.pl', # ~/farm/wiki.pl
  env => {WikiDataDir => '/home/alex/alexschroeder'},
  errlog => 'alexschroeder.log', # path to where STDERR from cgi script goes
};

app->start;

In this case, wiki.pl is simply the original Oddmuse script, unchanged. :)

Tags: RSS RSS RSS RSS

Comments on 2015-10-09 Oddmuse and Mojolicious

Cool! That’s interesting.

What about side effects? Like, if you forget to initialize some variable somewhere, what is going to happen? (Thinking about mod_perl and all stuff related to it)

Also, I miss one important thing from your report… What about page load time? How much faster is it right now, comparing to regular cgi script?

– AlexDaniel 2015-10-09 11:21 UTC


I did not measure page load time. I also think that it keeps forking to run the script, so globals start empty every time? Not sure about the details. I haven’t seen any side effects, yet. Let me know if you see anything!

– Alex Schroeder 2015-10-09 11:39 UTC


Could you be so kind to measure the load time?

– AlexDaniel 2015-10-09 11:50 UTC


I think I was under the illusion that it was working. Now it is working. Hah! I had to rename my old wrapper script to realize that my ProxyPass instructions had been wrong.

Things to remember:

Check the config file for weird stuff.

# if ($ENV{SERVER_PORT} == 8080) {
#   $ScriptName  = 'https://alexschroeder.ch:8080/wiki';
#   $FullUrl     = 'https://alexschroeder.ch:8080/wiki';
# }

Make sure the user running toadfarm can write alle the files in your data directory. In my case, the data directory belongs to www-data.alex but there were temp files that alex could not write.

– Alex Schroeder 2015-10-09 13:27 UTC


Hm. Current setup, I get more or less the same time when I look at network analysis with my browser when I’m loading /wiki/About and when I’m loading /wiki2/About with wiki2.pl being my old wiki.pl wrapper script. I get numbers between 200ms and 700ms. I do note that /wiki/About keeps getting me a 200 OK response instead of a 304 NOT MODIFIED. I’m not quite sure what the problem is.

– Alex Schroeder 2015-10-09 13:40 UTC


Oh, and the code shows that our script is being exec’d. I’m not sure about the headers. I think if our script prints HTTP headers, we should be fine.

– Alex Schroeder 2015-10-09 13:47 UTC


Well, at least it seems to work right now. I think I got confused about the various restarts required.

The missing 304 NOT MODIFIED remains a mistery, however. I’ve tried $q = new CGI; $q->nph(1) in the config file, $CGI::NPH = 1 in the config file, CGI::nph(1) in the config file, use CGI qw/-utf8 -nph/ in the core script, and print $q->header(-nph=>1, -status=>'304 NOT MODIFIED') and return if PageFresh(); in the core script, reloading my toadfarm between edits and found no change.

When I tried to export MOJO_PLUGIN_CGI_DEBUG=1 before starting the toadfarm, I got a 503 error when visiting the site.

This is very annoying.

– AlexSchroeder 2015-10-09 14:32 UTC


OK, so the question is: How often is 304 NOT MODIFIED actually used?

alex@kallobombus:~/farm$ sudo wc -l /var/log/apache2/access.log.1
259852 /var/log/apache2/access.log.1
alex@kallobombus:~/farm$ sudo grep ^oddmuse /var/log/apache2/access.log.1 | wc -l
32867
alex@kallobombus:~/farm$ sudo grep "^oddmuse.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l
270

Filtering out feed request we’re down to 8‰.

For my own site:

alex@kallobombus:~/farm$ sudo grep ^alexschroeder /var/log/apache2/access.log.1 | wc -l
79286
alex@kallobombus:~/farm$ sudo grep "^alexschroeder.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l
702

This reduces the percentage to 9‰.

So, if we redirect feed requests to files and use a cron job to produce some of them, the 304 NOT MODIFIED response seems to be something we can do without, even if I think it ought to work.

– AlexSchroeder 2015-10-09 14:54 UTC


In other words, no noticeable performance boost? Do I read it right? But then why bother?

– AlexDaniel 2015-10-09 21:26 UTC


Hehe, indeed. The point is, however, that I want some sort of plan that will allow me to incrementally develop a solution. Mojolicious::Plugin::CGI does a lot of what I want. It integrates into Mojolicious and thus it can do logging, forking, and so on. It handles input and output. The only thing that’s missing is that it exec’s the script – loading Perl and compiling all the modules on every request.

I think I’ll try and modify the Plugin for my purposes. Then it’ll require Oddmuse once and just run DoWikiRequest on every request, mod_perl style.

– AlexSchroeder 2015-10-10 07:40 UTC


These are the changes I had to make to Mojolicious::Plugin::CGI: Add run option for a code reference and Make sure the status code is actually used and Add HTTP_IF_NONE_MATCH to environment.

This is the new server script I used:

#! /usr/bin/env perl

use lib '/Users/alex/src/mojolicious-plugin-cgi/lib';

use Mojolicious::Lite;

plugin CGI => {
  support_semicolon_in_query_string => 1,
};
 
plugin CGI => {
  route => '/wiki',
  script => 'wiki.pl',
  run => \&OddMuse::DoWikiRequest,
  before => sub {
    $OddMuse::RunCGI = 0;
    $OddMuse::DataDir = '/tmp/oddmuse';
    require 'wiki.pl' unless defined &OddMuse::DoWikiRequest;
  },
  env => {},
  errlog => 'wiki.log', # path to where STDERR from cgi script goes
};

get '/' => sub {
  my $self = shift;
  $self->redirect_to('/wiki');
};
 
app->start;

– AlexSchroeder 2015-10-11 06:51 UTC


With the above setup now installed on alexschroeder.ch, I’m getting response times of 150–230ms when looking at /wiki/About (and a 304 Not Modified response). The old CGI setup under /wiki2/About is giving response times of 250–720ms.

So, much better?

I’ve also started noticing the first signs of trouble: Edit page, save, look at it (looks good), edit page again and notice that it’s showing the old text, not the new text.

– AlexSchroeder 2015-10-11 10:28 UTC


When I tried it with this page, I saw my browser requesting the page using the following:

GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1
...
If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com

Oddmuse replied with a 304 Not Modified and the content was wrong. As I checked, however, the timestamp of the index file does in fact match:

alex@kallobombus:~/alexschroeder$ date -r pageidx +%s
1444395217

Looking at the log of recent changes, however:

alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log
1444559479Comments_on_2015-10-09_Oddmuse_and_Mojolicious1With the above setup now installed on alexschroeder.ch, I'm getting response times of 150–230ms when looking at wikiAbout (and a 304 Not Modified response). So, much better? 127.0.0.1AlexSchroeder23en

I guess something about TouchIndexFile isn’t working.

sub TouchIndexFile {
  my $ts = time;
  utime $ts, $ts, $IndexFile;
  $LastUpdate = $Now = $ts;
}

I’ve made sure that all the files belong to alex.alex instead of www-data.alex. And that seems to help. After saving:

alex@kallobombus:~/alexschroeder$ date -r pageidx +%s
1444560771
alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log
1444560771Comments_on_2015-10-09_Oddmuse_and_MojoliciousWhen I tried it with this page, I saw my browser requesting the page using the following: {{{ GET wiki?action=edit;id=Commentson2015-10-09OddmuseandMojolicious HTTP1.1 … If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com }}} Oddmuse replied with a 304 Not Modified and the content…127.0.0.1AlexSchroeder24en

And the next request:

GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1
...
If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com

HTTP/1.1 200 OK
...
Etag: 1444560771%1e1%1eAlexSchroeder%1ekensanata%40gmail.com
...

Yay?

– AlexSchroeder 2015-10-11 10:52 UTC


OK, added one final patch to make it work without having to use the -nph option for the CGI library and submitted a pull request. And I installed it for this site as well. “Eat your own dog food,” and all that.

– AlexSchroeder 2015-10-11 12:41 UTC


Wow, many more commits after talking to the maintainer.

– AlexSchroeder 2015-10-11 16:14 UTC

Add Comment

2015-10-06 Pink Bliss

Pink Bliss has improved somewhat. Mostly, I just added some magit faces.

You know you want it! Download from GitHub!

Emacs, in it’s maximized glory:

https://farm6.staticflickr.com/5775/21940845326_d05aa31cc9_c.jpg

Tags: RSS RSS

Add Comment

2015-10-05 Character Sheet Generator Using New Face Generator

My Character Sheet Generator now uses my Face Generator. The rewrite is done and it’s time to go back to drawing some face elements! Elves!

Tags: RSS RSS RSS

Comments on 2015-10-05 Character Sheet Generator Using New Face Generator

Now drawing: dwarf beards.

– Alex Schroeder 2015-10-07 09:39 UTC

Add Comment

2015-10-02 RPG Web Apps I Have Written

I’m trying to make a list of web apps I have written for role-playing games…

If you like what I’m doing, you can sponsor future web apps via Patreon or just send some money to kensanata@gmail.com using PayPal.

Tags: RSS RSS

Comments on 2015-10-02 RPG Web Apps I Have Written

I think that you forgot to mention Gridmapper.

– AlexDaniel 2015-10-03 16:26 UTC


Oops! Thanks, added.

– Alex Schroeder 2015-10-03 22:16 UTC


And I have finished rewriting the Character Generator as a Mojolicious::Lite application and both the Face Generator are now running as two apps within a Toadfarm, and Toadfarm is beeing monitored by Monit. I think I’m happy now!

– Alex Schroeder 2015-10-05 07:35 UTC

Add Comment

2015-09-28 Better

Everything is better! My Face Generator is progressing nicely. I think the galleries for men and women are already pretty good. The entire app is running on Hypnotoad behind Apache/mod_proxy. It’s fast! And I’m going to support multiple artists. Please help me if you can! I’m not great with faces. I have barely drawn any until this project started. I had to read How to Draw a Face on WikiHow, that’s how bad I was!

Here’s how I’m doing it right now. The basic instructions are on GitHub.

First, print a copy of my empty.pdf. If you need a thinner face, take my elf.pdf. If you need a rounder face, take my dwarf.pdf.

If you printed the default file, this is what it would look like:

https://farm1.staticflickr.com/595/21790351671_13a47f2a39_c.jpg
Download the PDF!

Make sure you don’t scale the image to fit the page. Try to use 100% of the image.

Then you draw 25 images of a particular element: hair, eyes, nose, ears, mouth, chin. 10% of all faces also get an extra element. Good for scars. My experience is that you need around 50 components for every element to provide enough variety – two sheets for each element is what I suggest.

Then you scan it. You should get something like the following:

https://farm1.staticflickr.com/637/21593038178_06595f84b8_c.jpg

There are more examples on Flickr.

If you have something like the above your work is done. Send the images to me and I will do the rest!

What does the rest involve? You crop it and resize it to 2250×3000 pixels. My scanner is notoriously bad about these proportions.

I have scripts to extract whatever you draw from the helping lines of your scan. I have three variants ready to go:

  1. The standard reduces the image to a few shades of blue: convert -blur 0x1 +dither -remap tintenblau.png scan1.jpg source1.png
  2. This is good if you draw using a pencil and want to keep that pencil look: convert -blur 0x1 +dither -level 30%,80% scan1.jpg source1.png
  3. This is good if you draw using a pencil and want to have it look like it was inked without going through the trouble of inking it yourself: convert -blur 0x1 +dither -level 50%,80% -grayscale rec709luma scan1.jpg source1.png

I think you need to think about speed. Two sheet per element with six elements means 12 sheets. Save yourself the time of inking! Fiddle with the -level argument instead. The first percentage means that every pixels darker than this much is going to end up black. The second percentage means that every pixel lighter than this much is going to end up white. Raise the first number until you’re happy with the darkness of your image. Lower the second number until the helping lines disappear.

You should end up with something like the following:

https://farm6.staticflickr.com/5827/21521333488_74f104033a_c.jpg

Now’s the time to use my cutter script. It splits the image into 25 images of 450 x 600 pixels each. You must tell it the artist (alex), the element (hair, eyes, nose, ears, mouth, chin, extra) and the type (man, woman, elf, dwarf) you want to save it under: perl helpers/cutter.pl source1.png alex eyes_all.

Eventually, you’ll end up with a large number of files.

And that’s it! I commit it to my repository on GitHub and deploy it on my server.

If you’re running the web application, there’s even the possibility to debug faces.

I’m going to link to the debug interface so that you can take a look – but in the end, you won’t be able to save any changes unless you’re the artist of one the collections.

  1. Choose an element to debug
  2. List all the components for this element (in this case, eyes)
  3. Edit one component (in this case, eyes_all_59.png)
  4. When you click any of the zones, you should get an error… :)

Tags: RSS RSS RSS

Comments on 2015-09-28 Better

Airi Ayame has been adding manga style face elements. Thank you!! :)

Not yet complete, but already recognizable: → Women's gallery by Airi!

– Alex Schroeder 2015-09-28 12:30 UTC


So, if I want to get digital, a single pic with 450*600 pixels, maybe even with a blue palette would be great. I just found a book on my harddrive with some art from early 19th century - So no scanning but copying and pasting… I was worried that it is a little different stylewise, but, hey Manga… :-)

rorschachhamster 2015-09-28 15:56 UTC


I’ve drawn face elements for elves and dwarves and noticed that having a different head shape was essential for me. That’s why the repository now contains elf.pdf and dwarf.pdf.

Another thing I noticed was that having opaque elements is tricky. This is most relevant if you plan on having white or blond hair cover parts of the face. As you’ll be using a lot of white filling, and since white is considered to be transparent, the face will will shine through your hair.

In order to handle this, I have seen two solutions. Perhaps there are more?

  1. Don’t do it – anything that hangs “in front” of the face needs to be inked. This is what I do.
  2. Use some other color to fill areas that should be opaque and white. Use #FFFFFF for the transparent background and use #FEFEFF for the white-looking opaque foreground. This usually involves flood-filling those areas in Gimp or Photoashop and therefore requires well defined, enclosed areas. This is not always easy to achieve and takes a lot of time.
  3. Use colors. I haven’t seen this in actual use but it should work. If you need something light and opaque, use a light color as you draw. A very light blue, or beige, or grey. Just make sure it’s different from the white background and make sure it’s not interspersed whith white pixels. Using a light pencil might leave too many white pixels, for example.

– AlexSchroeder 2015-10-07 21:15 UTC

Add Comment

2015-09-18 Emacs and TTY

I’m connecting to Kallobombus via SSH. It runs Debian 7.9 aka Wheezy. Recently a new problem appeared. I can no longer use Emacs unless I’m root.

alex@kallobombus:~$ emacs
emacs: Could not open file: /dev/tty
alex@kallobombus:~$ sudo emacs
alex@kallobombus:~$ ls -l /dev/tty
crw------- 1 root root 5, 0 Sep 18 08:28 /dev/tty
alex@kallobombus:~$ cat /etc/debian_version 
7.9
alex@kallobombus:~$ which emacs
/usr/bin/emacs

What’s the problem?

On a different server, running Ubuntu 14.04.1 LTS:

kensanata@po6:~$ cat /etc/debian_version 
jessie/sid
kensanata@po6:~$ ls -l /dev/tty
crw-rw-rw- 1 root tty 5, 0 Sep 15 11:42 /dev/tty
kensanata@po6:~$ emacs

So, is the solution to chmod o+rw /dev/tty?

Oh well, I ran sudo chmod go+rw /dev/tty and that seemed to fix the problem. I just hope it didn’t create any security issues.

Tags: RSS RSS

Add Comment

2015-09-16 Mojolicious

Yesterday I needed to write a little prototype of a site that would allow users to spell check their documents. The idea being that we want to add new dictionaries to the site, demo it to the right owners and thus secure an open license for their word lists. This is for an association interested in providing tech support for minority languages called Korero.

I used the Text::Hunspell library and the Hunspell dictionaries available from Debian. Yay!

Oddmuse is my Perl 5 project and has been since 2003. At the time, all we needed was a CGI script. These days, however, people often seem to use their own web servers that are part of their application. And so I decided to take a look at Mojolicious. And it was simple to use and easy to get started. I like it!

More info on Github.

Plans:

  1. add mobile support, the CSS currently depends on the :hover attribute
  2. better menu placement, the CSS currently just uses position: absolute which places the menu at the left edge (this might require Javascript)
  3. the menu should replace the text in question
  4. a JSON interface to allow other apps to do thrir spell checking here

At the same time, we’re working on a Hunspell dictionary for one of the many Rumantsch idioms in Switzerland.

Tags: RSS RSS

Add Comment

2015-08-03 Switching to Postfix

What I want to do is read all my email using Google. I like Gmail, I like Inbox, I like their spam filtering. Cool. I also want to receive email for my various domains. Thus, all these domains have an MX record pointing to my Raspberry Pi. It runs Postfix and Dovecot, and my accounts have .forward files pointing to kensanata@gmail.com. This ought to be cool, too. If I remove that .forward file, I can read my email on the Raspberry Pi using IMAP.

Sadly, I’m getting a lot of these:

<kensanata@gmail.com>: host gmail-smtp-in.l.google.com[173.194.78.26] said:
    550-5.7.1 [77.57.27.11      12] Our system has detected that this message
    is 550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to
    Gmail, 550-5.7.1 this message has been blocked. Please visit 550 5.7.1
    https://support.google.com/mail/answer/188131 for more information.
    eu1si12802336wic.49 - gsmtp (in reply to end of DATA command)

So now I’m trying to setup up things to fix this. I’ve tried doing these things using Exim but I find the official documentation to be hard to understand for newbies like me. So now I’m trying to switch to Postfix.

SPF

In order to reduce spam, I have a TXT record in my zone file with some SPF1 information. There is some info by Google on what they suggest (include:_spf.google.com ~all) and there is the SPF Record Syntax documentation which shows you how to indicate who else can send mail for your domain (mx means “the MX records” and a means “the current domain”).

@ 10800 IN TXT "v=spf1 ip6:2001:4830:134:3::10 mx a include:_spf.google.com ~all"

The IP6 number is for egg.gnu.org. When I send mail from my Raspberry Pi to my gnu.org address without this information, it arrives on Gmail with the following headers:

Received: from eggs.gnu.org (eggs.gnu.org. [2001:4830:134:3::10])
        by mx.google.com with ESMTPS id g4si5396462igt.36.2015.08.03.02.14.24
        for <kensanata@gmail.com>
        (version=TLSv1 cipher=RC4-SHA bits=128/128);
        Mon, 03 Aug 2015 02:14:24 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning alex@alexschroeder.ch does not designate 2001:4830:134:3::10 as permitted sender) client-ip=2001:4830:134:3::10;
Authentication-Results: mx.google.com;
       spf=softfail (google.com: domain of transitioning alex@alexschroeder.ch does not designate 2001:4830:134:3::10 as permitted sender) smtp.mail=alex@alexschroeder.ch
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <alex@alexschroeder.ch>)
	id 1ZMBp9-0002rf-2D
	for kensanata@gmail.com; Mon, 03 Aug 2015 05:14:23 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40 autolearn=disabled
	version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37492)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <alex@alexschroeder.ch>)
	id 1ZMBp8-0002rb-VG
	for kensanata@gmail.com; Mon, 03 Aug 2015 05:14:22 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45863)
	by fencepost.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256)
	(Exim 4.82)
	(envelope-from <alex@alexschroeder.ch>)
	id 1ZMBp8-0005RP-Lv
	for alex@gnu.org; Mon, 03 Aug 2015 05:14:22 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <alex@alexschroeder.ch>)
	id 1ZMBp5-0002r8-Ar
	for alex@gnu.org; Mon, 03 Aug 2015 05:14:22 -0400
Received: from 77-57-27-11.dclient.hispeed.ch ([77.57.27.11]:48751 helo=mail.alexschroeder.ch)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <alex@alexschroeder.ch>)
	id 1ZMBp5-0002qy-3w
	for alex@gnu.org; Mon, 03 Aug 2015 05:14:19 -0400
Received: by mail.alexschroeder.ch (Postfix, from userid 1001)
	id BA59610FC; Mon,  3 Aug 2015 10:35:13 +0200 (CEST)

So, the mail went from mail.alexschroeder.ch to eggs.gnu.org, was scanned for spam and forwarded to fencepost.gnu.org which is where my account is, and my .forward file there tells the Gnu mail server to send it on to gmail.com. There, they find that eggs.gnu.org is not allowed to send mail in the name of alexschroeder.ch. Adding the IP6 number 2001:4830:134:3::10 to my SPF record changes that.

Greylisting

sudo apt-get install postfix postgrey

That should install Greylisting.

TLS

Transport Layer Security makes sure nobody can eavesdrop on your emails. This used to be a separate step when I was using Exim. When I switched to Postfix, it was the standard.

pi@raspberrypi ~ $ grep -i tls /etc/postfix/main.cf 
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

Authentication

I’m not sure how this is supposed to work. Looking at the Postfix SASL Howto it would seem that Postfix can use my Dovecot installation. But to be honest, I’m not quite sure I need this.

DKIM

I’m reading How To Install and Configure DKIM with Postfix on Debian Wheezy.

pi@raspberrypi ~ $ sudo apt-get install opendkim opendkim-tools

I then edited /etc/opendkim.conf and added the following:

# How To Install and Configure DKIM with Postfix on Debian Wheezy
# https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy

AutoRestart             Yes
AutoRestartRate         10/1h
UMask                   002
Syslog                  yes
SyslogSuccess           Yes
LogWhy                  Yes

Canonicalization        relaxed/simple

ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts
InternalHosts           refile:/etc/opendkim/TrustedHosts
KeyTable                refile:/etc/opendkim/KeyTable
SigningTable            refile:/etc/opendkim/SigningTable

Mode                    sv
PidFile                 /var/run/opendkim/opendkim.pid
SignatureAlgorithm      rsa-sha256

UserID                  opendkim:opendkim

Socket                  inet:12301@localhost

The instructions mentioned something about editing /etc/default/opendkim and adding SOCKET="inet:12301@localhost" but this seems unnecessary. A comment at the top says Command-line options specified here will override the contents of /etc/opendkim.conf. I’m not doing anything for the moment since I see the «Socket» setting in /etc/opendkim.conf.

I added the following lines to /etc/postfix/main.cf:

milter_protocol = 2
milter_default_action = accept
smtpd_milters = inet:localhost:12301
non_smtpd_milters = inet:localhost:12301

I created the necessary directories:

sudo mkdir -p /etc/opendkim/keys

And I moved the key pair I had created there. These I had created using the instructions elsewhere instead of using opendkim-genkey.

openssl genrsa -out dkim.private 1024
openssl rsa -in dkim.private -out dkim.public -pubout -outform PEM

As far as I can tell, the key cannot be more than 1024 because the TXT record in the zone (see below) is limited in size.

Change ownership and permissions of the private key:

sudo chown opendkim:opendkim dkim.private
sudo chmod 440 keys/dkim.private 

Previously, my permission for the private key had been -rw-r--r-- and I was getting key data is not secure errors in my /var/log/mail.err. as they say, if anyone on the system can change the key, or generate signatures, or steal the key, that’s not good. When I tried to use -r-------- I was getting a different error: can't load key from /etc/opendkim/keys/dkim.private: Permission denied. So now I’m using -r--r-----.

I also added the following line to my zone file:

dkim._domainkey 10800 IN TXT "k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDGMDmbDQEk62j5n8pcHsWNAkrnvhes2g+c2O/gXEytr40E7nqX21YuIDj1e6S07+A8aM/0TSV5nIiWEhOocJtG91wNeM7ypqAgR5zihYoolkXk4FWsdIPbuwZbIEbLBLkgJdD2qbgLLOe/OFMIF3/MQROrfVh+rrxHun2RFJLKDwIDAQAB"

Test it using dig -t TXT dkim._domainkey.alexschroeder.ch. This should tell you whether the record is correct. In addition to that, Check a DKIM Core Key.

Then I created the following files. Note that I’m using the selector dkim instead of mail because of the guide I had followed in an earlier attempt, linked above.

/etc/opendkim/TrustedHosts

127.0.0.1
localhost
192.168.0.1/24

*.alexschroeder.ch
*.communitywiki.org
*.orientalisch.info
*.campaignwiki.org
*.arabisch-lernen.net
*.oddmuse.org
*.emacswiki.org

/etc/opendkim/KeyTable

dkim._domainkey.alexschroeder.ch alexschroeder.ch:dkim:/etc/opendkim/keys/dkim.private
dkim._domainkey.communitywiki.org communitywiki.org:dkim:/etc/opendkim/keys/dkim.private
dkim._domainkey.orientalisch.info orientalisch.info:dkim:/etc/opendkim/keys/dkim.private
dkim._domainkey.campaignwiki.org campaignwiki.org:dkim:/etc/opendkim/keys/dkim.private
dkim._domainkey.arabisch-lernen.org arabisch-lernen.org:dkim:/etc/opendkim/keys/dkim.private
dkim._domainkey.oddmuse.org oddmuse.org:dkim:/etc/opendkim/keys/dkim.private
dkim._domainkey.emacswiki.org emacswiki.org:dkim:/etc/opendkim/keys/dkim.private

/etc/opendkim/SigningTable

*@alexschroeder.ch dkim._domainkey.alexschroeder.ch
*@communitywiki.org dkim._domainkey.communitywiki.org
*@orientalisch.info dkim._domainkey.orientalisch.info
*@campaignwiki.org dkim._domainkey.campaignwiki.org
*@arabisch-lernen.org dkim._domainkey.arabisch-lernen.org
*@oddmuse.org dkim._domainkey.oddmuse.org
*@emacswiki.org dkim._domainkey.emacswiki.org

With all of this done, it’s time to restart our services.

sudo service postfix restart
sudo service opendkim restart

Getting Off Blacklists

I had to request the removal of my Raspberry Pi’s IP number from some blacklists.

Accepting Mail

Sending mail to my various domains resulted in “Relay access denied” messages:

554 5.7.1 <admin@campaignwiki.org>: Relay access denied

I realized what the problem was: What domains to receive mail for needs to list all of them!

In /etc/postfix/main.cf I changed this line:

mydestination = alexschroeder.ch, mail.alexschroeder.ch, communitywiki.org, orientalisch.info, campaignwiki.org, arabisch-lernen.org, oddmuse.org, emacswiki.org, localhost.alexschroeder.ch, localhost

Aliases

This is how most of my mail is handled:

pi@raspberrypi ~ $ cat /etc/aliases
# /etc/aliases
mailer-daemon: postmaster
postmaster: root
nobody: root
hostmaster: root
usenet: root
news: root
webmaster: root
www: root
ftp: root
abuse: root
noc: root
security: root
admin: root
root: alex

Fighting Spam

I’m following Timo’s blog:

apt-get install spamassassin spamc

This pulls a ton of dependencies… And then edit /etc/default/spamassassin, setting ENABLED=1.

Then edit /etc/spamassassin/local.cf and add my contact email. I also experimented with rewrite_header but found that it made scanning my Junk folder harder than necessary. That’s also the reason why I disabled report_safe.

# rewrite_header Subject *****SPAM*****
report_contact kensanata@gmail.com

#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
report_safe 0

No Bayes auto-learning because I’m planning to forward all my mails to Gmail eventually. At that point, no more learning would occur.

Now edit /etc/postfix/master.cf and add a line to smtp as follows:

smtp      inet  n       -       -       -       -       smtpd
        -o content_filter=spamassassin

Also add the following at the end:

spamassassin unix -     n       n       -       -       pipe
        user=debian-spamd argv=/usr/bin/spamc -f -e
        /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Restart everything.

sudo service spamassassin restart
sudo service postfix restart

Moving Spam to Junk Folder

Sadly, this doesn’t work as intended. It should be easy but I have permission problems. I get errors like these in my mail log: dovecot: lda(alex): Error: setegid(privileged) failed: Operation not permitted. I think I’m only getting this error when my mail is not spam and should therefore end in /var/mail/alex. Perhaps I have to move from LDA to LMTP.

This is what I had in /etc/postfix/main.cf:

# mailbox_command = procmail -a "$EXTENSION"
mailbox_command = /usr/lib/dovecot/deliver

deliver is linked to deliver-lda and somehow deliver-lda is not allowed to do its job?

So I switched to dovecot-lmtp.

I followed the instructions on the Dovecot wiki. Running apt-get install dovecot-lmtp already added the protocol via /usr/share/dovecot/protocols.d/lmtpd.protocol.

In /etc/dovecot/conf.d/10-master.conf I made the required change:

service lmtp {
 unix_listener /var/spool/postfix/private/dovecot-lmtp {
   group = postfix
   mode = 0600
   user = postfix
  }
}

No changes were required for /etc/dovecot/conf.d/20-lmtp.conf.

I told Postfix about the change in /etc/postfix/main.cf.

# mailbox_command = procmail -a "$EXTENSION"
# mailbox_command = /usr/lib/dovecot/deliver

# Using Dovecot LMTP
# http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP
mailbox_transport = lmtp:unix:private/dovecot-lmtp

At this point I restarted both services.

sudo service dovecot restart
sudo service postfix restart

But now I was getting a different error when I sent myself a test email:

Aug  9 09:36:40 raspberrypi postfix/lmtp[29502]: 61A5618B6: to=<alex@alexschroeder.ch>, orig_to=<alex>, relay=mail.alexschroeder.ch[private/dovecot-lmtp], delay=0.57, delays=0.19/0.07/0.13/0.18, dsn=5.1.1, status=bounced (host mail.alexschroeder.ch[private/dovecot-lmtp] said: 550 5.1.1 <alex@alexschroeder.ch> User doesn't exist: alex@alexschroeder.ch (in reply to RCPT TO command))

The problem was that I’m using existing system accounts on my server, not virtual accounts. That’s why the user lookup needs to ignore the domain. I made the following change in /etc/dovecot/conf.d/10-auth.conf:

# auth_username_format = %Lu
# fixing username lookup
# http://www.0xf8.org/2012/11/trouble-with-postfix-and-dovecotpam-unknown-user/
auth_username_format = %Ln

The rest of the file remained unchanged. Which is why it still says at the end:

!include auth-system.conf.ext

It seems to work!

My users now all have the following ~/.dovecot.sieve file:

require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
	fileinto "Junk";
}

I wanted to set up server-side email filtering with Dovecot Sieve but I couldn’t find the /etc/dovecot/sieve/default.sieve file it told me to edit. Oh well, I just have to users, so copying the tiny file was no problem. :)

Postfix Queue Management

The commands I’ve used the most often:

mailq
sudo postcat -q 1EF1841F
sudo postsuper -d 1EF1841F

(From here.)

Tags: RSS RSS RSS RSS

Add Comment

2015-07-24 Gmail, Gnus and GPG

I wrote some instructions for setting up Gmail, Gnus and GPG. I used Emacs 24.5 and GPG 2.0 for this tutorial.

If you’re on Windows

I installed gnupg 1.4 using Cygwin and it worked just as well.

If you’re on a Mac

I tried this using my Mac and a Guest account. A Guest account has all its files deleted when you log out. It’s as “clean” as I could get.

You can install Emacs in various ways. You could use Emacs For Mac OS X. “Pure Emacs! No Extras! No Nonsense!” This is what I used to run. I’ve also built Emacs from source many times. Just remember to ./configure --with-ns before you start. You can install it from Homebrew: You could brew install emacs or you could install Yamamoto Mitsuharu’s Emacs Mac Port. This last variant is what I’m using right now.

brew tap railwaycat/emacsmacport
brew install emacs-mac --with-gnutls --with-imagemagick --with-official-icon --with-xml2
brew linkapps emacs-mac

As for GPG, you have two options. You can install GPG via GPG Suite. This is what I used at first. gpg --version said gpg (GnuPG/MacGPG2) 2.0.27 (the current, stable version is 2.0.28). But I wanted to install GPG via Homebrew. If you want to try it as well, make sure you uninstall the GPG Suite if you had it installed.

If you just install gpg2, you won’t be able to enter your passphrase from within Emacs with a GUI. You’ll see a simple, cut off message saying epa-file--find-file-not-found-function: Opening input file: Decryption failed,. If you only use Emacs within terminal windows, no problem. No need to do anything. But if you like to use Emacs with fonts and colors, menus and scrollbars, then you need to install a special variant of pinentry in order to get a separate Pinentry window:

brew install gpg2
brew install pinentry-mac

Once you have done that, change your ~/.gnupg/gpg-agent.conf file so that it has only this one line:

pinentry-program /usr/local/bin/pinentry-mac

When you install gpg2, it tells you to put use-standard-socket in your ~/.gnupg/gpg-agent.conf file. Don’t do that unless you only want to use Emacs from a terminal. Here’s a sample session in a terminal, showing you what you’ll get, if you don’t use pinentry-mac:

alex@Megabombus:~$ echo "This is a test." > test
alex@Megabombus:~$ gpg --recipient kensanata --encrypt test
alex@Megabombus:~$ ls test*
test     test.gpg
alex@Megabombus:~$ gpg --output test2 test.gpg 

You need a passphrase to unlock the secret key for
user: "Alex Schroeder <kensanata@keybase.io>"
8192-bit RSA key, ID 4529A45C, created 2015-03-01 (main key ID ACECFEAE)

gpg: encrypted with 8192-bit RSA key, ID 4529A45C, created 2015-03-01
      "Alex Schroeder <kensanata@keybase.io>"
alex@Megabombus:~$ ls test*
test     test.gpg test2
alex@Megabombus:~$ diff test test2

When asked for your passphrase, you’ll see something like the following:

               ┌────────────────────────────────────────────────────────────────────────────────────┐
               │ Please enter the passphrase to unlock the secret key for the OpenPGP certificate:  │
               │ "Alex Schroeder <kensanata@keybase.io>"                                            │
               │ 8192-bit RSA key, ID 4529A45C,                                                     │
               │ created 2015-03-01 (main key ID ACECFEAE).                                         │
               │                                                                                    │
               │                                                                                    │
               │ Passphrase *****************************************************************______ │
               │                                                                                    │
               │            <OK>                                                  <Cancel>          │
               └────────────────────────────────────────────────────────────────────────────────────┘

This pinentry works in a terminal, but it won’t show up when GPG is called by GUI Emacs! Install pinentry-mac and you’ll be able to provide your passphrase when using Emacs.

If you already tinkered with gpg before changing ~/.gnupg/gpg-agent.conf, you’ll have to reload gpg-agent.

alex@Megabombus:~$ echo RELOADAGENT | gpg-connect-agent
OK

Tags: RSS RSS RSS

Comments on 2015-07-24 Gmail, Gnus and GPG

Today I was able to copy that config file for my Emacs installation at work, use the public keys of both my work account and my private account for mml2015-signers, and it worked! Yay. :)

– Alex Schroeder 2015-11-23 20:24 UTC

Add Comment

More...

Comments


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. See Info for text formatting rules. You can edit the comment page if you need to fix typos. You can subscribe to new comments by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.