Emacs

The pages where I write about Emacs. It doesn’t happen very often. I think you’re probably best off following Planet Emacsen.

2017-05-18 Emacs Wiki Down

Sadly it seems that Emacs Wiki is down. Currently Nic Ferrier is paying for the servers, so I sent him an email but haven’t heard back. So I’m thinking of resurrecting the site on my own servers.

I still remember the fiasco that started with 2014-12-18 Emacs Wiki Migration.

These are my notes. Perhaps they’re useful in case I have to restore another backup, or they might be useful to you if you want to fork Emacs Wiki.

First, restore the backups from Chile provided by zeus. Thanks, man!

rsync -az kensanata@theshire.emacs.cl:emacswiki.org .
rsync -az kensanata@theshire.emacs.cl:emacswiki .

I want to run the script from my Mojolicious Toadfarm. I added the following lines to my Toadfarm setup:

# emacswiki.org

mount "$farm/emacswiki.pl" => {
  "Host" => qr{^emacswiki\.org:8080$},
  mount_point => '/wiki',
};

And this is the Mojolicious CGI Plugin wrapper, emacswiki.pl:

#! /usr/bin/env perl

use Mojolicious::Lite;

plugin CGI => {
  support_semicolon_in_query_string => 1,
};

plugin CGI => {
  route => '/',
  script => '/home/alex/farm/wiki.pl', # not necessary
  errlog => '/home/alex/farm/emacswiki.log',
  run => \&OddMuse::DoWikiRequest,
  before => sub {
    no warnings;
    $OddMuse::RunCGI = 0;
    $OddMuse::DataDir = '/home/alex/emacswiki';
    require '/home/alex/farm/wiki.pl';
  },
};

app->start;

In order for this to work, I need an Apache site. I created /etc/apache2/sites-available/500-emacswiki.org.conf with the following:

<VirtualHost *:80>
    ServerName emacswiki.org
    ServerAlias www.emacswiki.org
    Redirect permanent / https://emacswiki.org/
</VirtualHost>
<VirtualHost *:443>
    ServerAdmin alex@emacswiki.org
    ServerName emacswiki.org
    ServerAlias www.emacswiki.org
    DocumentRoot /home/alex/emacswiki.org
    <Directory /home/alex/emacswiki.org>
        Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
	# legacy CGI scripts like info-ref.pl                                                                                                                
        AddHandler cgi-script .pl
        AllowOverride All
        Require all granted
    </Directory>

    SSLEngine on
    SSLCertificateFile      /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
    SSLCertificateKeyFile   /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
    SSLVerifyClient None

    ProxyPass /wiki             http://emacswiki.org:8080/wiki
    ProxyPass /mojo             http://emacswiki.org:8080/mojo

</VirtualHost>

I remove all the *.pl files except for info-ref.pl in the emacswiki.org directory.

Reloded the farm using ./farm reload. Check the log file for Mounting emacswiki with conditions.

Activating the site using sudo a2ensite 500-emacswiki.org.

Check the config using sudo apachectl configtest. Oops! This is an obvious erorr, of course: SSLCertificateFile: file '/etc/letsencrypt.sh/certs/emacswiki.org/cert.pem' does not exist or is empty.

I need to get the SSL certificates, too.

I added emacswiki.org www.emacswiki.org to /etc/letsencrypt.sh/domains.txt and ran /etc/letsencrypt.sh/letsencrypt.sh -c but that doesn’t work. I guess it doesn’t work because the name emacswiki.org still points to the old server. I guess for the moment I’ll try to do without HTTPS.

So this is what I’ll be using instead for the site:

<VirtualHost *:80>
    ServerName emacswiki.org
    ServerAlias www.emacswiki.org
    ServerAdmin alex@emacswiki.org
    DocumentRoot /home/alex/emacswiki.org
    <Directory /home/alex/emacswiki.org>
        Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
        # legacy CGI scripts like info-ref.pl                                                                                                                
        AddHandler cgi-script .pl
        AllowOverride All
        Require all granted
    </Directory>

    ProxyPass /emacs            http://emacswiki.org:8080/wiki
    ProxyPass /wiki             http://emacswiki.org:8080/wiki
    ProxyPass /mojo             http://emacswiki.org:8080/mojo

</VirtualHost>

Now sudo apachectl configtest says Syntax OK.

Reloaded Apache using sudo service apache2 reload.

Added a line to my /etc/hosts file:

192.71.233.105  emacswiki.org

Testing w3m http://emacswiki.org/wiki and w3m http://emacswiki.org/wiki seems to work!

Better make the wiki read-only: touch ~/emacswiki/noedit.

Following links doesn’t work. w3m http://www.emacswiki.org/emacs/HowToNavigate tells me: Can't load http://www.emacswiki.org/emacs/HowToNavigate. The problem is that Apache has www.emacswiki.org as a server alias, but the Toadfarm only listens for emacswiki.org.

Change that:

# emacswiki.org                                                                                                                                              

mount "$farm/emacswiki.pl" => {
  "Host" => qr{^(www\.)?emacswiki\.org:8080$},
  mount_point => '/wiki',
};

And reload: ./farm reload.

That didn’t work. Hah, of course not. I need to add www.emacswiki.org to my /etc/hosts, of course!

Now it works.

OK, next problem: Why does w3m http://www.emacswiki.org/ give me the directory listing? Surely I’m missing my .htaccess file. Is it not being read? The /var/log/apache2/error.log file has not suspicious. Well, it does mention something about the emacswiki.org/rss directory but I just deleted it. Are the permissions wrong? I did a chmod g-w .htaccess just to be sure and now it says:

-rw-r--r-- 1 alex alex 1955 May 29  2016 emacswiki.org/.htaccess

This looks correct to me.

In there, it says DirectoryIndex emacs.pl. Ah. That might be a problem because I removed that script. Changing that to DirectoryIndex emacs did the job!

OK, so anybody who has access to their own /etc/hosts file can now access a read-only copy of the site.

Here’s what I have planned:

  1. change the DNS entry ✓
  2. see how the site explodes 🔥🔥🔥
  3. add HTTPS

When I tried to add a News page, I noticed that I was unable to get the wiki back into writeable mode. I had to remove the noedit file I had created earlier using rm ~/emacswiki/noedit.

Then, when I tried to save, the wiki complained about some page that looked like spam not being readable and I figured that the page index must have been out of sync so I simply removed it using rm ~/emacswiki/pageidx.

And finally I recreated the lock using touch ~/emacswiki/noedit.

OK, now I’m waiting for the DNS change to spread and watching my Munin graphs.

Also, all the people with HTTPS bookmarks will get errors like the following: Bad cert ident from emacswiki.org: dNSName=alexschroeder.ch rpg.alexschroeder.ch www.alexschroeder.ch : accept? (y/n). That’s because emacswiki.org is currently longer listening on port 443 and the default site is alexschroeder.ch. Oh well! In a few hours I’m hoping that Let’s Encrypt will allow me to regenerate certificates for Emacs Wiki and then we’ll move to HTTPS.

Hours later, I checked again and HTTP access was working. So I ran sudo /etc/letsencrypt.sh/letsencrypt.sh -c to get the certificates and this time it worked. I reverted the changes to the site config file /etc/apache2/sites-available/500-emacswiki.org.conf and we’re no using this:

<VirtualHost *:80>
    ServerName emacswiki.org
    ServerAlias www.emacswiki.org
    Redirect permanent / https://emacswiki.org/
</VirtualHost>
<VirtualHost *:443>
    ServerAdmin alex@emacswiki.org
    ServerName emacswiki.org
    ServerAlias www.emacswiki.org
    DocumentRoot /home/alex/emacswiki.org
    <Directory /home/alex/emacswiki.org>
        Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
	# legacy CGI scripts like info-ref.pl
	AddHandler cgi-script .pl
        AllowOverride All
	Require all granted
    </Directory>

    SSLEngine on
    SSLCertificateFile      /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
    SSLCertificateKeyFile   /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
    SSLVerifyClient None

    ProxyPass /emacs            http://emacswiki.org:8080/wiki
    ProxyPass /wiki             http://emacswiki.org:8080/wiki
    ProxyPass /mojo             http://emacswiki.org:8080/mojo

</VirtualHost>

Notice that both /emacs and /wiki will work. Is this a bad idea? sudo apachectl configtest says the changes are good and so I ran sudo service apache2 reload. Everthing seems to be working!

What about load? It’s definitely going up! :(

OK, time to read up on mod_cache. I think I want something like the following:

# Turn on caching
CacheSocache shmcb
CacheSocacheMaxSize 102400
<Location "/emacs">
    CacheEnable socache
</Location>

Well, before diving into this, I think we should just monitor how load develops over the next few hours.

A few hours later it would seem to me that there are no open issues so there is no need for extra caching.

And that also means, I can try and make the website editable again.

Let’s see, what else do we need to check?

  1. does git work?
  2. what about cronjobs?

As for git, this is simple. I created the page 2017-05-18 and I expect to see it on the emacsmirror master branch. Sadly, it isn’t there. Why not?

Let’s take a look:

alex@sibirocobombus:~/emacswiki/git$ git log
commit a08f867084896e9892d148f76a54976166cd75db
Author: Alex Schroeder <anybody@emacswiki.org>
Date:   Thu May 18 13:56:01 2017 +0200

    Backup site!

Oops! Apparently, the git repository wasn’t checked out. It makes sense, actually. But now I need to fix this. git remote -v shows no remotes. Let’s add it, and fetch data. This works because my public key is already part of the emacswirror org on GitHub.

git remote add origin git@github.com:emacsmirror/emacswiki.org.git
git fetch
git branch -m master temp
git checkout master

At this point it should tell you Branch master set up to track remote branch master from origin.

git cherry-pick a08f86
git push
git branch -D temp

OK. Time to test it! In order to be able to save, I now have to change the site URL in the config file back to HTTPS. It should read

my $root = "https://www.emacswiki.org";

I made the page edit and that seems to do the trick. git log in the git directory lists the new edit.

This brings me to the next part: cron jobs. Somebody has to push those commits, right?

  1. I added emacswiki and emacswiki.org to the shell script that uses rsync to store daily backups in Chile. Thanks again, zeus!
  2. I found an old emacs-git-update in my bin directory and added an appropriate entry to my crontab using crontab -e.
  3. I found an old maintain-emacswiki in my bin directory, fixed it, and also added it to crontab. I definitely need to check the maintenance page a few times over the next few days.
  4. I found an old update-ell in my bin directory and decided to check the XML file referenced. The timestamp says Wed 24 Dec 2014 11:36:00 GMT so I think it’s safe to say that this won’t be required anymore.
  5. I did not find a copy of the emacs-elisp-area script. I checked the code in my config file and now I remember: this job used to call the expensive Elisp Area URLs and save the result to disk, and then URL rewriting made sure that the Elisp Area URLs called from the web would read those files instead. I just tried those links (”Alphabetical list of all Elisp libraries on this Wiki” with and without context, and “Chronological list of all Elisp libraries on this Wiki”) and it seems to work just fine. It takes a few seconds, but nothing terrible. I’ll say that this won’t be required anymore.
  6. I found an old copy of emacs-rss in my bin directory. That one precomputes some resource intensive RSS feeds. I should definitely get those back into working condition. When I run it, the four files are generated, and they’re the four RSS feed advertised in the HTML of the wiki, so that’s great.

And that’s all the jobs I found in an old crontab file!

Current status, one day later:

The only suspicious thing is the spike around 2:30 in the morning. But the explanation might be simple enough, looking at my crontab:

#m   h  dom mon dow   command
 02  5  *   *   *     /home/alex/bin/maintain-campaignwiki
 47 4,16 *  *   *     /home/alex/bin/backup
 28  4  *   *   *     /home/alex/bin/subscriptions
 14  3  *   *   *     /home/alex/bin/emacs-git-update
 32  2  *   *   *     /home/alex/bin/maintain-emacswiki

At 2:32, the maintenance job runs. The curl output is available in the maintenance directory:

--2017-05-19 02:32:01--  https://www.emacswiki.org/emacs
Resolving www.emacswiki.org (www.emacswiki.org)... 192.71.233.105
Connecting to www.emacswiki.org (www.emacswiki.org)|192.71.233.105|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘/home/alex/emacswiki.org/maintenance/last.html’

     0K .......... .......... .......... .......... .......... 43.1K
    50K .......... .......... .......... .......... .......... 11.3K
   100K .......... .......... .......... .......... .......... 3.66K
   150K .......... .......... .......... .......... .......... 21.3K
   200K .......... .......... .......... .......... .......... 35.7K
   250K .......... .......... .......... .......... .......... 7.86K
   300K .......... .......... .......... .......... .......... 12.4K
   350K .......... .......... .......... .......... .......... 22.8K
   400K .......... .......... .......... .......... .......... 19.3K
   450K .......... .......... .......... .......... .......... 3.64K
   500K .......... .......... .......... .......... .......... 5.91K
   550K .......... .......... .......... .......... .......... 14.6K
   600K .......... .......... .......... .......... .......... 10.3K
   650K .......... .......... .......... .......... .......... 15.4K
   700K .......... .......... .......... .......... .......... 20.5K
   750K .......... .......... .......... .......... .......... 15.3K
   800K .......... .......... .......... .......... .......... 15.3K
   850K .......... .......... .......... .......... .......... 15.4K
   900K .......... .......... .......... ......                18.5K=86s

2017-05-19 02:34:03 (10.9 KB/s) - ‘/home/alex/emacswiki.org/maintenance/last.html’ saved [959329]

I’m guessing that these two minutes are causing the spike.

When I did some testing with the CSS, I ran into problems. If you choose a different theme via the CSS page, it gets stored in a cookie. Cookies are specific to a site, and so cookies set on http://emacswiki.org, https://emacswiki.org, http://www.emacswiki.org and https://www.emacswiki.org are separate from each other. This will not do. I’ve now changed the Apache config file to create the appropriate redirections. At the same time, I wanted to clean up the https://www.emacswiki.org/wiki vs. https://www.emacswiki.org/emacs situation.

<VirtualHost *:80>
    ServerName www.emacswiki.org
    ServerAlias emacswiki.org
    Redirect permanent / https://www.emacswiki.org/
</VirtualHost>
<VirtualHost *:443>
    ServerName emacswiki.org
    Redirect permanent / https://www.emacswiki.org/
    SSLEngine on
    SSLCertificateFile      /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
    SSLCertificateKeyFile   /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
    SSLVerifyClient None
</VirtualHost>
<VirtualHost *:443>
    ServerAdmin alex@emacswiki.org
    ServerName www.emacswiki.org
    DocumentRoot /home/alex/emacswiki.org
    <Directory /home/alex/emacswiki.org>
        Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
        # legacy CGI scripts like info-ref.pl                                                               
        AddHandler cgi-script .pl
        AllowOverride All
        Require all granted
    </Directory>
    SSLEngine on
    SSLCertificateFile      /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
    SSLCertificateKeyFile   /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
    SSLVerifyClient None
    Redirect permanent /wiki    https://www.emacswiki.org/emacs
    ProxyPass /emacs            http://emacswiki.org:8080/wiki
    ProxyPass /mojo             http://emacswiki.org:8080/mojo
</VirtualHost>

I wonder whether it’s important to prevent outside access to http://emacswiki.org:8080/wiki. I see no problem?

Tags:

Comments on 2017-05-18 Emacs Wiki Down

Heroic effort, and amazing notes! Thanks for Emacs Wiki!

AaronHawley 2017-05-19 01:27 UTC


Thanks!

– Alex Schroeder 2017-05-19 07:04 UTC

Add Comment

2017-02-03 Homebrew and Emacs

Today:

alex@Megabombus:~$ emacs
dyld: Library not loaded: /usr/local/opt/imagemagick/lib/libMagickWand-6.Q16.3.dylib
  Referenced from: /usr/local/bin/emacs
  Reason: image not found
Abort trap: 6

OK, reinstall...

alex@Megabombus:~$ brew reinstall emacs-mac
==> Reinstalling railwaycat/emacsmacport/emacs-mac --with-xml2 --with-official-icon --with-gnutls --with-imagemagick
==> Downloading https://bitbucket.org/mituharu/emacs-mac/get/emacs-25.1-mac-6.1.tar.gz
Already downloaded: /Users/alex/Library/Caches/Homebrew/emacs-mac-emacs-25.1-z-mac-6.1.tar.gz
==> Downloading https://s3.amazonaws.com/emacs-mac-port/Emacs25.icns
Already downloaded: /Users/alex/Library/Caches/Homebrew/emacs-mac--official-icon-25.icns
==> ./autogen.sh
==> ./configure --enable-locallisppath=/usr/local/share/emacs/site-lisp --infodir=/usr/local/Cellar/emacs-mac/emacs-25.1-z-mac-6.1/shar
==> make
...
==> Summary
🍺  /usr/local/Cellar/emacs-mac/emacs-25.1-z-mac-6.1: 4,067 files, 109.7M, built in 20 minutes 31 seconds

Tags:

Add Comment

2016-12-08 Cider Woes

I’m trying to get Cider to work on my laptop running OSX and failing. This makes me unhappy because I managed to do it on a Windows machine with lots of Cygwin so I felt that getting it to work on a Mac should be a breeze.

Basically, what I have set up my little project using lein and can run it from the command line and from eshell, and I use Clojure Mode, but when I run M-x cider-jack-in I see is this in my *Messages* buffer:

Starting nREPL server via /Users/alex/bin/lein update-in :dependencies conj \[org.clojure/tools.nrepl\ \"0.2.12\"\ \:exclusions\ \[org.clojure/clojure\]\] -- update-in :plugins conj \[cider/cider-nrepl\ \"0.15.0-SNAPSHOT\"\] -- repl :headless...
Mark set
nREPL server started on 53929
[nREPL] Establishing direct connection to localhost:53929 ...
error in process filter: nrepl--direct-connect: [nREPL] Direct connection failed
error in process filter: [nREPL] Direct connection failed

But now I have three buffers named *nrepl-server ... with contents such as nREPL server started on port 53872 on host 127.0.0.1 - nrepl://127.0.0.1:53872.

This reminds me... My Mac keeps deleting my /etc/hosts file which means that localhost is borked:

$ ping localhost
ping: cannot resolve localhost: Unknown host

So I added it back to /etc/hosts.

127.0.0.1 localhost

Then I killed the three nrepl-server buffers and tried C-c M-j again.

And it works. Phew!

Sometimes writing these posts just helps me focus as I’m troubleshooting.

Tags:

Comments on 2016-12-08 Cider Woes

Your mac deleting etchosts file sounds really bad

– nickf 2016-12-08 22:53 UTC


Yeah. I think it rewrites the file whenever the system updates? So any code that depends on localhost mapping to 127.0.0.1 usually breaks after an update. Gah!

– Alex Schroeder 2016-12-08 22:59 UTC


Hi Alex, that deletion of etchosts sounds vary odd. I use emacs, cider, lein etc almost daily on OSX and have not run itno that issue ever. I do use hombrew for emacs and lein. I am now running macOSSierra, but have been doing clojure et. al. on a Macbook for the past few years and used the last2 or 3 OSX versions.

– Tim X 2016-12-09 08:30 UTC


Strange. I also have OpenVPN installed on the Mac and it puts stuff into the file. Googling for the problem shows that some versions of a Cisco VPN used to do the same. I should just deinstall it and see whether that resolves it.

– AlexSchroeder 2016-12-09 11:47 UTC

Add Comment

2016-12-02 Emacs Avent of Code Day 2

First question of day 2: You start on key 5 on a keypad as shown below. With instructions to go up, down, left or right. Ignore instructions that would take you off the keypad.

1 2 3
4 5 6
7 8 9

The example input provided for the four numbers is shown below. At the end of each line you’ll end up on the key which is part of the answer. In this example the correct answer is 1985.

ULL
RRDDD
LURDL
UUUUD

Code:

(let ((pos 5)
      (code nil)
      (instructions '((U L L)
		      (R R D D D)
		      (L U R D L)
		      (U U U U D))))
  (dolist (instruction instructions)
    (dolist (move instruction)
      (case move
	((U) (when (> pos 3) (setq pos (- pos 3))))
	((D) (when (< pos 7) (setq pos (+ pos 3))))
	((L) (when (not (= (mod pos 3) 1)) (setq pos (- pos 1))))
	((R) (when (> (mod pos 3) 0) (setq pos (+ pos 1))))))
    (push pos code))
  (reverse code))

For the second star, I was too lazy to figure out how to compute the next position and decided to implement tables instead:

    1
  2 3 4
5 6 7 8 9
  A B C
    D

Given a direction, I just list all the possible positions and the results. It’s crude, but it works.

(let ((pos 5)
      (code nil)
      (instructions '((U L L)
		      (R R D D D)
		      (L U R D L)
		      (U U U U D))))
(dolist (instruction instructions)
    (dolist (move instruction)
      (setq pos (funcall move pos)))
    (push pos code))
  (reverse code))

(defun U (pos)
  (case pos
    ((3) 1)
    ((6) 2)
    ((7) 3)
    ((8) 4)
    ((A) 6)
    ((B) 7)
    ((C) 8)
    ((D) 'B)
    (t pos)))

(defun D (pos)
  (case pos
    ((1) 3)
    ((2) 6)
    ((3) 7)
    ((4) 8)
    ((6) 'A)
    ((7) 'B)
    ((8) 'C)
    ((B) 'D)
    (t pos)))

(defun L (pos)
  (case pos
    ((6) 5)
    ((3) 2)
    ((7) 6)
    ((B) 'A)
    ((4) 3)
    ((8) 7)
    ((C) 'B)
    ((9) 8)
    (t pos)))

(defun R (pos)
  (case pos
    ((5) 6)
    ((2) 3)
    ((6) 7)
    ((A) 'B)
    ((3) 4)
    ((7) 8)
    ((B) 'C)
    ((8) 9)
    (t pos)))

Tags:

Add Comment

2016-12-01 Emacs Advent of Code

I looked at Advent of Code and foolishly decided to solve the first riddle. Perhaps we’ll learn something about Elisp coding style? Feel free to leave comments regarding the code.

You must follow instructions: R or L means turn left or right, followed by a number of steps to take. What’s the final distance in steps? Example: “R5, L5, R5, R3 leaves you 12 blocks away.”

(let ((instructions (mapcar (lambda (s)
			      (cons (aref s 0)
				    (string-to-number (substring s 1))))
			    (split-string "R5, L5, R5, R3")))
      (direction 0)
      (x 0)
      (y 0))
  (dolist (instruction instructions)
    (destructuring-bind (turn . steps) instruction
      (setq direction (mod (funcall (if (= turn ?R) '1+ '1-) direction) 4))
      (case direction
	(0 (setq y (+ y steps)))
	(1 (setq x (+ x steps)))
	(2 (setq y (- y steps)))
	(3 (setq x (- x steps))))))
  (+ (abs x) (abs y)))

Sadly, for the second star, my solution fails... The question is now: which location was visited twice? The example provided says “if your instructions are R8, R4, R4, R8, the first location you visit twice is 4 blocks away, due East.” The code works for the example given but fails for the data I was provided with in the test.

(let ((instructions (mapcar (lambda (s)
			      (cons (aref s 0)
				    (string-to-number (substring s 1))))
			    (split-string "R8, R4, R4, R8")))
      (direction 0)
      (x 0)
      (y 0)
      (last-x 0)
      (last-y 0)
      (seen nil)
      (final-x nil)
      (final-y nil))
  (or (catch 'twice
	(dolist (instruction instructions)
	  (destructuring-bind (turn . steps) instruction
	    (setq direction (mod (funcall (if (= turn ?R) '1+ '1-) direction) 4))
	    (case direction
	      (0 (setq y (+ y steps)))
	      (1 (setq x (+ x steps)))
	      (2 (setq y (- y steps)))
	      (3 (setq x (- x steps)))))
	  (dolist (line (cdr seen))	; skip the last element
	    (destructuring-bind ((x1 . y1) (x2 . y2)) line
	      (when (or (and (= x last-x) ; last move was vertical 
			     (= y1 y2)	  ; looking at a horizontal
			     (>= x x1)  (<= x x2)
			     (or (and (<= y y1)  (>= last-y y1))
				 (and (>= y y1)  (<= last-y y1)))
			     (setq final-x x
				   final-y y1))
			(and (= y last-y) ; last move was horizontal
			     (= x1 x2)	  ; looking at a vertical
			     (>= y y1)  (<= y y2)
			     (or (and (<= x x1)  (>= last-x x1))
				 (and (>= x x1)  (<= last-x x1)))
			     (setq final-x x1
				   final-y y)))
		;; (error "Moving from %S to %S crosses the line from %S to %S in %S"
		;; 	   (cons last-x last-y) (cons x y)
		;; 	   (cons x1 y1) (cons x2 y2)
		;; 	   seen)
		(throw 'twice (+ (abs final-x) (abs final-y))))))
	  (push (list (cons last-x last-y) (cons x y)) seen)
	  (setq last-x x
		last-y y)))
      (+ (abs x) (abs y))))

Well, I decided to brute force it and remember every point I moved through because my “crossing lines” solution was obviously off by 3. Perhaps I just reported the wrong result, who knows.

(let ((instructions (mapcar (lambda (s)
			      (cons (aref s 0)
				    (string-to-number (substring s 1))))
			    (split-string "R8, R4, R4, R8")))
      (direction 0)
      (x 0)
      (y 0)
      (last-x 0)
      (last-y 0)
      (seen '((0 . 0))))
  (or (catch 'twice
	(dolist (instruction instructions)
	  (destructuring-bind (turn . steps) instruction
	    (setq direction (mod (funcall (if (= turn ?R) '1+ '1-) direction) 4))
	    (case direction
	      (0 (setq y (+ y steps)))
	      (1 (setq x (+ x steps)))
	      (2 (setq y (- y steps)))
	      (3 (setq x (- x steps)))))
	  (while (not (and (= last-x x) (= last-y y)))
	    (if (not (= last-x x))
		(setq last-x (funcall (if (< last-x x) '1+ '1-) last-x))
	      (setq last-y (funcall (if (< last-y y) '1+ '1-) last-y)))
	    (let ((last (cons last-x last-y)))
	      (if (member last seen)
		  (error "Repetition seen: %S in %S" last seen)
		(push last seen))))))
      (error "No repetitions in %S" seen)))

Tags:

Add Comment

2016-09-21 OSX Emacs

Same procedure as always, one would think. But somehow, it manages to be different every single time.

This is what I want:

alex@Megabombus:~$ brew install railwaycat/emacsmacport/emacs-mac --with-gnutls --with-imagemagick --with-official-icon --with-xml2
==> Installing emacs-mac from railwaycat/emacsmacport
...

Homebrew also wanted me to do some other stuff, oh well.

alex@Megabombus:~$ sudo chown root:wheel /usr/local
alex@Megabombus:~$ sudo mv /opt/homebrew-cask/Caskroom /usr/local

Tags:

Comments on 2016-09-21 OSX Emacs

Today...

alex@Megabombus:~$ brew update
Checking out v1.0.6 in /usr/local/Homebrew...
To checkout master in /usr/local/Homebrew run:
  'cd /usr/local/Homebrew && git checkout master
Checking out v1.0.6 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.6' at the same time.
...

And many repeats. WTF!

Google found an answer on Stack Exchange that seemed similar enough for me to try it:

alex@Megabombus:~$ cd $(brew --repo); git fetch; git reset --hard origin/master; brew update
HEAD is now at faebc64 Merge pull request #1216 from reitermarkus/fix-language-audit
Updated 11 taps (caskroom/cask, homebrew/boneyard, homebrew/core, homebrew/dupes, homebrew/fuse, homebrew/games, homebrew/science, homebrew/services, homebrew/tex, homebrew/versions, homebrew/x11).
...

Looks good!

Also, I learned that now --enable-mac-app is required...

And so:

alex@Megabombus:~$ brew install railwaycat/emacsmacport/emacs-mac --with-gnutls --with-imagemagick --with-official-icon --with-xml2 --enable-mac-app
Warning: railwaycat/emacsmacport/emacs-mac-emacs-25.1-z-mac-6.0 already installed
alex@Megabombus:~$ brew remove railwaycat/emacsmacport/emacs-mac
Uninstalling /usr/local/Cellar/emacs-mac/emacs-25.1-z-mac-6.0... (4,066 files, 109.9M)
emacs-mac emacs-24.5-z-mac-5.17, emacs-24.5-z-mac-5.18 are still installed.
Remove all versions with `brew uninstall --force emacs-mac`.
alex@Megabombus:~$ brew uninstall --force emacs-mac
Uninstalling emacs-mac... (7,874 files, 225.7M)
alex@Megabombus:~$ brew install railwaycat/emacsmacport/emacs-mac --with-gnutls --with-imagemagick --with-official-icon --with-xml2 --enable-mac-app
==> Installing emacs-mac from railwaycat/emacsmacport
==> Installing dependencies for railwaycat/emacsmacport/emacs-mac: nettle, imagemagick
==> Installing railwaycat/emacsmacport/emacs-mac dependency: nettle
...
==> Caveats
This is YAMAMOTO Mitsuharu's "Mac port" addition to
GNU Emacs 25. This provides a native GUI support for Mac OS X
10.6 - 10.12. After installing, see README-mac and NEWS-mac
in /usr/local/Cellar/emacs-mac/emacs-25.1-z-mac-6.0 for the port details.

Emacs.app was installed to:
  /usr/local/Cellar/emacs-mac/emacs-25.1-z-mac-6.0

To link the application to default Homebrew App location:
  brew linkapps
or:
  ln -s /usr/local/Cellar/emacs-mac/emacs-25.1-z-mac-6.0/Emacs.app /Applications
Other ways please refer:
  https://github.com/railwaycat/homebrew-emacsmacport/wiki/Alternative-way-of-place-Emacs.app-to-Applications-directory

For an Emacs.app CLI starter, see:
  https://gist.github.com/4043945

.app bundles were installed.
Run `brew linkapps emacs-mac` to symlink these to /Applications.
==> Summary
🍺  /usr/local/Cellar/emacs-mac/emacs-25.1-z-mac-6.0: 4,067 files, 109.8M, built in 15 minutes 56 seconds
alex@Megabombus:~$ brew linkapps emacs-mac
Linking: /usr/local/opt/emacs-mac/Emacs.app
Linked 1 app to /Applications

– Alex Schroeder 2016-10-06 19:47 UTC

Add Comment

2016-05-28 nginx as a caching proxy

Remember the fiasco at the end of 2014 and the beginning of 2015, when Emacs Wiki was down as I tried to switch to Fast CGI, and it kept staying down as I tried to use mod_perl, when finally Nic Ferrier stepped in and started paying for new hosting. He used nginx as a caching proxy for Apache, which continued to call the wiki as a simply CGI script.

The comments reveal quite a bit of caching issues, though. The problem was that nginx seemed to ignore that two pages with an identical URL but for different language preferences could be different. This in turn meant that a German visitor to a page not in the cache would put the page in the cache but with German menus at the bottom, for example (”Diese Seite bearbeiten” instead of “Edit this page”). And if the next visitor to the same page didn’t understand German, this was annoying.

I thought the answer was that Apache should send the “Vary” header telling the cache that accept-language should be part of the cache key. See this discussion of the Vary header, for example: Understanding the HTTP Vary Header and Caching Proxies. But it just didn’t work.

Recently, I suddenly realized that perhaps nginx doesn’t care about the Vary header Apache is sending. I learned about the proxy_header_key setting in the nginx caching guide. As you can see, the default is inadequate! $scheme$proxy_host$request_uri is not what I’m expecting. I needed to add http_accept_language to that key:

1. proxy cache key is $scheme$proxy_host$request_uri by default
proxy_cache_key $scheme$proxy_host$request_uri$http_accept_language;

I get the feeling that things are working, right now! If so, it only took 17½ month to find the solution. I’m writing it down in case anybody else stumbles over this problem (and in case I need to remember in the future).

Perhaps the problem is also related to how I compute Etags for my pages: the wiki doesn’t take accept-language into account when computing Etags. Now that I think about it, perhaps that would also have helped solve the problem?

Tags:

Add Comment

2016-05-18 Renaming Files

I had a bunch of files named like this, from various albums.

Cure - [1984] The top - 01 - Shake dog shake.mp3
Cure - [1984] The top - 02 - Bird mad girl.mp3
Cure - [1984] The top - 03 - Wailing wall.mp3

I wanted to move them into subdirectories, one for each album.

(dolist (file (directory-files "c:/Users/asc/Music/The Cure" t "\\.mp3$"))
  (let* ((name (file-name-nondirectory file))
	 (data (split-string name " - "))
	 (album (nth 1 data)))
    (unless (file-directory-p (concat "c:/Users/asc/Music/The Cure/" album))
      (make-directory (concat "c:/Users/asc/Music/The Cure/" album)))
    (rename-file file (concat "c:/Users/asc/Music/The Cure/" album "/" name))))

Tags:

Comments on 2016-05-18 Renaming Files

If your files have ID3 tags, you can do some thing like this, too:

exiftool '-filename<${Artist;} - ${Year} - ${Album}/${Track;} ${Artist;} - ${Title;}.%e' **.mp3

– Sam 2016-05-18 11:54 UTC


Nice! And using exiftool! :)

– Alex Schroeder 2016-05-18 13:17 UTC


A graphic alternative is exfalso (from quodlibet music player).

– Anonymous 2016-05-19 09:58 UTC


Here is a way to do it combining dired, wdired and multiple-cursors. This approach requires the existence of the album directory which might not make it particularly useful for your exact use case. Still, I find it cool and so should you :-).

Damien Cassou 2016-05-19 18:10 UTC


Very cool. And a nice video on top. Thumbs up!

– Alex Schroeder 2016-05-21 06:33 UTC

Add Comment

2016-05-15 Gmail, Gnus, GPG Guide Update

I updated my Gmail, Gnus, GPG Guide. It now includes all the stuff about setting up the GPG Agent.

The GPG Agent is a service that will remember your passphrase for a short while. If you don’t use it, Gnus will ask you for your passphrase for every backend it uses (because it needs to log in) and for every encrypted mail you read and for every encrypted mail you send. You’ll be typing your passphrase a lot. You want to use it.

Please let me know if you tried to follow the instructions and it didn’t work. I’d like to make the instructions very easy to follow and troubleshoot. So if you missed something and ran into errors, I’d still like to know so that I can start adding typical problems people might have and how to resolve them.

And finally, feel free to test this system by sending me encrypted email! See Contact for more.

Tags:

Add Comment

2016-04-11 Atreus, Windows, and Emacs

So, I want to define an interesting layout for my Atreus keyboard. On OSX, I used Ukulele to do it. On Windows, I’m using Microsoft Keyboard Layout Creator 1.4.

+---+---+---+---+---+          +---+---+---+---+---+
|Q  |W <|E >|R (|T )|          |Z {|U }|I [|O ]|P €|
|q 6|w 7|e 8|r 9|t 0|          |z 1|u 2|i 3|o 4|p 5|
+---+---+---+---+---+          +---+---+---+---+---+
|A  |S  |D  |F  |G $|          |H ^|J &|K  |L ×|: ||
|a  |s  |d  |f  |g @|          |h =|j +|k -|l *|. …|
+---+---+---+---+---+          +---+---+---+---+---+
|Y  |X  |C  |V  |B %|          |N ~|M 」|_ —|´ ’|` ”|
|y  |x  |c  |v  |b #|          |n ¨|m 「|- –|' ‘|" “|
+---+---+---+---+---+---+  +---+---+---+---+---+---+
|   |   |   |   |DEL|   |  |   |   |   |; »|? \|   |
|ESC|TAB|SUP|SH |BCK|CTR|  |ALT|SPC|FN |, «|! /|RET|
+---+---+---+---+---+---+  +---+---+---+---+---+---+

When I switch to my new layout, I still have some problems with it:

  • I was able to define my extra stuff using AltGr, which is the equivalent of Alt+Ctrl at the same time, which means that under Windows, I need to press both of them to access @ on g, for example. Is there a way to make this work for both Mac and Windows? Change the Alt key to Right Alt and it’ll work?
  • When using Emacs, I get A-C-g is undefined. Other programs, like Firefox, will correctly interpret A-C-g as AltGr + g and give me the @. What can I do to fix this?
  • When using Emacs, I want to use the windows key as meta. On my keyboard, that’s SUP (Super). But in this case, I get some Windows action. So I start experimenting.
  (setq w32-lwindow-modifier 'meta ; windows is meta
	w32-pass-lwindow-to-system nil
	w32-rwindow-modifier 'meta ; windows is meta
	w32-pass-rwindow-to-system nil
	w32-recognize-altgr t
	w32-pass-alt-to-system t   ; alt is alt
	w32-alt-is-meta nil)
   ;; w32-phantom-key-code??

Perhaps it would be easier if I changed that key to Meta using the keyboard’s firmware and it would work correctly on both OSX and Windows?

And I’m still not sure how to change bindings for simple keys using the Alt key. Perhaps I need a different utility? A good Microsoft Keyboard Layout Creator alternative...

I guess the alternative would be to use Fn together with these extra keys. But then I can’t bind them to something interesting like “...” because in order to that, I would need access to these keys on Microsoft Keyboard Layout Creator, which I do not.

Tags:

Comments on 2016-04-11 Atreus, Windows, and Emacs

Well, right now I am back at the US standard layout. Perhaps I can do everything else using dead keys? Might be simpler than all this extra fiddling. I’m hating everything about it already. I’m spending too much time worrying about programming the keyboard but as it stands, I won’t ever get a system, where I can just carry my keyboard to another desk and start typing. It just requires too many features by the operating system and the various solutions are incompatible anyway.

Current mood: frustrated.

– Alex Schroeder 2016-04-11 19:40 UTC


I want a keyboard with Super and Hyper that works on both Windows and OS X. I still don’t have a precise answer but this seems good ot know: https://ergodox-ez.com/pages/our-firmware They define a “Hyper” and a “Meh” key. I pre(re)invented that approach using AutoHotKey and duplicated bindings, and decided it was a poor use of my time. If it is built into your keyboard, however, might be nice. I still want to answer “Why can’t we have a GUI key that is separate from the SUPER key?”. Seems possible, and I would like it a lot because we do need a GUI key, too.

Grant Rettke 2016-04-11 21:00 UTC


Oh! Right. Whenever I have to use another computer I usually connect to it with my laptop using, er, Synergy. Synergy is great, but… it is an interesting case of a GPL-ed free software getting crippled so that they can sell it…

So I guess that it is just waiting for a fork to appear. Meanwhile you can get a bit older version from here: http://synergy-project.org/download/free/ (or compile it yourself, though in case of crippled software it is usually not as simple. But who knows). Same version is debian repos (so I haven’t really noticed).

It’s kinda interesting that instead of starting some crowd funding they decided to turn the software into something that I don’t really want to recommend to anybody just because of their “monetization“ shit. Awful.

But! It allows me to use my laptop (and therefore my own configuration) for any computer I connect with. So, perhaps that’s an option?

– AlexDaniel 2016-04-11 21:09 UTC


Grant, I saw you blog post just now and I’m reminded of what I discovered when I looked at the keycodes available in the firmware:

/* Modifiers */
KC_LCTRL            KC_LCTRL        E0 Keyboard LeftControl
KC_LSHIFT           KC_LSFT         E1 Keyboard LeftShift
KC_LALT                             E2 Keyboard LeftAlt
KC_LGUI                             E3 Keyboard Left GUI(Windows/Apple/Meta key)
KC_RCTRL            KC_RCTL         E4 Keyboard RightControl
KC_RSHIFT           KC_RSFT         E5 Keyboard RightShift
KC_RALT                             E6 Keyboard RightAlt
KC_RGUI                             E7 Keyboard Right GUI(Windows/Apple/Meta key)

No KC_RHYPER or KC_LHYPER! No KC_RSUPER or KC_LSUPER! The comment points to another document: “Keycodes are defined in ’common/keycode.h’. Range of 00-A4 and E0-E7 are identical with HID Usage.” In the header file mentioned, it says: “Virtual keycodes are defined out of above range to support special actions. Keycodes based on HID Usage Keyboard/Keypad Page(0x07) plus special codes.” As the URLs are no longer pointing anywhere, here’s what I think this refers to the USB HID Usage Tables on page 53ff.

I think this means that there cannot be a USB keyboard with a Hyper key. I think what you therefore need to do is make a key that sends multiple modifiers at the same time, ie. mapping Caps Lock to Shift+Ctrl+Option+Command or something like that. The solution for a Mac involves Karabiner and Seil for OSX. Like the ideas I had myself, this setup is hardly portable to other computers, unfortunately. Apparently, you can do something similar using AutoHotKey.

Xah Lee has collected some info the topic: Emacs: How to Define Super Hyper Keys, AutoHotkey Tutorial.

– Alex Schroeder 2016-04-12 07:31 UTC


How is it going?

– AlexDaniel 2016-04-24 06:23 UTC


I’ve been using the Atreus with my la-top at home, using the US standard layout. I use the Latin Prefix input method to write German. This also meant that I no longer require Alt/Option as AltGr and so I use Alt as Hyper again, but have no keys bound using Hyper. Boooo.

– Alex Schroeder 2016-04-24 16:37 UTC


Alex can you show your keyboard layout (like you did up above) showing your current mappings?

I’m confused by your post: did you get what you wanted when you re-configured left and right GUI?

I guessed that you would place LEFT_ALT and where you wanted it to be META, and RIGHT_ALT wherever you wanted it to be ALT. It would look confusing because the keycaps would have ALT all over, but for Emacs, it would be exactly what you want. Right? I think that I might have understood something wrong here.

Grant Rettke 2016-05-14 19:00 UTC


The end result of all the above is that I am using the default firmware for the keyboard. This means that I’m short one modifier on the Mac: I need the command key to run OSX commands; I need Alt to enter German characters; there’s no key for Meta. Under OSX, the usual solution is to use the Command key as Meta, of possible. So, in order to get that, I would have to swap Alt and Super on the Atreus firmware, assuming I wanted to use my right thumb for Meta. I guess I do?

But the main thing that I wanted to talk about when I replied to your message above was this: All USB keyboards will be limited by the USB standard for keyboards. You can find the necessary information in USB HID Usage Tables on page 53ff. There, on page 59, you’ll note that there is no way a USB keyboard will be able to send the Hyper modifier: There is only Control, Shift, Alt, and “GUI” (Super) for left and right. In order to get Hyper in addition to those present, you’ll have to map your key to something like F13 in the firmware and then you’ll have tell your operating system that F13 should act as Hyper. I think I could figure this out for a GNU/Linux environment, but I’m unwilling to spend the time necessary to figure it out for OSX and Windows.

And so I determined that the two easiest options right now are the following:

  1. don’t use the Atreus keyboard
  2. if I use the Atreus keyboard, use the default firmware, switch to a US layout in OSX, and use something like the following for Emacs:
(defun atreus-layout ()
  "If using the Atreus on a Mac, we want the Alt key to be Meta."
  (interactive)
  (setq mac-option-modifier 'meta)
  (setq default-input-method 'german-prefix)
  (set-input-method 'german-prefix))

But, I don’t like any of it and I’m unhappy with the current state of affairs. The sad part is that I don’t see an easy way forward. Maybe during my summer break.

– Alex Schroeder 2016-05-15 00:19 UTC


Gotcha. I am sorry that it isn’t going as planned. Here is what I tried https://www.wisdomandwonder.com/article/10146/every-emacser-can-use-hyper-on-every-usb-hid-keyboard . This is how I came up with “using hyper in Emacs” given that there are only 8 modifiers (left or right, shift control alt gui) in the USB HID spec and there is no xmodmap option for Windows. It seemed happy on my Mac and Windows box; I tried the following on both boxes. The important thing for me was that Windows and OS X had their important modifiers: ALT and GUI so I could switch window focus and use OS actions like inserting special characters and opening the Run dialog in Windows. I read your post like it would work the same way for the Altreus. The way I read your post I figured that you MIGHT modify your default keyboard https://atreus.technomancy.us/ by changing the default layout (by names there): super as RIGHT_GUI (aka option), ctrl as RIGHT_CONTROL, alt as RIGHT_ALT (aka command), and then moving - ’ , to the Fn layer, so you can redefine - as LEFT_CONTROL (Meta), ’ as LEFT_ALT (Super), and , as LEFT_GUI (Hyper) only for Emacs using the mac-right-*-modifier and mac-left-*-modifier settings. I didn’t want the CONTROL-ALT-GUI-SHIFT style hyper key since I only want these modifiers in Emacs. I tried that approach and didn’t like it only because I only wanted the modifiers inside of Emacs. If things work like I am seeing them work on the Mac, then this will work on your Altreus. Otherwise I am wrong and need to revise my post ASAP.

Grant Rettke 2016-05-15 09:31 UTC


Hah, excellent! I’ll give this a try. :)

– Alex Schroeder 2016-05-15 10:44 UTC


Alex, how did it go?

Grant Rettke 2016-07-15 10:58 UTC


My summer break just started at the beginning of the month and I still haven’t used my Atreus keyboard for anything. :(

– Alex Schroeder 2016-07-15 20:19 UTC


First, have a great break. Do what you enjoy!

If you happen to enjoy configuring your Emacs keyboard then know that we are all dying to hear the results!

Grant Rettke 2016-07-18 18:31 UTC


Good afternoon Alex. Have you tried the keyboard with Emacs yet?

Grant Rettke 2016-12-03 21:13 UTC


Right now I’m trying to get used to the US keyboard layout again. This requires some changes in my motor patterns. The Atreus keyboard with standard config is here, next to the laptop, but I rarely use it.

– Alex Schroeder 2016-12-03 23:35 UTC

Add Comment

More...

Comments


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. See Info for text formatting rules. You can edit the comment page if you need to fix typos. You can subscribe to new comments by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.