Diary

Welcome! 🙂

This is both a wiki (a website editable by all) and a blog (an online diary about the stuff Alex Schroeder reads and does). If you’re a friend or relative, you might be interested in reading Life instead of this page. If you’ve come here from an RPG blog, you might want to head over to RPG. There are other similar categories to be found on the SiteMap.

Für Rollenspieler gibt es ebenfalls eine eigene RSP Kategorie.

2020-11-23 Writing streaming servers in Perl using Mojo::IOLoop

I’m working on rewriting Phoebe to use the Mojo::IOLoop framework. That framework allows me to create multiple servers listening on multiple ports, for multiple addresses, using different TLS certs… I wrote about a minimal example, yesterday. Today I want to look at another problem: the framework gives me a streaming server. That means there’s bytes arriving in bursts and it isn’t entirely clear when the client is done sending.

Now, if all I was doing was writing a simple Gopher or Gemini server, I could simply assume that the client sends the entire request (the Gopher selector or the Gemini URL) in one burst: one line terminated by CR LF (\r\n). But if I’m accepting uploads (like I need to, for Titan support), then I need to continue listening if the client indicated that it’s going to send more bytes. For Titan, this means that the URL carries a “size” parameter; and parameters are separated from the rest of the URL by a semicolon.

So basically, I need a buffer per connection. I need to append the incoming bytes to the buffer and check: do I have a complete line ending in CR LF in my buffer? If so, process it. If that first line (let’s call it a header) indicated that more bytes will follow, keep listening until the necessary number of bytes have arrived. As long as we’re listening, don’t write to the stream. When we write to the stream, we’re done and we should close the connection.

OK, fine. But there’s more: in an asynchronous world, where do we keep these buffers? If ever connection has an id, we could maintain a big hash map, of course. But how do we make sure these buffers are freed when the connections end, over the weeks and months this server is going to be running? Better to use a language construct that does this for us: a closure.

In Perl, anonymous functions are closures. They keep access to the local variables they had access to at the time they were defined.

A simple example where we have a list of names, for every name we call “greeting” which returns an anonymous function (a sub with no name) that retains access to the variable $name at the time it was defined, and so when we call the code references later, the names are still greeted.

use Modern::Perl;

sub greeting_for {
  my $name = shift;
  return sub { say "Hello $name" }
}

my @greetings = map { greeting_for($_) } qw(Alex Berta);
for my $code (@greetings) {
  $code->();
}

Output:

Hello Alex
Hello Berta

The thing I learned yesterday is that this doesn’t work for named functions!

use Modern::Perl;

sub greeting_for {
  my $name = shift;
  sub greeting { say "Hello $name" }
  return \&greeting;
}

my @greetings = map { greeting_for($_) } qw(Alex Berta);
for my $code (@greetings) {
  $code->();
}

Output:

Variable "$name" will not stay shared at - line 5.
Hello Alex
Hello Alex

Anyway, back to the tiny server I’m trying to write. At it’s core, I can provide some code anytime there are bytes to read:

  $stream->on(read => sub { my ($stream, $bytes) = @_; ... });

That is, if this sub is going to be closure, then I can refer to variables it knows about:

  my ($buffer, $length, $header);
  $stream->on(read => sub {
    my ($stream, $bytes) = @_;
    $buffer .= $bytes;
    ...
  });

Sadly, this means that a lot of code ends up in the closure:

use Mojo::IOLoop;
use Modern::Perl;

Mojo::IOLoop->server({address => 'localhost', port => '3000'} => sub {
  my ($loop, $stream) = @_;
  my ($buffer, $length, $header);
  $stream->on(read => sub {
    my ($stream, $bytes) = @_;
    $buffer .= $bytes;
    if (not $header) {
      if ($buffer =~ /^(.*?)(?:;size=(\d+))?\r\n/) {
	$header = $1;
	$length = $2;
	warn "$header ($length)\n";
	if (not $length) {
	  echo($stream, $header, "none");
	  $stream->close_gracefully();
	} else {
	  $buffer =~ s/^.*\r\n//;
	}
      }
    }
    my $actual = length($buffer);
    if ($actual == $length) {
      echo($stream, $header, $buffer);
      $stream->close_gracefully();
    } elsif ($actual > $length) {
      $stream->write("ERROR! TOO MUCH DATA: $actual bytes > $length bytes!\r\n");
      $stream->close_gracefully();
    }
    warn "Waiting for " . ($length - $actual) . " more bytes\n";
	      })});

sub echo {
  my ($stream, $header, $buffer) = @_;
  # Write response
  $stream->write("Header: $header\n");
  $stream->write("Buffer: $buffer\n");
}

# Start event loop if necessary
Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

Once you start it, you can test it with telnet:

$ telnet localhost 3000
Trying ::1...
Connected to localhost.
Escape character is '^]'.
hallo;size=3
x
Header: hallo
Buffer: x

Connection closed by foreign host.

The reason I have a size of 3 and yet I only type the single letter “x” is that telnet appends a CR LF for me, so the server is seeing “x\r\n” as the second line.

Add Comment

2020-11-22 Writing Servers in Perl using Mojo::IOLoop

I’m thinking of rewriting Phoebe using Mojo::IOLoop instead of Net::Server. One problem I needed to solve was listening on different ports (e.g. 443 and 1965), for different hostnames (alexschroeder.ch, emacswiki.org, communitywiki.org, transjovian.org, toki.transjovian.org, vault.transjovian.org, next.oddmuse.org), with different TLS certificates...

This is what I had:

Mojo::IOLoop->server({port ⇒ ... , address ⇒ ... , tls ⇒ 1, tls_cert ⇒ ..., tls_key ⇒ ... } ⇒ sub { ... })

I started wondering: how can I have multiple ports, multiple addresses, and multiple TLS certs in here? Can they take arrays? The man pages was inconclusive. I looked at the source. The address and port are passed on to IO::Socket::IP. I saw something about PeerAddrInfo... and it got more and more confusing, so I asked on the Perl IRC channel.

“What on earth are you doing, and why do you believe that’s the solution? – LeoNerd

I don’t know why I often end up in these dead ends when I read the man pages for these packages.

If you want to listen on multiple ports, you use multiple servers – LeoNerd

I... I guess that makes sense!

So here’s something that works:

use Mojo::IOLoop;
use Modern::Perl;

# Listen on port 3000
for my $address (qw(localhost 127.0.0.2)) {
  for my $port (qw(79 70)) {
    Mojo::IOLoop->server({address => $address, port => $port} => sub {
      my ($loop, $stream) = @_;
      $stream->on(read => \&echo)})}};

sub echo {
  my ($stream, $bytes) = @_;
  # Write response
  $stream->write("Echo: $bytes");
  $stream->close_gracefully();
}

# Start event loop if necessary
Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

Starting it needs sudo because the ports are below 1024: gopher is on port 70 and finger is on port 79.

sudo perl test.pl

Testing it:

$ echo alex | netcat localhost 70
Echo: alex
$ echo alex | netcat localhost 79
Echo: alex
$ echo alex | netcat 127.0.0.1 79
Ncat: Connection refused.
$ echo alex | netcat 127.0.0.2 79
Echo: alex
$ finger alex@localhost
Echo: alex
$ finger berta@127.0.0.1
finger: connect: Connection refused
$ finger berta@127.0.0.2
Echo: berta

Yes! I think I can work with that. 🙂

Add Comment

2020-11-20 Small computer

I’m interested in small computers. It’s hard to put it into words. In terms of aesthetics, I have this to offer: I like to log onto my laptop, use Emacs full-screen, with 20t font size. I know it’s not for everybody, but I like it.

My laptop running Emacs

One day… Emacs standing alone on a Linux Kernel. 😂 I must confess I like the fonts and colours of a graphical display. I use them sparingly, but I like them to be there. I don’t want to go back to the old terminals.

Anyway, using mostly Emacs, maximized and big, feels a bit like the earliest computers I had: first the ZX Spectrum 48k and then the C64. It’s weird: I’m booting a multi-user system, with a gazillion layers: there’s the kernel, the encrypted filysystem, the display manager, the window manager, layers upon layers, and then Emacs, Lisp, fonts, colours – it’s fantastic and dizzying.

But the old aesthetic is still there. Single user systems that boot quickly and then you’re not in a shell but in some sort of BASIC programming environment. A read-eval-print loop (REPL) of some kind.

I had such things on my Mastodon timeline and so I started to collect some links.

I’m not trying to stoke the fires of consumerism, honest! 😀 More than anything I’m looking for happy stories of people using them, not of people buying them, if that makes sense. Like, does your kid use it? Do you?

@tsturm said: “I have a Spectrum Next and I program for it on occasion as a fun little diversion from “normal” programming. I’ve done some Basic tutorials with the kid (10yr) and he enjoyed playing with it - and also enjoyed some of the 35-year-old games we were loading up.” Yay! This is exactly the kind of story I was hoping for!

@TauPan says: “The Cosmo is my daily driver right now and it’s really nice to have a proper keyboard with emacs org-mode, python2 and a proper shell with tmux and openssh always with me (via termux).”

All of this was triggered by me seeing posts about the DevTerm. Let’s have some links.

Links

DevTerm: “ClockworkPi v3.14 integrates up to 12 interfaces in the ultra-small size of 95x77mm, ensuring sufficient connectivity for your work and entertainment. Following an easy-to-upgrade modular design of CPU and memory, clockworkPi v3.14 allows you to freely choose a suitable “Core” for various application scenarios.”

The C64 (on a website that doesn’t work without Javascript): “The C64 is back, this time full-sized with a working keyboard for the dedicated retro home-computer fan. Featuring three switchable modes – C64, VIC 20, and Games Carousel. Connect to any modern TV via HDMI for crisp 720p HD visuals, at 60 Hz or 50 Hz.”

Spectrum Next: “an updated and enhanced version of the ZX Spectrum totally compatible with the original, featuring the major hardware developments of the past many years packed inside a simple (and beautiful) design by the original designer, Rick Dickinson, inspired by his seminal work at Sinclair Research.”

MakerLisp Machine: “a portable, modular computer system, designed to recapture the feel of classic computing, with modern hardware. The CPU is a Zilog eZ80 running at 50 MHz, which supports up to 16 MB of zero wait state RAM. The system software is ’MakerLisp’, a ’Lisp on Bare Metal’ system that allows direct access to system hardware, while providing the concise expressive power of a functionally complete Lisp environment.”

Raspberry Pi 400: “Featuring a quad-core 64-bit processor, 4GB of RAM, wireless networking, dual-display output, and 4K video playback, as well as a 40-pin GPIO header, Raspberry Pi 400 is a powerful, easy-to-use computer built into a neat and portable keyboard.”

BMC64: “a bare metal fork of VICE’s C64 emulator optimized for the Raspberry Pi. It has 50hz/60hz smooth scrolling, low video/audio latency and a number of other features that make it perfect for building your own C64 replica machine. For more details visit the github link below.”

RISC OS: “It’s small. It’s fast. RISC OS is a full desktop OS, where the core system including windowing system and a few apps fits inside 6MB. It was developed at a time when the fastest desktop computer was an 8MHz ARM2 with 512KB of RAM. That means it’s fast and responsive on modern hardware. The memory taken by apps is usually counted in the kilobytes. A 700MHz 256MB Raspberry Pi is luxury - what to do with all that memory?”

Gemini PDA and Cosmo Communicator: “The size of a smartphone, with the capabilities of a laptop. Work, create and socialise on the move with a single device.”

Add Comment

2020-11-20 Moku pona is on CPAN

Now that I’m slowly learning how to package code for CPAN – all of that after realizing that I could in fact upload tools and scripts without regular Perl modules for others to use – I’ve packaged and uploaded moku pona as well.

Thus, if you’re a Perl user, here’s how you’d install it:

cpan install App::mokupona

I hope it works for you. 🙂

Moku pona is a Gemini based feed reader. It can monitor URLs to feeds or regular pages for changes and keeps and updated list of these in a Gemini list. Moku pona knows how to fetch Gopher URLs, Gemini URLs, and regular web URLs.

You manage your subscriptions using the command-line, with moku pona.

You serve the resulting file using a Gemini server.

You read it all using your Gemini client.

Comments on 2020-11-20 Moku pona is on CPAN

I added lupa pona to CPAN as well (App::lupapona), which should help you self-host a directory of files, such as the moku pona directory.

– Alex 2020-11-21 23:29 UTC

Add Comment

2020-11-20 Geolocation using Exif data and Open Street Map

As I’ve shared this Perl script twice, now, I figured perhaps it should be on the blog as well. It allows me to tell where a picture was taken, if and only if the picture contains geolocation metadata, which most cell phone images should. I also used to have a regular camera with a built-in GPS, but then I bought a camera that took better pictures and had no GPS…

#!/usr/bin/env perl
use Modern::Perl;
use Image::ExifTool;
use Geo::Coder::OSM;
use JSON;
use Data::Dumper;
binmode(STDOUT, ':utf8'); # force UTF-8 output
my $geocoder = Geo::Coder::OSM->new;
my $exifTool = Image::ExifTool->new;
$exifTool->Options(CoordFormat => q{%+.6f},
		   DateFormat => "%Y-%m-%d %H:%M:%S");
for my $file (@ARGV) {
  die "Cannot read $file" unless -f "$file";
  say "$file";
  $exifTool->ExtractInfo("$file");
  # Date
  my $date = $exifTool->GetValue('CreateDate', '');
  say $date if $date;
  # City
  my $lat = $exifTool->GetValue('GPSLatitude', '');
  my $long = $exifTool->GetValue('GPSLongitude', '');
  my $location = $geocoder->reverse_geocode(lat => $lat, lon => $long);
  die "No location data found\n" unless $geocoder->response;
  my $json = decode_json($geocoder->response->content);
  die $json->{error} . " $lat/$long\n" if $json->{error};
  say $location->{display_name} if $location;
}

Add Comment

2020-11-19 Trying to wrap my head around async Perl

On the Perl channel, they like to recommend IO::Async and Mojo::IOLoop but and in all the years I’ve tried using them, I never got anywhere. Somehow my brain just can’t digest the documentation. I find it so difficult to parse, in fact, that I can’t even tell you whether the documentation is incomplete or not.

Today I decided to give it another try. See how far I get! I want to spider a few links using moku pona and I means having to call some code for a bunch of URLs (fetching stuff and processing it) and I want to return a string from each one of them.

Here’s what I cobbled together as I tried to get IO::Async::Loop to work. The run function simulates getting my results, the list of to-dos are the URLs I’m “fetching”.

use Modern::Perl;
use IO::Async::Loop;
say time() . " start";
my $loop = IO::Async::Loop->new;
my @todos = qw(a b c);
my @procs;
sub run { sleep rand(5); say time() . " " . shift; }
for my $todo (@todos) {
  push(@procs, $loop->run_process(
	 code => sub { run($todo) }));
}
for my $proc (@procs) {
  my ($exitcode, $stdout) = $proc->get;
  print $stdout;
}
say time() . " end";

And here’s the kind of output it produces.

1605792335 start
1605792338 a
1605792336 b
1605792337 c
1605792338 end

I can’t tell you whether this how you’re supposed to use it. All I know is that it seems to work: the tasks are accomplished in random order (in the example above, a is the latest to finish), and information is communicated back to the parent process (I’m not even sure it’s forking at all).

I don’t under understand how the loop knows when to end, for example. I don’t know how to read the documentation in order to find out.

I guess it’s creating a bunch of Futures, and as I’m calling “get” on them in the last loop it starts waiting for them, one after another. I think this is fine. We’ll wait the longest for the first one, but the others might all have finished in the background and so when it’s their turn, the Future is already done.

OK, I think I could replace the “run” sub with something more interesting, doing network requests. It feels weird to use STDOUT to get data back, but perhaps this is how forking and inter-process communication ought to work? I don’t know whether this is what IO::Async does.

I’d be using something like this, except instead of returning the string I’d have to print it to STDOUT, I guess.

sub query_gemini {
  my $url = shift;
  my($scheme, $authority, $path, $query, $fragment) =
      $url =~ m|(?:([^:/?#]+):)?(?://([^/?#]*))?([^?#]*)(?:\?([^#]*))?(?:#(.*))?|;
  die "⚠ The URL '$url' must use the gemini scheme\n" unless $scheme and $scheme eq 'gemini';
  die "⚠ The URL '$url' must have an authority\n" unless $authority;
  my ($host, $port) = split(/:/, $authority, 2);
  $port //= 1965;
  my $socket = IO::Socket::SSL->new(
    PeerHost => $host,
    PeerService => $port,
    SSL_verify_mode => SSL_VERIFY_NONE)
      or die "Cannot construct client socket: $@";
  # send data in one go
  print $socket "$url\r\n";
  # read response
  local $/ = undef;
  my ($header, $response) = split(/\r\n/, <$socket>, 2);
  return $response;
}

As I looked at Mojo::IOLoop, I noticed that this might be more interesting: it has code to create clients that get their data piece by piece. I’ll have to rewrite my three clients (Gopher, Gemini, Web), but it might work...

Here’s code using a finger requests (like Gopher) on port 79:

use Modern::Perl;
use Mojo::IOLoop;
say time() . " start";
my @requests = qw(Alex About Diary);
my %responses;
for my $request (@requests) {
  Mojo::IOLoop->client(
    {port => 79, address => 'alexschroeder.ch' }
    => sub {
      my ($loop, $err, $stream) = @_;
      $stream->on(read => sub { my ($stream, $bytes) = @_; $responses{$request} .= $bytes;});
      $stream->write("$request\x0d\x0a");
    })}
# Start event loop if necessary
Mojo::IOLoop->start unless Mojo::IOLoop->is_running;
say time() . " end";
# Results:
for my $request (@requests) {
  say $request;
  say $responses{$request};
  say "--------------------";
}

What’s new is that I can store the (partial) responses in a hash and keep adding to it.

I don’t know how this loop knows that it’s done.

In any case, I think I’m ready to move on to TLS. I can’t just use Mojo::UserAgent because I’m not just interested in HTTPS (which I’ll need to fetch RSS and Atom feeds) but I’m also interested in Gemini, which is a bit like Gopher over TLS, on port 1965.

use Modern::Perl;
use Mojo::IOLoop;
say time() . " start";
my @requests = qw(Alex About Diary);
my %responses;
for my $request (@requests) {
  Mojo::IOLoop->client(
    {port => 1965, address => 'alexschroeder.ch' }
    => sub {
      my ($loop, $err, $stream) = @_;
      $stream->on(
	read => sub { my ($stream, $bytes) = @_; $responses{$request} .= $bytes;});
      $stream->write("gemini://alexschroeder.ch/page/$request\x0d\x0a");
    })}
# Start event loop if necessary
Mojo::IOLoop->start unless Mojo::IOLoop->is_running;
say time() . " end";
# Results:
for my $request (@requests) {
  say $request;
  say $responses{$request};
  say "--------------------";
}

This doesn’t work. I need to add Mojo::IOLoop::TLS to the mix and I have no idea how to do this. Something about that stream not being a handle or whatever. But on the Perl channel user mst pointed out to me that there’s a TLS option for the client that does all that for me! And it works.

use Modern::Perl;
use Mojo::IOLoop;
use Mojo::IOLoop::TLS;
say time() . " start";
my @requests = qw(Alex About Diary);
my %responses;
for my $request (@requests) {
  Mojo::IOLoop->client(
    {port => 1965, address => 'alexschroeder.ch', tls => 1 }
    => sub {
      my ($loop, $err, $stream) = @_;
      $stream->on(read => sub { my ($stream, $bytes) = @_; $responses{$request} .= $bytes;});
      $stream->write("gemini://alexschroeder.ch/page/$request\x0d\x0a");
    })}
# Start event loop if necessary
Mojo::IOLoop->start unless Mojo::IOLoop->is_running;
say time() . " end";
# Results:
for my $request (@requests) {
  say $request;
  say $responses{$request};
  say "--------------------";
}

Sadly, something about the sort of documentation used by IO::Async and Mojo::IOLoop does not agree with my way of reading documentation, unfortunately. I often feel that I only understand it if I already know it. As I kept working on my IO::Async::Loop and Mojo::IOLoop examples, I was confused about many things, read the docs up and dow for about three hours, wasted time reading Mojo::IOLoop::TLS and trying to get that to work, trying to figure out how Mojo::IOLoop::Client plays into this. It was a very frustrating experience, for me.

But, on a happier note: moku-pona now starts all the downloads in parallel. I’m not sure how job management actually works. I have no idea how many “worker threads” (or whatever the correct terminology is) Mojo::IOLoop uses, but updating is now definitely faster than back when I downloaded every link one after the other.

Add Comment

2020-11-18 Feed reader, page watcher, via Gopher, Gemini, or the Web

Yeah, it’s moku pona 2! It’s not quite ready for release, yet. But if you want to try it, the repo is still the same. And if you only care about Gopher, you shouldn’t ugrade. Just stay on release 1.1 and you’ll be fine. There haven’t been any updates in a very long time and I don’t think there’s anything that needs changing. But for Gemini, I needed some changes:

  • I want to use my Gemini client to browse the moku pona updates
  • I want to subscribe to Gopher and Gemini pages
  • I want to subscribe to Atom and RSS feeds
  • I want to subscribe to feeds served via HTTP, HTTPS, and Gemini

Moku pona was close, but not quite there yet. If you look at the current repository, you’ll see that the main branch is still struggling with producing the correct output. It seems hard to believe but my problem is that the page with the updates has bugs. For a while now the code can’t seem drop older updates of the same feed and it drives me nuts.

Well, I’m still working on it. But it you’re interested, give it a look.

Eventually, I’m hoping to get it onto CPAN…

Add Comment

2020-11-14 Computerless Computer Science

@zens has been sharing a ton of links about computerless computering on Mastodon. It all started with @neauoire asking for “anything relevant to paper computing, diy punchards machines, graph-paper coding, vedic mathematics, mechanical programming, and other things you have stumbled upon that you found interesting related to computerless computing?”

The Curta hand crank calculator, where multiplication is done by repeated addition.

Slide rules are graphical analogue calculators “At its simplest, each number to be multiplied is represented by a length on a sliding ruler. As the rulers each have a logarithmic scale, it is possible to align them to read the sum of the logarithms, and hence calculate the product of the two numbers.”

The whole Abacus field, including Chinese Suanpan and Japanese Soroban is super fascinating. The Roman section on the Abacus page has a picture where you can see the use of pebbles (calculi).

The entire complex of tools built relating to the stars is fascinating, from the Antikythera Mechanism to the Sextant and the Astrolabe.

The MONIAC: “The MONIAC (Monetary National Income Analogue Computer) … was created in 1949 … to model the national economic processes … The MONIAC was an analogue computer which used fluidic logic to model the workings of an economy.”

Ramakrishnan VU3RDD @vu3rdd linked me to CS Unplugged which appears to be something in the same vein, where kids are taught CS concepts without using a computer.

Add Comment

2020-11-13 Phoebe is on CPAN

I think this my first CPAN contribution. I added the Gemini wiki Phoebe.

Based on the recommendations of @wim_v12e and @e1e0 I tried to set it all up using Dist::Milla but in the end, the approach based on How to upload a script to CPAN by David Farrell (2016) just worked, without befuddling me with magic that I did not comprehend. OK, to be honest, I also forgot to include the tests in my first release which means that the tests results were “unknown” or “not available” – no wonder, right. But some of the things I didn’t understand about Dist::Milla: it reportedly couldn’t extract the license from the lib and there was no documentation of how it worked. Well, I finally discovered that it was using Software::LicenseUtils by reading the source, and even then knowing that I couldn’t get it to recognise the license unless I put “afford g” in the file. And when one of the tests failed with “milla test” I couldn’t reproduce it. Even if I entered the latest build directory itself and reran the tests, it worked. Too much magic under the hood.

Comments on 2020-11-13 Phoebe is on CPAN

Hm. Something is still wrong. Version 1.1 is considered to be the latest one, and 1.1.1 and 1.1.2 are considered to be lower somehow? I’m going to schedule 1.1 for deletion and then we’ll see whether 1.1.2 ends up being the “current” version.

– 2020-11-14 12:20 UTC


Apparently I made a beginner’s mistake: “That’s obvious, but wrong; version specifiers aren’t numbers. To work correctly, you must quote them …”

– 2020-11-14 20:49 UTC


Here we go, 1.20 is released! 😅

– 2020-11-14 21:18 UTC


And the “CPAN Testers matrix” reports a ton of errors because I didn’t declare File::Slurper in Makefile.PL. Oh well, here goes 1.21!

– 2020-11-14 22:47 UTC


I wonder how one would determine the actual minimum versions of dependencies except through bitter experience...

– 2020-11-15 11:11 UTC


Sadly, on some platforms the SSL certificate generation is now the point where tests are failing:

Generating a 2048 bit EC private key
writing new private key to 't/key.pem'
-----
Could not finalize SSL connection with client handle (SSL accept attempt failed error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher)
Cannot construct client socket: SSL connect attempt failed error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure at ./t/test.pl line 114.

I guess the problem is that I need to specify that a certain openssl version must be around, but only when testing?

– 2020-11-15 12:04 UTC


OK, apparently “some platforms” means RHEL7. It uses OpenSSL 1.0 and thus has no access to TLSv1.3. I’m not sure this is the real culprit, though. IO::Socket::IO seems to share at least some of the ciphers! The OpenSSL wiki makes me think that perhaps there is something else at work. I’ve added more debugging info to release 1.22 and will try again.

– 2020-11-15 19:28 UTC


Yeah, SSL is complicated. But it’s no good to make broad and sweeping statements about complexity. The problem of encryption and certificates on a high level is easy to understand; it’s when you get down to the complexities of the openssl command line, the protocol details (deprecation of some versions), the vague definitions of common name and alternative subjects…

All of this, from a distance, is unnecessary complexity. It’s uncalled for weight explained by history, legacy, and so on.

It’s not an inherent complexity that necessarily needs to crop up somewhere.

Or perhaps that just goes to show that cryptography is a field with an inherent dynamic that I don’t fully understand which necessaril leads to this bewieldering complexity.

And that’s not even talking about the various implementation details. People say that the Python part is tricky, I don’t know. But the framework I was using in my Perl code had a bug but only when handling client certificates which forced me to delve into the Perl framework, the OpenSSL bindings, the OpenSSL documentation, and by the end of it I was ready to burn it all down.


Test results

At last! The automatic testing done by Slaven Rezić is all green. Now I can rest. 🙂

Clicking through to the actual test results, however, it seems that as luck would have it, no RHEL7 was included...

No RHEL7

– 2020-11-16 08:33 UTC


Ah, RHEL7 results are showing up... and the output produced confirms the verdict on the OpenSSL wiki:

If you use a key or certificate without without the OPENSSL_EC_NAMED_CURVE flag (i.e., one that looks like the image on the right), then the SSL connection will fail with the following symptoms: … Server … 140339533272744:error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher:s3_srvr.c:1353:

For reference, what I’m seeing is:

1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher

I think what I’ll do is I’ll either skip the tests if the OpenSSL version is too old, or not use elliptic curves.

– 2020-11-16 18:02 UTC

Add Comment

2020-11-11 Buying stock

I sort of assumed it was a given that the stock market had a better return than treasury bonds. But today I saw a blog post saying that the excess return was minimal: less than 5%! And if you looked at cohorts (people saving for 40 years) over the last few decades you’ll see that returns at the end vary between plus/minus 10% with a median of again, less than 5%. Now, you don’t know in what sort of cohort you’ll end up in, so you don’t know where in that range you’ll fall, but chances are you won’t do a lot better than buying long term bonds (and you’ll probably stress less). At least that’s what I see in the blog post.

I did not expect this.

Comments on 2020-11-11 Buying stock

I would assume it’s an excess return of 5% per annum.

But in any case, everything depends on the starting yield of the bonds. 40 years ago in 1980 was the top in interest rates after the highly inflationary 1970s. Treasury Bonds yielded more than 20%! That was hard do beat for stocks for many years.

But now the same 10y Bonds just yield 0.9%. Almost impossible to outperform stocks with this low starting yield. Bonds offer returnless risk at the moment (risk of default or more likely of inflation).

– Peter 2020-11-11 23:08 UTC

Add Comment

More...

Comments

You probably want to contact me via one of the means listed on the Contact page. This is probably the wrong place to do it. 😄

– Alex Schroeder 2020-05-22 12:19 UTC

Referrers: Diary Diary Diary