Welcome! 🙂

This is both a wiki (a website editable by all) and a blog (an online diary about the stuff Alex Schroeder reads and does). If you’re a friend or relative, you might be interested in reading Life instead of this page. If you’ve come here from an RPG blog, you might want to head over to RPG. There are other similar categories to be found on the SiteMap.

Für Rollenspieler gibt es ebenfalls eine eigene RSP Kategorie.

2021-01-24 Traveller missions for solo adventuring

I’m sorry to say that this blog post has more questions than answers because I’m trying something new. It’s my semi-solo Traveller campaign. I’ve been reading “Solo” by Zozer Games, but I’m not yet convinced. There, Paul Elliot uses “the Plan” as the mechanism: the player considers the adventure situation, formulates a plan, writes it down, and makes two rolls, one depending on the estimated chances of success, the other depending on the estimated risk.

Right now, that’s a bit too abstract for me. So I’m using the regular Traveller rules. I roll for surprise, for encounter distance, I roll for all the things, trying to learn the system as I go. It’s slow but I don’t mind. Occasionally I’ll ask my wife for a roll. She’s sitting next to me, reading a book, and humours me. Or I’ll ask her about her character’s opinion. It works for me. 😁

What remains unsolved in either system is the generating of adventures. Having adventure ideas, and imagining their outcomes, to then resolve either using abstract rolls or the complete system, remains a problem. I guess I’m just good at imagining a calm afternoon at the spaceport, where nothing happens.

Perhaps this resolves itself over time as the people and ships reappear. And I need to roll for reactions more often, clearly. And I need to imagine more adventurous ideas, more drama, more action.

Perhaps using the Burning Wheel technique of writing down three beliefs would help drive the story?

Like, for my Louis character:

  • I believe I’m the better pilot than Lux and I will prove it to her.
  • I believe that hunting animals for sports is terrible and I will free all the animals kept on the Tiktok (whose hunter we just supported)
  • I believe that we really need to make a lot of money and will take any job, even if illegal

Then, for my wife’s Lux character I’ll write up three other beliefs. And then we go to town, less interested in the “simulation” of this backwater class E starport but as two characters driving the plot forward with their beliefs.

Hm. This Burning Wheel mindset is growing on me. It’s that two player game I’m listening to as a podcast. It’s influencing me. I get the feeling that if a player writes down beliefs, it presupposes some form of negotiation at the table. What’s the game going to be about? It is framed in terms of things the characters believe in, but it’s no different that agreeing on a set of scenes when starting to play Western City, for example. What’s the game going to be about. And then, during the session, we’re all interested in pushing the story towards these beliefs. Again, it’s framed as fighting for or challenging beliefs, but it could also be thought of as everybody trying to stay on topic. We don’t need the exact words to have similar effects.

So, for this Traveller game, where the two scouts, Lux and Louis are stranded on a backwater planet with a starport in a jungle, with dense and tainted atmosphere, I’m thinking… OK, so here are my thoughts as a referee, based on what’s been happening on the planet, in another campaign, and what the current situation might be.

A lost generation that grew up in a mine that broke down, now resettled to this backwater planet. What would the effect be? I’d say, rampant xenophobia. So that’s one “front” that’s building, if we’re talking tools imported from other systems. I’d say: tensions are rising. The lost generation is lacking skills and thus cannot be put to work immediately. Education is costly and nobody wants to do it. So the original settlers are blaming the immigrants for their lack of education and the new settlers are uprooted and angry. Soon, there will be fighting in the streets.

Simerik Firik is a cult leader among the new settlers. He’ll ride the violent storm, for sure.

Gehen Aam is the regular strong arm guy, fitness, self defence, security, that’s his business. It’s small, it’s local, but he’s a rallying point for the unhappy old settlers, that is: poor farmers.

Bron Banta represents business interests. He’s interested in peace and programming, “not interested in politics,” meaning he’s interested in things staying as they are, but quiet.

I also have some names, family heads that control various business branches. The settlement is so small that none of them have competition. Their only problem is getting space on ships to export their goods. Hm. Visius controls the spice, Schneid controls liquor production, Viktoriana controls the laquer, Limbus controls the wood.

So how to make money? Banta might be interested in a security operation against Firik’s cult, and Aam’s angry mob. Visius and Limbus might be having problems in one of their outposts. In fact, if this is another “front”, then we should perhaps write it up as follows: two wood and spice transports have been disappearing and the clans are angry. Who is sabotaging their work? Nobody is making any demands. Therefore, this isn’t somebody who’s out to change business practice, it’s somebody who’s trying to drive them out of business. Already they are stepping up their security. Soon, there will be firefights, and eventually these people will leave the system, resulting in this backwater system to drop off the radar entirely.

Vibes of the Mandalorian, season 2, and of the storming of the capitol on January 6, 2021.

Comments on 2021-01-24 Traveller missions for solo adventuring

Do you have Stars Without Number? I bet some of the tables and resources in there would be helpful for this type of solo play. Lots of generative content.

Derik 2021-01-25 19:54 UTC

Good point!

– Alex 2021-01-26 07:50 UTC

I’ve run into many of the same issues. I like both Traveller and SWN, and the worldbuilding is fun, but I just can’t transition to solo play with characters.


Saxinis Kion 2021-01-26 15:10 UTC

Let’s see. I still hope that it will be like a calm first season: lots of new characters being introduced, but then after a while the connections between the events start to build up and a plot emerges in retrospective.

– Alex 2021-01-27 08:52 UTC

Add Comment

2021-01-19 Listening to Burning Wheel Actual Play

I just listened to the first two episodes of Sean Nittner’s and Judd Karlman’s actual play podcast about their Shoeless Peasant game. It’s the first time I’ve listened to other people play. I did this while making bread, making salad, cleaning up the kitchen, simple manual stuff to do with my hands. I don’t think I’d like to see it on video, but on a podcast, it works well. Hearing two people do nothing but play, practically no banter, very slow and deliberate, very much concerned about the rules and the dice, and forking and helping, beginner’s luck, scripting for the duel of wits and all the other details that used to turn me off, turns out to work really well.

I used to like a raucous table of six players or more, where people are exited, there’s shouting and groaning, cheering. Things go back and forth quickly. What a change in tempo to my usual games. Everything is slow and deliberate. The dice are cast and some words are spoken, the hosts talk about what they see is happening, decisions are made, scenes change, but dice rolling is used like an oracle to see which way a story goes, even for minor things: does the woman run when she sees the elvish runes? Does she cut her feet on a sharp edged stone when she runs? Everything is tested.

At the same time, the rules seem to push for poignant moments – the kind I love so much. In the duel of wits, the two hosts have to add some words to make their points, their dismissals, their feints, and it leads to some wonderful lines. The girl needs to be calmed and it leads to some wonderful lines. A decision needs to be made and it leads to some wonderful lines.

Now, you could say that nothing stops you from saying those lines in a different game. I remember our last session in the Humblewood campaign using D&D 5. Previously, we had kicked some mousling thieves from the upper market. When the cleric saw how desperately poor they were, he arranged for them to be taken into the temple he serves. So when we returned from a long fight in the bandit camp, our pockets full of coin, I made it a point to visit the temple, to inquire about the mouslings, and to donate most of my money for the time when they finished their school and left the temple. I felt the short exchange with the referee was moving. This is also something I like about role-playing games. And right now I think perhaps I can better appreciate how Burning Wheel might be better suited to produce maybe not the scenes themselves but the opportunities that we can use to create such scenes, such poignant scenes, moving scenes.

Comments on 2021-01-19 Listening to Burning Wheel Actual Play

So glad to hear you’ve been enjoying the show. I agree that listening to AP is the way to go!

Sean Nittner 2021-01-19 23:12 UTC

I was so thankful that Sean made the Podcast available. I take lots of mental notes on Judd’s GM style and how he handles the numerous failures of Vena. It’s not about hitting hard and beating Vena down, it’s about not giving Sean exactly what Vena’s trying for, but taking that focused desire and redirecting with a complication; and those complications are grandiose.

Jeremy Friesen 2021-01-20 01:50 UTC

I saw this post and thought of a hex describe feature.

It users the therefore and but rather than and then tactic from South Park.


Also on contact page you list Matrix, is there a room for that or what is your contact info there?

– Oliver 2021-01-21 20:34 UTC

For Matrix, I’m @kensanata:matrix.org, and I’m not in any rooms.

When I read your statement it sounded pretty confusing, but the blog post makes a ton of sense. And then this happened, and then this happened, and then this happened, this is why one of the ways to telling a boring story, absolutely. I loved the Swordfish Island example you have for the other method. This happened, but then this happened, therefore this happened, but then this happened, therefore this happened. In my mind’s eye, I can see fans talking about the plot, excited, using exactly those conjunctions. I like it a lot. Thanks for that link!

I’m not quite sure how to use this to build a generator. Perhaps I would have to start small… Hm. 🤔 Perhaps if we could group these statements. Like… how did the ruler of this village rise to power?

[this village was built], but [this village was razed], therefore [this village has a secret], but [the current lord does not know], therefore [the opposition is strong]. I’m not sure, it feels like the results from the table about the opposition would have to “know” about the secret. Or perhaps this is a sign of strong coupling that is unnecessary?

[this village was built], but [this village was razed], therefore [this village is not important], but then [something happened in this village] and therefore [this village is now doing changing]. Now the parts are not interdependent.

this village was built

  • In the old days [old human name] convinced [some giants] to build this castle
  • Many years ago, [wizard] built this tower using spells and slaves

this village was razed

  • in the War of Dragons is was laid to waste by [dragon name]
  • in the days of our grandparents the [disease name] killed almost all of us

something happened in this village

  • they discovered a gold mine upriver
  • [wizard name], who ruled during the time of my parents got involved in some wizard power struggle and disappeared

this village is now doing changing

  • but now [human name] is handling the affairs of [trade society] and we’re recovering from all that
  • but that opportunity was squandered and when [human name] showed up last year, we practically begged them to stay

What do you think, something like that?

– Alex 2021-01-21 21:29 UTC

I think it sounds great! I initally thought about it when you mentioned generating in game books.

For town/ regional history I think its a great idea!

Glad you liked the post!

I was also thinking about Hex Describe and Rougelikes.

The tile and ASCII games look simple but the code behind them is often very smart and often free/ open source (so you can use parts for Hex Describe)

For generating worlds I think : Dwarf Fortress, Cataclysm Dark Days Ahead, Caves of Qud (not free but very Gamma World/A Subsurface Environment), ADOM and TOME (renamed Tales of Maj’Eyal) (Dungeon Crawl Stone Soup is really good but maybe not as useful as its only dungeon based)

These gamea do a really good job at generating lived in places and are often Open Source!

– Oliver 2021-01-22 13:48 UTC

Add Comment

2021-01-18 Traveller Solo Text Based

I revived an old Traveller campaign wiki of mine and started creating two characters, and asked my wife to create a character… I’m excited!

The existing wiki is mostly in German because I was expecting to run a Solo campaign. I’m not opposed to mixing languages or using English if my wife isn’t there (as we like to say in the German speaking parts: “Denglish”, haha). So English could be OK.

At the same time, I guess I’m also looking for a smaller game. Fewer people. I’m hoping this makes the game more intense; perhaps that makes for more satisfying remote gaming sessions. I wonder. These days I’m a player in two D&D 5 games, and I’m feeling ambivalent. It’s nice to play with friends, but I’m not enjoying the mechanics and the fighting and the Roll 20 aspect too much. We use Roll 20 in just one of the games and it does invite a lot of token pushing. That’s OK, too. When these games started, we had all agreed to focus on fighting and dice rolling when we meet and to focus on character interplay via posting on the wiki between games. The writing on the wiki has sometimes been fantastic, deep, moving, a whole new level! Then again, I also feel it’s one player in particular that’s pushing me to join in. Mutual inspiration at its best!

So now I’m finding myself moved in two directions:

On the one hand, the online sessions aren’t intense enough. I’m easily distracted. The presence of actual people at the table took care of that, for me. The monkey brain is easily engaged when people are there. So perhaps, mediated via the screen and the Internet, fewer people is better for me.

On the other hand, I’m starting to appreciate the writing between sessions. Sometimes things happen, and we know there must be consequences, the soul sighs and is weary, the mind suspicious, the friendship deepened, the courage shaken, the triumph glorious, the epiphany blows our mind – and that’s hard to go into at the table, but we can work on it in our writing. I used to say: none of us is an author. True. But we could be. We could be working on it. I’m finding myself enjoying that part.

So here’s what I’m thinking. I’m going to start with some solo stuff. Creating characters was a good start. Think of an adventure, roll some dice, write it up. Maybe ask my wife to join. Again, roll some dice, and write it up. Short games, perhaps 1–2h games with just one or two players might be key. And then a lot of writing on the wiki. Perhaps chatting on the phone when we’re not gaming, via Signal, or Threema.

Playing via a wiki and maybe an instant messenger with a chat group is a possibility I’m willing to try.

Comments on 2021-01-18 Traveller Solo Text Based

Interesting, never been one for such games myself.

– cosPgSKkiPM 2021-01-19 05:43 UTC

Yeah, me neither. But given the pandemic, and given that I’m in two games via video chat already, I’m willing to try new things.

– Alex 2021-01-19 09:27 UTC

Add Comment

2021-01-16 How to use Gemtext for writing

Here’s something I do quite often: I talk about a thing, I quote a longer passage somebody else wrote, and I link to the source, hopefully naming the author and title, or something like it. How do you do that, given the tools that Gemini markup (also known as gemtext) provides, and how do you render it on your client? Screenshots, HTML, CSS, plain text, it’s all good. How do you write it, and how do you want it to look?

When I write for the web, I know what it looks like:

This is the quote. – _Title of the piece_, by the named author

“Title of the piece” is a link.

If I were to use gemtext, this is what I’d write:

> This is the quote.
=> Title of the piece, by the named author

Unfortunately, viewed via the web, this looks weird:

    This is the quote.
 • _Title of the piece, by the named author_

Perhaps I can improve this using simple CSS?

Sadly, it doesn’t look too good using Elpher, either.

> This is the quote.

→ _Title of the piece, by the named author_

Perhaps the look would be improved if I didn’t treat these links as list items but simply as link “paragraphs”? I don’t know.

Comments on 2021-01-16 How to use Gemtext for writing

Here’s how I do it:

=> protocol://link something by someone:
> Quote excerpt quote excerpt

That colon is important!

TimurIsmagilov 2021-01-16 19:34 UTC

Convert those lines differently. Linkline followed by contiguous set of quote lines: transpose, delete the newline, add the emdash. Other linklines render as li a.

– Sandra Snan 2021-01-16 22:44 UTC

Add Comment

2021-01-16 Facebook and WhatsApp

“Is there a good link to a simple text in German or English that explains why Facebook cannot be trusted, for a layperson?” I asked this question on Mastodon and got back a ton of links. I’ll use the opportunity to start a link collection.

@ColinTheMathmo, @deejoe, and @dredmorbius linked that story where a company is analysing your purchasing history and ends up knowing that a woman is pregnant before she tells her family.

Target Knows You're Pregnant, by Kelly Bourdet, for VICE: “A man walks into a Target store outside Minneapolis and angrily confronts the store manager. He shows him an advertisement that was sent to his high school daughter, filled with maternity clothing and baby items. The perplexed manager apologizes and even calls the man at home the following week to further apologize for the advertising faux pas. However, when he does the man sheepishly admits he’s found out that his daughter is, in fact, pregnant.”

@alienghic linked an opinion piece. I’m not quite sure I agree with the exact wording of the conclusion. I’m not convinced the business model actually works. For the purposes of explaining Facebook’s monopolist power, however, that doesn’t matter. All that matters is that the advertizers think it works.

Facebook’s Ad Scandal Isn’t a ‘Fail,’ It’s a Feature, by Zeynep Tufekci, for The New York Times. “Facebook has become the go-to site for anyone hoping to reach a big audience — whether to sell shoes or to sell politics, and it’s become profitable by doing so. That is because most of its systems are either largely or entirely automated. This lets the site scale up — it is up to two billion monthly users now — and keeps costs down … — ad-targeting through deep surveillance, emaciated work force, automation and the use of algorithms to find and highlight content that entice people to stay on the site or click on ads or share pay-for-play messages — works.”

@deejoe reminded me of this experiment, where somebody tries to live without the “big five” tech giants out of their life. It’s hard.

I Cut the 'Big Five' Tech Giants From My Life. It Was Hell. Week 6: Blocking them all, by Kashmir Hill, for Gizmodo. “The tech giants laid down all the basic infrastructure for our data to be trafficked. They got us to put our information into public profiles, to carry tracking devices in our pockets, and to download apps to those tracking devices that secretly siphon data from them. … The tech giants were long revered for making the world more connected, making information more accessible, and making commerce easier and cheaper. Now, suddenly, they are the targets of anger for assisting the spread of propaganda and misinformation, making us dangerously dependent on their services, and turning our personal information into the currency of a surveillance economy.”

@deejoe also recommended Shoshana Zuboff’s book, The Age of Surveillance Capitalism. I must confess that I haven’t read it. All I’ve read are comments here and there, and I’ve seen her in the movie The Social Dilemma (2020).

@maikek linked the Wikipedia pages collecting the criticism of Facebook: Kritik an Facebook, and Criticism of Facebook. Just look at a selection of the table of contents:

  • Data mining
  • Inability to voluntarily terminate accounts
  • Photo recognition and face tagging
  • Tracking of non-members of Facebook
  • Facebook and Cambridge Analytica data scandal
  • Sharing private messages and contacts’ details without consent
  • Denial of location privacy, regardless of user settings
  • Health data from apps sent to Facebook without user consent
  • International lobbying against privacy protections
  • Unencrypted password storage
  • Promotion of service as “free”
  • Providing ads through “snooping”

Uuuuugh. 🤮

@jamesmullarkey linked a report from Amnesty International. Facebook and Google’s pervasive surveillance poses an unprecedented danger to human rights. “As a first step, governments must enact laws to ensure companies including Google and Facebook are prevented from making access to their service conditional on individuals ‘consenting’ to the collection, processing or sharing of their personal data for marketing or advertising. Companies including Google and Facebook also have a responsibility to respect human rights wherever and however they operate.”

The report is available for download at the end of the article. In the executive summary: “But despite the real value of the services they provide, Google and Facebook’s platforms come at a systemic cost. The companies’ surveillance-based business model forces people to make a Faustian bargain, whereby they are only able to enjoy their human rights online by submitting to a system predicated on human rights abuse. Firstly, an assault on the right to privacy on an unprecedented scale, and then a series of knock-on effects that pose a serious risk to a range of other rights, from freedom of expression and opinion, to freedom of thought and the right to non-discrimination.”

Indeed: “This isn’t the internet people signed up for.” 🤮

@jamesmullarkey also reminded me of something that I had practically forgotten. A Genocide Incited on Facebook, With Posts From Myanmar’s Military, by Paul Mozur, for The New York Times. “The Facebook posts were not from everyday internet users. Instead, they were from Myanmar military personnel who turned the social network into a tool for ethnic cleansing, according to former military officials, researchers and civilian officials in the country. Members of the Myanmar military were the prime operatives behind a systematic campaign on Facebook that stretched back half a decade and that targeted the country’s mostly Muslim Rohingya minority group, the people said.”

@ashwinvis linked me Richard Stallman’s Reasons not to be used by Facebook page which led me to Get your loved ones off Facebook. “Through its labyrinth of re-definitions of words like information’, ‘content’ and ‘data’, you’re allowing Facebook to collect all kinds of information about you and expose that to advertisers. With your permission only they say, but the definition of ‘permission’ includes using apps and who knows what else.”

Facebook Is a Doomsday Machine, by Adrienne LaFrance, for The Atlantic. “In the days after the 2020 presidential election, Zuckerberg authorized a tweak to the Facebook algorithm so that high-accuracy news sources … would receive preferential visibility in people’s feeds, and hyper-partisan pages … would be buried, … offering proof that Facebook could, if it wanted to, turn a dial to reduce disinformation—and offering a reminder that Facebook has the power to flip a switch and change what billions of people see online.”

@trianderror linked resources in German, collected by Netzcourage.

@dredmorbius linked Why Zuckerberg’s 14-Year Apology Tour Hasn’t Fixed Facebook, by Zeynep Tufekci, for Wired. “By 2008, Zuckerberg had written only four posts on Facebook’s blog: Every single one of them was an apology or an attempt to explain a decision that had upset users. … I’m going to run out of space here, so let’s jump to 2018 and skip over all the other mishaps and apologies and promises to do better … in the intervening years.”

@michel_slm mentioned Does Facebook Use Sensitive Data for Advertising Purposes? by José González Cabañas, Àngel Cuevas, Aritz Arrate, Rubén Cuevas, in Communications of the ACM. “we demonstrated that Facebook (FB) labels 73% of users within the EU with potentially sensitive interests (referred to as ad preferences as well), which may contravene the GDPR. FB assigns user’s different ad preferences based on their online activity within this social network. Advertisers running ad campaigns can target groups of users that have been assigned a particular ad preference (for example, target FB users interested in Starbucks). Some of these ad preferences may suggest political opinions (for example, Socialist party), sexual orientation (for example, homosexuality), personal health issues (for example, breast cancer awareness), and other potentially sensitive attributes. In the vast majority of the cases, the referred sensitive ad preferences are inferred from the user behavior in FB without obtaining explicit consent from the user. Then advertisers may reach FB users based on ad preferences tightly linked to sensitive information.”

I might be adding to this list in the future.

Comments on 2021-01-16 Facebook and WhatsApp

All the different links and stories make me wonder: “Is there a good link to a one simple text in German or English that explains why Facebook cannot be trusted, for a layperson?” 😉

– Björn Buckwalter 2021-01-20 14:00 UTC

Good question! What I took from all of the above is the Wikipedia page, Criticism of Facebook. It’s a bit more than a typical family member is willing to read but you can link to it and “just look at the number of issues!” and if you want to offer an example, just link to a specific section like “the app is listening to what people are saying!” (Providing ads through "snooping"Is Facebook listening to me? Why those ads appear after you talk about things) In other words, the tracking is so good it’s as if the the company is listening to your conversations. Super creepy.

That’s my current assessment of the situation.

– Alex 2021-01-20 14:19 UTC

Add Comment

2021-01-15 Traveller Madness

Yikes, in a moment of madness I bought the two Bundles of Holding for Classic Traveller. I’d say I’m set for life when it comes to Traveller, now. 😅

Maybe I should think about running a game. Slowly my itch to run games is coming back. I’m thinking that perhaps I’d like to try smaller groups, and something other than Halberds and Helmets for a bit.

Option 1, of course, is Classic Traveller. I still have an old Subsector lying around. The benefit of reusing old setting material is that you get an in-game setting history and plenty of non-player characters for free, buried in the campaign wiki.

Option 2, as is often the case, would be something like Burning Wheel, mostly because Judd Karlman keeps mentioning it in his podcast. There are many clunky bits I don’t particularly look forward to, but sometimes I remember elements of that one mini-campaign I ran, and I really liked those: rolling for contacts using the circles attribute; rolling to buy stuff using the resources attribute; one-roll conflict resolution, also known as Bloody Versus.

Then again, when I look at the list of things I enjoyed, perhaps it’s actually not in the rules per se, but in the approach to the game… I keep thinking that a way to play it using Just Halberds should be possible.

Perhaps the first thing to do is to find two people willing to play a game with, via a conference call, and then get excited about something via mutual inspiration.

Comments on 2021-01-15 Traveller Madness

Have you listened to the podcasts with Sean Nittner and Judd as they play Burning Wheel? It’s been wonderful listening to how they use the rules of BW to facilitate the conversation of the game.

In the first four sessions there are two dual of wits and one circles test. Most of it is conversations at the table.

Jeremy Friesen 2021-01-15 21:59 UTC

No, do you have a link?

– Alex 2021-01-16 00:19 UTC

It’s at: https://anchor.fm/actualplay

The Shoeless Peasant is the BW campaign. There’s also a Youtube or Twitch stream for this.

Jeremy Friesen 2021-01-16 15:01 UTC


In the mean time, I’ve started creating two characters, and my wife created a character for me – I’m not sure she really wants to play a 1:1 campaign with me, but at least she’s interested, and I’m interested, and I can use it for some Wiki-based semi-solo play. At least that’s the idea right now.

– Alex 2021-01-17 20:47 UTC

Started listening to Shoeless Peasant episode 1.

– 2021-01-18 22:14 UTC

See 2021-01-19 Listening to Burning Wheel Actual Play.

– Alex 2021-01-19 22:51 UTC

Add Comment

2021-01-14 Player Types

I posted a link to a blog post about player types on Mastodon and added: “Thinking about play styles… I dunno, I used to be a lot more interested in these things. These days I guess everybody should read such a typology at least once so that we can all at least use the same words. But then my take is that enjoyment is a multifaceted thing that I enjoy all the things, some more so, some less so, some more when I do it, some more when others do it. And I think casual players are important for cohesion (like at the work-place, actually).”

Happy surprise, @jaranta is a game scholar! “Post-doc at JYU & Coe GameCult. Specialized in philosophy, game studies and digital culture.” Some of my favourite subjects. Jonne told me that actual studies do exist. We don’t need to rely on our own individual observations (valuable as they may be). So now I’m reading a study. 🙂

Add Comment

2021-01-11 Hex Describe

I’ve been at it again... Text Mapper can generate Gridmapper-compatible maps. Sadly, that means I need an algorithm to create the maps, and that’s not easy. Recently, I needed some caves. What I did was take the existing mini-dungeon generator and just turn all the walls into rock walls. It doesn’t look bad, actually!

I’m using them for the post-apocalyptic mini-settings I’m working on and off with Josh Johnston. Visit Hex Describe → click “random Apocalypse” → pick “Josh Johnston (best for Apocalypse maps)” → click submit. If you’re lucky, you’ll spot some vat people caves, now!

I’m hoping the vat people caves are easy to extend with the structure in place, which I lifted from the structure that ktrey parker and had developed years ago to generate random dungeons. And still so good!

For the post-apocalypse I had developed a book name generator which I then used to populate libraries, and to create little library related book quests: finding book thieves, for example. I liked this so much that I wanted that for my Alpine mini settings, too. So recently I’ve been working on fantasy book titles, too. And missions to go along with them.

I need to improve those book missions, though.

Next up: more secret agents. 😀

Comments on 2021-01-11 Hex Describe

You may be interested in looking at Caves of Qud for book generation. It’s been a while since I played but while I remember the contents of the books being barely legible (whatever algorithm they were using was not convincing at all) the titles were all very evocative for the post-apocalyptic setting the game features. Dwarf Fortress is similarly a good source to mine for the fantasy generator.

Also, is there a possibility of the Hex Describe books referring to things generated in the mini-setting - perhaps even other books? One of my favourite tidbits of history is the anti-philosophy book “Incoherence of the Philosophers”, which prompted one philosopher to write a book rebutting it titled “Incoherence of the Incoherence”. Generating the context in which books were written could provide plenty of adventure seeds.

– Malcolm 2021-01-13 09:29 UTC

Indeed, referring to things in the setting is important! So currently I’m doing the following:

  • referring to magic schools and elements (that one was easy)
  • golem types that exist in the setting
  • referring to implied past wars based on magic items
  • referring to past empires of the implied setting, which are also used for ruins, wights, and the like
  • using the legend generator that is also used by bards and elsewhere
  • refer to the existing secret societies in the setting

Things I don’t do, but which I should be doing, perhaps:

  • generate a dozen scholars beforehand and have them write on topics (and refute each other) – an excellent idea!
  • instead of generating random elf and dwarf names for magic items, pick them from a pregenerated list of maybe a dozen each and then use the same names to generate books about them
  • refer to the warring factions in the setting

– Alex 2021-01-13 11:32 UTC

As for adventure seeds... perhaps I need some unique magic items, and legends about them, and thus songs and book titles could be a good place to “tell their story” (tiny as it may be). Perhaps other rare elements could be hinted at via book titles, like the passages to other realms. The teleporting monoliths, or the alien space ships come to mind.

– Alex 2021-01-14 13:07 UTC

Add Comment

2021-01-01 Oddmuse memory issues

Not really! It’s Phoebe… 😭

I just noticed that three of my server processes seem to consume a lot of memory. The following three wikis were reporting more than 2G used, each:

  • Community Wiki
  • Emacs Wiki
  • Oddmuse Wiki

Strangely, this site, or Campaign Wiki seemed not to be affected. I don’t see any performance problems.

Memory issues started on week 51, 2020

I’m suspecting that something else is going on. I restarted the the three processes using Hypnotoad, a hot deployment. But htop is still telling me that we’re using a lot of memory. When I sort by memory, however, I see it’s Phoebe at the very top.

Phoebe is consuming 40% if all memory

So now I’m wondering:

  • did I install some memory leaking version of Phoebe in week 51 of 2020?
  • is Phoebe being counted for the three sites instead of counting as Phoebe?

Something to investigate in the upcoming days. For now, I’m going to restart Phoebe.

Memory use is halved, now

I fixed the regular expressions for my Munin setup so that all the command line parameters I’m using to start Phoebe don’t trigger matches for other processes such as Community Wiki, Emacs Wiki, or Oddmuse Wiki.

That still leaves the question: What did I install, what changes did I make around December 14? I have no idea...

Comments on 2021-01-01 Oddmuse memory issues

Yep, Phoebe is at fault.

Memory use increasing for Phoebe only

So now I have to find a way to reproduce this result locally. Ideally we’ll find that there’s a single config file at fault... If I’m unlucky it’s because Phoebe 2 switched from Net::Server to Mojo::IOLoop and I’m somehow not closing the connections correctly, or some reference remains that prevents them from closing 100%. Yikes.

– Alex 2021-01-03 09:57 UTC

OK, I think I have a test setup that works. Let me know if you see an error in my thinking.

The test starts a server in the background, listening to a random port ($port) like all the other tests. I need its process id so that I can determine the memory it uses:

lsof -i:$port -F p

Given the process id ($pid), I can now ask for the memory size of the process:

ps -q $pid -o size

So what I’m doing is I get this size after starting the server, then I run 100 requests, and I get the size again. I run just Phoebe, no problem. If I install the speed bump extension (see 2020-12-25 Defending against crawlers), size goes up.

Well, I guess I’m fine with it going up a bit since we need to keep some numbers in memory, right? There problem is only truly a problem if the memory keeps going up. Let’s try.

100 requests:

$ make test TEST_FILES=t/leak.t
PERL_DL_NONLAZY=1 "/home/alex/perl5/perlbrew/perls/perl-5.30.0/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/leak.t
t/leak.t .. 1/?
#   Failed test 'No memory lost (42524 = 43256)'
#   at t/leak.t line 33.
#          got: '42524'
#     expected: '43256'
# Looks like you failed 1 test of 1.
t/leak.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests

Test Summary Report
t/leak.t (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=1, Tests=1,  2 wallclock secs ( 0.02 usr  0.00 sys +  0.71 cusr  0.07 csys =  0.80 CPU)
Result: FAIL
Failed 1/1 test programs. 1/1 subtests failed.
make: *** [Makefile:940: test_dynamic] Error 1


#   Failed test 'No memory lost (42528 = 43256)'


#   Failed test 'No memory lost (42524 = 43256)'

OK, clearly it just stays the same. Hm...

Perhaps it’s time to take a look at Devel::MAT. Too bad Devel::MAT::Dumper doesn’t install on the server. Something about xlocale.h missing. Installing libnewlib-dev (which provides /usr/include/newlib/xlocale.h) appears to have no effect:

Building Devel-MAT-Dumper
cc -I/home/alex/perl5/perlbrew/perls/perl-5.26.1/lib/5.26.1/x86_64-linux/CORE -DVERSION="0.42" -DXS_VERSION="0.42" -fPIC -c -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -o lib/Devel/MAT/Dumper.o lib/Devel/MAT/Dumper.c
In file included from lib/Devel/MAT/Dumper.xs:8:
/home/alex/perl5/perlbrew/perls/perl-5.26.1/lib/5.26.1/x86_64-linux/CORE/perl.h:738:13: fatal error: xlocale.h: No such file or directory
 #   include <xlocale.h>
compilation terminated.

OK. Before doing anything else, I’m going to use Perlbrew to upgrade Perl 5.26 to 5.32. Then we’ll see about installing Devel::MAT.

– Alex 2021-01-03 17:38 UTC

OK, that worked! I wrote a little extension that allows me to write a heap dump to disk on the server.

Install it in your conf.d directory, set the known fingerprints to the hash of your favourite fingerprint in your config file (or edit the heap-dump.pl file):

our @known_fingerprints = qw(

Install the Devel::MAT::Dump package on your server, and Devel::MAT on your development machine. Restart Phoebe.

Finally, visit the path /do/heap-dump on your Phoebe instance. You’ll be prompted for your client certificate. Point your Gemini client to the one whose fingerprint you added your config above, and now it should say: “Heap Dump Saved” – copy the file it created to your local machine and examine it, using the user guide linked above as your guide.

Here’s the output right after I restarted it:

$ pmat phoebe.pmat
Perl memory dumpfile from perl 5.32.0 non-threaded
Heap contains 217470 objects
pmat> largest
HASH(24948) at 0x558c89875548=strtab: 1.7 MiB
HASH(5672) at 0x558c89b3c450: 261.0 KiB
SCALAR(PV) at 0x558c89beaa18: 215.7 KiB
ARRAY(1,!REAL) at 0x558c89899098: 92.5 KiB
HASH(2231) at 0x558c8aac5240: 84.3 KiB
others: 15.2 MiB

I’m going to let it run over night and we’ll see how it does tomorrow.

– Alex 2021-01-03 21:34 UTC

OK, a few hours later...

$ pmat phoebe.pmat
Perl memory dumpfile from perl 5.32.0 non-threaded
Heap contains 313361 objects
pmat> largest
HASH(39010) at 0x558c89875548=strtab: 2.4 MiB
HASH(8940) at 0x558c89b8f2a8: 337.6 KiB
HASH(5672) at 0x558c89b3c450: 261.0 KiB
SCALAR(PV) at 0x558c98eaf6e8: 215.7 KiB
SCALAR(PV) at 0x558c89beaa18: 215.7 KiB
others: 21.0 MiB

That is not very encouraging. There apparently is no single thing that grew in limitless ways. 😢

pmat> count
  Kind        Count (blessed)        Bytes  (blessed)
  ARRAY       11108         7      1.2 MiB  520 bytes
  CODE        10387                1.3 MiB
  GLOB        14796        16      2.1 MiB    2.4 KiB
  HASH        23305      3182      7.4 MiB  849.6 KiB
  INVLIST       655              674.4 KiB
  IO             33        33      5.2 KiB    5.2 KiB
  PAD          7870                1.5 MiB
  REF         41199         1    965.6 KiB   24 bytes
  REGEXP       2503       253    528.0 KiB   53.4 KiB
  SCALAR     200878        12      8.0 MiB 1022 bytes
  STASH         627              801.6 KiB
  -------    ------ ---------    --------- ----------
  (total)    313361      3504     24.5 MiB  912.1 KiB

What I don’t really understand is that the numbers I’m seeing are all around 20 MiB, and yet Munin is telling me that the minimum memory is 97.31 MiB and the maximum is 266.54 MiB (also the current value). So where are the 170 MiB I’m missing?

Hm. Let’s take a look at the two scalar values we saw up in the list of “largest” SVs. Surprisingly, they are exactly the same size. Let’s examine them.

SCALAR(PV) at 0x558c98eaf6e8: 215.7 KiB
SCALAR(PV) at 0x558c89beaa18: 215.7 KiB

pmat> show 0x558c98eaf6e8
SCALAR(PV) at 0x558c98eaf6e8 with refcount 1
  size 215.7 KiB (220840 bytes)
  PV="// This Source Code Form is subject to the terms of the Mozilla "...
  PVLEN 220798

pmat> show 0x558c89beaa18
SCALAR(PV) at 0x558c89beaa18 with refcount 1
  size 215.7 KiB (220840 bytes)
  PV="// This Source Code Form is subject to the terms of the Mozilla "...
  PVLEN 220798

OK, that is surprising. I don’t recognize this text. Perhaps there are more like it? After all, the count shows 200878 scalars. What’s with the refcount? Who is keeping those references?

pmat> inrefs --all 0x558c98eaf6e8

pmat> inrefs --all 0x558c89beaa18
s  a constant  CODE(PP) at 0x558c89b8f080

OK, nobody is referring to the first one, which is even more surprising. Perhaps it’s about to be garbage collected? The second value gives us a hint:

pmat> show 0x558c89b8f080
CODE(PP) at 0x558c89b8f080 with refcount 1
  size 128 bytes
  named as &IO::Socket::SSL::PublicSuffix::_builtin_data
  no hekname
  stash=STASH(19) at 0x558c89bd0f98
  glob=GLOB(&*) at 0x558c89bea460
  location=/home/alex/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Socket/SSL/PublicSuffix.pm line 348
  scope=CODE() at 0x558c89c4f820
  pad[0]=PAD(1) at 0x558c89b8f188

The SSL stuff. 😭 This reminds me of the only test in gemini-diagnostics Phoebe is currently failing: “Server should send a close_notify alert before closing the connection.”

Perhaps that’s my problem. The SSL/TLS connections aren’t closed correctly, so they hang around.

I’ll have to think about this some more.

– Alex 2021-01-04 08:12 UTC

Ten hours later, Munin says 418.04 MiB, and the heap dump says:

pmat> largest
HASH(46066) at 0x558c89875548=strtab: 3.3 MiB
HASH(15996) at 0x558c89b8f2a8: 631.0 KiB
SCALAR(PV) at 0x558c8af5d8a0: 305.0 KiB
HASH(5672) at 0x558c89b3c450: 261.0 KiB
SCALAR(PV) at 0x558c98cc4d48: 232.0 KiB
others: 24.5 MiB

pmat> count
  Kind        Count (blessed)        Bytes  (blessed)
  ARRAY       12116         7      1.3 MiB  520 bytes
  CODE        11395                1.4 MiB
  GLOB        14796        16      2.1 MiB    2.4 KiB
  HASH        30361      3182      9.6 MiB  849.6 KiB
  INVLIST       655              674.4 KiB
  IO             33        33      5.2 KiB    5.2 KiB
  PAD          8878                1.8 MiB
  REF         56319         1      1.3 MiB   24 bytes
  REGEXP       2510       253    529.5 KiB   53.4 KiB
  SCALAR     228143        12      9.7 MiB 1022 bytes
  STASH         627              801.6 KiB
  -------    ------ ---------    --------- ----------
  (total)    365833      3504     29.2 MiB  912.1 KiB

– Alex 2021-01-04 17:40 UTC

Ok, let’s look at the ever growing hash at 0x558c8d32a720:

pmat> values 0x558c89b8f2a8
  {ptr_0x558c8d3d5f40} REF() at 0x558c8d29c850 => HASH(1) at 0x558c8d2ab990
  {ptr_0x558c8d3eabc0} REF() at 0x558c8d2ec190 => HASH(1) at 0x558c8d29fc48
  {ptr_0x558c8d3eb470} REF() at 0x558c8d3bdc18 => HASH(1) at 0x558c8d3bdc30
  {ptr_0x558c8d3f6cd0} REF() at 0x558c8d2f3e00 => HASH(1) at 0x558c8d32a720
  ... (15946 more)

pmat [more]> values 0x558c8d32a720
  {"ssleay_verify_callback!!func"} REF() at 0x558c8d3a4758 => CODE(PP,closure) at 0x558c8d32aac8

pmat> values 0x558c8d3bdc30
  {"ssleay_verify_callback!!func"} REF() at 0x558c8d3bdca8 => CODE(PP,closure) at 0x558c8d3bd8e8

pmat> values 0x558c8d29fc48
  {"ssleay_verify_callback!!func"} REF() at 0x558c8d2ac260 => CODE(PP,closure) at 0x558c8d32aac8

pmat> values 0x558c8d2ab990
  {"ssleay_verify_callback!!func"} REF() at 0x558c8d29d078 => CODE(PP,closure) at 0x558c8d32aac8

Ugh, SSL again! Verify callback – that also rings a bell. Phoebe code:

  SSL_verify_mode => SSL_VERIFY_PEER,
  SSL_verify_callback => \&verify_fingerprint);

Hm. So now I’m thinking: a hash with 16,000 SSL sockets?

– Alex 2021-01-04 20:05 UTC

I’ve made a small change: as the stream closes, I undefine the $data reference. I figured that something close to this loop (”for every socket?”) must be keeping references around.

  address => $address,
  port => $port,
  tls => 1,
  tls_cert => $server->{cert_file},
  tls_key  => $server->{key_file},
} => sub {
  my ($loop, $stream) = @_;
  my $data = { buffer => '', handler => \&handle_request };
  $stream->on(read => sub {
    my ($stream, $bytes) = @_;
    $log->debug("Received " . length($bytes) . " bytes");
    $data->{buffer} .= $bytes;
    $data->{handler}->($stream, $data) });
  $stream->on(close => sub {
    my $stream = shift;
   }) });

Twelve hours later and Munin reports a current RSS of 206 MiB. That’s much better than anticipated! I’ll keep it running for another day or so, see how it develops.

– Alex 2021-01-06 07:55 UTC

A few more hours and memory is down to 174 MiB, so memory actually went down. Amazing! 😄

pmat> largest
HASH(29739) at 0x55cf4fc7c548=strtab: 2.0 MiB
HASH(5672) at 0x55cf4ff43d60: 261.0 KiB
SCALAR(PV) at 0x55cf4fff2238: 215.7 KiB
HASH(4782) at 0x55cf4ff96aa8: 176.1 KiB
ARRAY(1,!REAL) at 0x55cf4fca01d8: 92.5 KiB
others: 17.2 MiB

pmat> count
  Kind        Count (blessed)        Bytes  (blessed)
  ARRAY       10198         7      1.2 MiB  520 bytes
  CODE         9603                1.2 MiB
  GLOB        14374        16      2.1 MiB    2.4 KiB
  HASH        11712      3181      5.0 MiB  884.5 KiB
  INVLIST       583              652.3 KiB
  IO             33        33      5.2 KiB    5.2 KiB
  PAD          7079                1.3 MiB
  REF         24869         1    582.9 KiB   24 bytes
  REGEXP       2303       253    485.8 KiB   53.4 KiB
  SCALAR     173493        12      6.8 MiB 1022 bytes
  STASH         610              776.3 KiB
  -------    ------ ---------    --------- ----------
  (total)    254857      3503     19.9 MiB  946.9 KiB

– Alex 2021-01-06 12:14 UTC

Too bad with memory climbing again. That’s wasn’t the answer, or not the entire answer, sadly! Back at 325 MiB.

Phoebe rising

When I look at this, at all the complications, and the memory used, and compare it to the implementations based on Net::Server (”old style”) with it’s 23 MiB (used for the Gopher and Finger server on ports 70 and 79, as well as the SSL enabled Gopher on port 7334) – I don’t have happy thoughts.

pmat> count --blessed
  Kind                                             Count (blessed)        Bytes  (blessed)
  ARRAY                                            11506         7      1.3 MiB  520 bytes
      Math::BigInt::Calc                               0         3      0 bytes  216 bytes
      Net::DNS::RR::OPT                                0         2      0 bytes  160 bytes
      Class::Struct::Tie_ISA                           0         1      0 bytes   64 bytes
      Net::DNS::RR                                     0         1      0 bytes   80 bytes
  CODE                                             10786                1.3 MiB
  GLOB                                             14792        16      2.1 MiB    2.4 KiB
      IO::Socket::IP                                   0        15      0 bytes    2.2 KiB
      IO::Socket::SSL                                  0         1      0 bytes  152 bytes
  HASH                                             26086      3182      8.2 MiB  884.6 KiB
      Specio::DeclaredAt                               0      1537      0 bytes  324.2 KiB
      Specio::Constraint::Simple                       0      1428      0 bytes  483.8 KiB
      DateTime::Format::Builder::Parser::Regex         0        67      0 bytes   15.9 KiB
      Specio::Constraint::Parameterizable              0        60      0 bytes   22.1 KiB
      Mojo::IOLoop::Server                             0        15      0 bytes    3.7 KiB
      Specio::Constraint::ObjectIsa                    0        15      0 bytes    5.5 KiB
      Specio::Constraint::Enum                         0        10      0 bytes    3.8 KiB
      Specio::Constraint::Union                        0         7      0 bytes    2.2 KiB
      Specio::Constraint::AnyCan                       0         6      0 bytes    2.2 KiB
      Specio::Constraint::ObjectCan                    0         5      0 bytes    1.8 KiB
      (others)                                         0        32      0 bytes   19.5 KiB
  INVLIST                                            651              662.6 KiB
  IO                                                  33        33      5.2 KiB    5.2 KiB
      IO::File                                         0        33      0 bytes    5.2 KiB
  PAD                                               8269                1.6 MiB
  REF                                              47156         1      1.1 MiB   24 bytes
      IO::Socket::SSL::SSL_HANDLE                      0         1      0 bytes   24 bytes
  REGEXP                                            2491       253    525.4 KiB   53.4 KiB
      Regexp                                           0       253      0 bytes   53.4 KiB
  SCALAR                                          211663        12      8.2 MiB 1022 bytes
      Encode::XS                                       0         5      0 bytes  360 bytes
      JSON::PP::Boolean                                0         4      0 bytes  288 bytes
      Cpanel::JSON::XS                                 0         2      0 bytes  292 bytes
      Math::BigInt                                     0         1      0 bytes   82 bytes
  STASH                                              627              803.2 KiB
  --------------------------------------------    ------ ---------    --------- ----------
  (total)                                         334060      3504     25.8 MiB  947.1 KiB

I don’t understand what this means… I looks like everything is OK?

Comparing the two heap dumps I have:

$ pmat-counts phoebe-2021-01-06T*
  Kind        Count (blessed)        Bytes  (blessed)
  ARRAY       10198         7      1.2 MiB  520 bytes
  CODE         9603                1.2 MiB
  GLOB        14374        16      2.1 MiB    2.4 KiB
  HASH        11712      3181      5.0 MiB  884.5 KiB
  INVLIST       583              652.3 KiB
  IO             33        33      5.2 KiB    5.2 KiB
  PAD          7079                1.3 MiB
  REF         24869         1    582.9 KiB   24 bytes
  REGEXP       2303       253    485.8 KiB   53.4 KiB
  SCALAR     173493        12      6.8 MiB 1022 bytes
  STASH         610              776.3 KiB
  -------    ------ ---------    --------- ----------
  (total)    254857      3503     19.9 MiB  946.9 KiB
  Kind        Count         (blessed)            Bytes              (blessed)
  ARRAY       11506(+1308)          7          1.3 MiB(+119.4 KiB)  520 bytes
  CODE        10786(+1183)                     1.3 MiB(+147.9 KiB)
  GLOB        14792(+418)          16          2.1 MiB(+62.0 KiB)     2.4 KiB
  HASH        26086(+14374)      3182(+1)      8.2 MiB(+3.2 MiB)    884.6 KiB(+168 bytes)
  INVLIST       651(+68)                     662.6 KiB(+10.3 KiB)
  IO             33                33          5.2 KiB                5.2 KiB
  PAD          8269(+1190)                     1.6 MiB(+330.7 KiB)
  REF         47156(+22287)         1          1.1 MiB(+522.4 KiB)   24 bytes
  REGEXP       2491(+188)         253        525.4 KiB(+39.7 KiB)    53.4 KiB
  SCALAR     211663(+38170)        12          8.2 MiB(+1.5 MiB)   1022 bytes
  STASH         627(+17)                     803.2 KiB(+26.8 KiB)
  -------    -------------- -------------    --------------------- ----------------------
  (total)    334060(+79203)      3504(+1)     25.8 MiB(+5.9 MiB)    947.1 KiB(+168 bytes)

The difference seems minimal. And yet, ps -e rss reports 325 MiB instead of 174 MiB used.

– Alex 2021-01-06 21:30 UTC

Well… What do we have? I have three heap dump files for the current process:

23M 2021-01-06 13:14 phoebe-2021-01-06T13:14.pmat
29M 2021-01-06 22:35 phoebe-2021-01-06T22:35.pmat
42M 2021-01-07 21:03 phoebe-2021-01-07T21:03.pmat

RSS reported by ps is 174 MiB, 325 MiB, and 822 Mib. It’s not looking good. What amazes me, however, is that the memory usage as reported by Munin using “/proc/meminfo” doesn’t seem to reflect that. Or maybe it does?

Anyway, I’m experimenting with pmat tools. The output of “pmat-leakreport” is extremly long. It’s pages and pages of this:

$ pmat-leakreport phoebe-2021-01-0[67]*
LEAK[2] HASH(1) at 0x55cf5e4d3e00
LEAK[2] REF() at 0x55cf5e472f08
LEAK[2] SCALAR(PV) at 0x55cf5e2c7318
LEAK[2] SCALAR(UV) at 0x55cf5e4b8b78
LEAK[2] HASH(1) at 0x55cf5ab96ce0
LEAK[2] HASH(1) at 0x55cf5b69f8b8
LEAK[2] HASH(1) at 0x55cf604236c8
LEAK[2] UNDEF() at 0x55cf62d869c0
LEAK[2] GLOB($*) at 0x55cf5e039088
LEAK[2] HASH(1) at 0x55cf60a59478
LEAK[2] REF() at 0x55cf62887cf0
LEAK[2] SCALAR(UV) at 0x55cf5e49a430
LEAK[2] REF() at 0x55cf63108b28
LEAK[2] UNDEF() at 0x55cf5d57cfb8
LEAK[2] SCALAR(UV) at 0x55cf5e401578
LEAK[2] UNDEF() at 0x55cf6086c7a0
LEAK[2] REF() at 0x55cf5ecf80c0
LEAK[2] REF() at 0x55cf5e369e00
LEAK[2] UNDEF() at 0x55cf5fa091e8
LEAK[2] HASH(1) at 0x55cf5bc07e48
LEAK[2] UNDEF() at 0x55cf62a23f60
LEAK[2] UNDEF() at 0x55cf5c78a818
LEAK[2] SCALAR(UV) at 0x55cf5e48d5c0
LEAK[2] REF() at 0x55cf6137ebf8

I don’t know what to make of this. The documenation says:

A leaking SV is one that appears and then never gets touched again. To detect them, we need to look for SVs that appear between two files, and then don’t disappear again. Any that do disappear were simply temporaries and can be ignored.

So, all of these are new appearances, and they don’t disappear again. 😟

Let’s look at them, starting at the end:

pmat> identify 0x55cf6137ebf8                                                                                                                      
REF() at 0x55cf6137ebf8 is:
└─value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf6137ec58, which is:
  └─(via RV) value {ptr_0x55cf614d66c0} of HASH(35071) at 0x55cf4ff96aa8, which is:
    └─not found

pmat> identify 0x55cf5e48d5c0                                                                                                                      
SCALAR(UV) at 0x55cf5e48d5c0 is:
└─value {"\\0"} of HASH(1) at 0x55cf5e48d5a8, which is:
  └─(via RV) value {rakkestad} of HASH(725) at 0x55cf5e44f090, which is:
    └─(via RV) value {no} of HASH(1526) at 0x55cf4ffdf998, which is:
      └─(via RV) value {tree} of HASH(2)=IO::Socket::SSL::PublicSuffix at 0x55cf5e546aa0, which is:
        └─(via RV) value {"1"} of HASH(1) at 0x55cf4ffd88a8, which is:
          └─the lexical %default at depth 1 of CODE(PP) at 0x55cf4ffd8440, which is:
            └─the symbol '&IO::Socket::SSL::PublicSuffix::default'

pmat> identify 0x55cf5c78a818                                                                                                                      
UNDEF() at 0x55cf5c78a818 is:
└─pad temporary 29 at depth 1 of CODE(PP,closure) at 0x55cf5c719058, which is:
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf5c681250, which is:
  │ └─(via RV) value {ptr_0x55cf5c8999e0} of HASH(35071) at 0x55cf4ff96aa8, which is (*A):
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf5c718b00, which is:
  │ └─(via RV) value {ptr_0x55cf5c891a30} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf5c86dea8, which is:
  │ └─(via RV) value {ptr_0x55cf5c88cff0} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf5c86def0, which is:
  │ └─(via RV) value {ptr_0x55cf5c89a180} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf5c7fc768, which is:
  │ └─(via RV) value {ptr_0x55cf5c871bd0} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf5c78a7d0, which is:
  │ └─(via RV) value {ptr_0x55cf5c8991c0} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  └─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf5c7fc7f8, which is:
    └─(via RV) value {ptr_0x55cf5c8995d0} of HASH(35071) at 0x55cf4ff96aa8, which is:
      └─not found

pmat> identify 0x55cf62a23f60                                                                                                                      
UNDEF() at 0x55cf62a23f60 is:
└─the lexical $ctx_store at depth 1 of CODE(PP,closure) at 0x55cf62a24770, which is:
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf62abd3d0, which is:
  │ └─(via RV) value {ptr_0x55cf62ba4960} of HASH(35071) at 0x55cf4ff96aa8, which is (*A):
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf62991a00, which is:
  │ └─(via RV) value {ptr_0x55cf62b68370} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf62667430, which is:
  │ └─(via RV) value {ptr_0x55cf62b628d0} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf62667f70, which is:
  │ └─(via RV) value {ptr_0x55cf62ba5100} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf62a24b48, which is:
  │ └─(via RV) value {ptr_0x55cf62b72e10} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  ├─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf62a24398, which is:
  │ └─(via RV) value {ptr_0x55cf62ba4550} of HASH(35071) at 0x55cf4ff96aa8, which is:
  │   └─not found
  └─(via RV) value {"ssleay_verify_callback!!func"} of HASH(1) at 0x55cf62abd160, which is:
    └─(via RV) value {ptr_0x55cf62ba4140} of HASH(35071) at 0x55cf4ff96aa8, which is:
      └─not found

pmat> values 0x55cf5bc07e48                                                                                                                        
  {"ssleay_verify_callback!!func"} REF() at 0x55cf5bb9e678 => CODE(PP,closure) at 0x55cf5bd3d8d8

OK, I’d it’s pretty clear by now: the “ssleay_verify_callback” stuff is bonkers!

Verify callback – remember the Phoebe code? Also note how “%default” is mentioned above.

  SSL_verify_mode => SSL_VERIFY_PEER,
  SSL_verify_callback => \&verify_fingerprint);

But I just read the IO::Socket::SSL man page. It says: “If you have a server and it looks like you have a memory leak you might check the size of your session cache. Default for Net::SSLeay seems to be 20480, see the example for SSL_create_ctx_callback for how to limit it.”

OK. I’m going to try that instead. The previous code had no effect. I’m limiting the session cache to 128, like in the example provided. Restarted the server. Memory is down below 100 MiB again:

# pgrep -f -l "phoebe"
29000 phoebe
# ps -q 29000 -eo rss,size
73392 62360

– Alex

I’m surprised. This seems to have helped!

Image 5

Ignoring the spike of Emacs Wiki and just looking Phoebe: we seem to be stable at 250 MiB.


Hm… Then again, at the same time, I changed the “phoebe.service” file for systemd:


That is… uncannily close! So here’s what I’m doing now: I decreased the SSH cache from 128 to 64, and I didn’t change the “phoebe.service” file. Let’s see if memory use is halved, i.e. the cache is relevant, or not, i.e. the memory limits are relevant.

– Alex 2021-01-09 15:00 UTC

OK, reducing the cache size didn’t help. We’re back at 250 MiB. This is a memory leak of sorts, but the resource limit I have in the service file is working.

I think I’m going to reduce those limits:


Let’s see whether it works well enough!

– Alex 2021-01-10 09:54 UTC

It works well enough. I added this to the documentation and stopped trying to understand where Mojo::IOLoop::TLS, IO::Socket::SSL, Net::SSLeay, or OpenSSL are failing.

– Alex 2021-01-11 11:26 UTC

Add Comment

2020-12-31 My feed aggregators

@solderpunk recently announced an interesting change to CAPCOM:

“On the first day of each month, a cron job runs the … pipeline to randomly select 100 unique feeds … Those feeds are then aggregated by CAPCOM for the remainder of the month. This way the scope of the CAPCOM instance can grow without limit over time, but the actual amount of new content to be found each day should remain fairly constant over time.”

I like it!

As for my own feed agregators, I actually wrote two of them in 2020.

For the traditional web, I wrote a replacement for Planet Venus which was powering the RPG Planet because Planet Venus is written in Python 2 and I was unable to port it to Python 3. Writing my own was more fun!

It powers all three of the RPG Planets I run:

I also use it for a list of other blogs I’m following that aren’t directly RPG related:

As you can see I didn’t bother to change the theme, haha! 😄

Since I also started getting interested in Gemini, I rewrote moku pona, the Gopher “aggregator” I had. The reason I’m using quotes here is that moku pona 1 was just a URL watcher: given a list of URLs, it would check them for changes and move changed resources to the top of the list. When I added support for Gemini feeds, I realised that this was not very different from supporting all sorts of feeds, so now moku pona supports URL watching, as well as RSS and Atom feeds, via Gopher, Gemini, and HTTP. The fact that it works amazes me every day. 😆

The main difference in terms of user interface is the granularity. Jupiter a “traditional” aggregator that shows an excerpt of every post: site name, post title, post except. Mona pona on the other hand simply shows the date of the last change per site, with the nickname you provided.

Add Comment



You probably want to contact me via one of the means listed on the Contact page. This is probably the wrong place to do it. 😄

– Alex Schroeder 2020-05-22 12:19 UTC

Referrers: Diary lostapp.ru/soft/gridmapper Diary d4 Caltrops: Hex-Describe & The Beauty Of Embracing Random Non-Sequitu...