Debugging MediaWiki extensions

Recently while updating the Auth_PHPBB MediaWiki extension I encountered some frustration debugging some issues happening on extension load. Here are some things that helped me track down the problem.

First, it’s useful to enable one or more debug flags in LocalSettings.php:

// Debugging
if (true) {
    // Show exceptions and DB backtraces
    $wgShowExceptionDetails = true;
    $wgShowDBErrorBacktrace = true;

    // Write out MediaWiki debug log messages
    $wgDebugLogFile = "/tmp/mw-debug.log";

    // And/or route specific log groups to individual files
    $wgDebugLogGroups = [
        "Auth_phpBB" => "/tmp/mw-debug-Auth_phpBB.log",
        "PluggableAuth" => "/tmp/mw-debug-PluggableAuth.log",
    ];

    // Disable ALL caching with the following two lines
    $wgEnableParserCache = false;
    $wgCachePages = false;
}

Enabling the full MediaWiki debug log file was the most helpful for debugging the extension as in some failure scenarios the server doesn’t return any content to the client. This was particularly frustrating since I expected such failures to output something in the PHP error log, but no. The PHP error log is also a very useful place to look for other failure scenarios though.

Remember to disable any debugging flags after you’re done with development and delete any debugging output files that were created!

Queer Books for Pride

For the month of June I posted a queer book a day to my Twitter feed under #queerbooksforpride. Here are all of them in one place. Some of these center queer relationships, others have queer characters as a natural part of the world / plot. All are fantastic.

  • The Starless Sea – Erin Morgenstern
  • Tarot Sequence – K.D. Edwards
  • Soulcraft Sequence – Max Gladstone (Full Fathom Five; Last First Snow)
  • The Lady’s Guide to Celestial Mechanics – Olivia Waite
  • Karen Memory – Elizabeth Bear (also Jacob’s Ladder, Carnival)
  • Magic or Die – JP Jackson
  • Saint of Steel series – T Kingfisher (Paladin’s Hope)
  • Lord of the White Hell – Ginn Hale
  • Broken Earth Trilogy – NK Jemisin
  • Fall of the Kings – Ellen Kushner (also Swordspoint)
  • The Goblin Emperor – Katherine Addison
  • The House in the Cerulean Sea – TJ Klune (also Under the Whispering Door)
  • Red, White, and Royal Blue – Casey McQuiston
  • A Pslam for the Wild Built – Becky Chambers
  • A Taste of Honey – Kai Ashante Wilson
  • Ancillary Justice – Ann Leckie
  • Battle of the Linguist Mages – Scotto Moore
  • A Master of Djinn – P. Djèlí Clark
  • Winter’s Orbit – Everina Maxwell
  • Nightrunner Series – Lynn Flewelling
  • The Prince and the Dressmaker – Jen Wang
  • Priory of the Orange Tree – Samantha Shannon

python, bytecode, and read-only containers

Upon first access, python compiles .py code into bytecode and stores it in .pyc files. Subsequent uses of those python sources are read from the .pyc files without needing to re-compile. This make startup time, but not runtime, faster.

But what about read-only filesystems? If python is running on a read-only filesystem no .pyc files are written and every use of a .py file involves compiling the code afresh. Everything works, the startup time of a script is just a little slower.

Read-only filesystems within a container are a security best practice in production environments. Often in kubernetes deployment manifests you might see something like:

securityContext:
readOnlyRootFilesystem: true

And if you’re running python only once within the container, the container has a little bit of overhead at startup as it compiles into bytecode and then everything is in memory and off it goes. But if you’re running python multiple times, or want to make even the single run start faster, we can pre-compile the code when we build the container with the built-in python compileall library.

# Compile code on sys.path (includes system packages and CWD):
RUN python -c "import compileall; compileall.compile_path(maxlevels=10)"

# Compile code in a directory:
RUN python -m compileall /path/to/code

This moves the compilation overhead to the container build where it happens once, and out of the startup.

Thanks to Itamar Turner-Trauring at https://pythonspeed.com/ for their excellent Production-ready Docker packaging for Python slide deck with this gem.

Sonos – suddenly unreliable

We’ve had our Sonos system, connected via wifi, for 5+ years and it’s been rock solid until late January. Suddenly, through no changes to our wifi (router, router firmware, or connected devices) or Sonos system, it started being completely unreliable. This seems to have coincided with the S2 v14 release on 2022-01-25 but that could have been coincidence.

Examples of problems:

  • Speakers would disappear from the controllers (macOS & Android apps).
  • Speakers would stop playing but show as playing in the controllers.
  • Speakers would be playing but show as stopped playing in the controllers.
  • Speakers would not respond to play/stop input from the controllers.
  • Different controllers would show different speakers available.

I did the usual things of restarting the wireless router & restarting the speakers. At the recommendation of Sonos support I changed the wireless channel on my router. All of these worked for 30 or so minutes and then things went sideways again.

Finally, fed up with it all, we plugged a speaker directly into the router instead of having it connect over the wireless thus forming its own separate wireless network. Everything has worked perfectly since then.

This isn’t exactly convenient given where the wireless router is, but our Sonos system is once again functional instead of being a collection of 6 very expensive paperweights.

MediaWiki VisualEditor: slashes and namespaces

Recently I upgraded the wiki at Distributed Proofreaders to the MediaWiki 1.35 LTS release. This comes with the fancy new VisualEditor which should be great for our users who already deal with phpBB markup in the forums and our own custom markup for project formatting.

Setting up the VisualEditor was a bit of a head-scratcher in a couple of ways and hopefully this helps others who encounter similar problems.

Error contacting the Parsoid/RESTBase server (HTTP 404)

This one frustrated me for quite some time. Everything pointed to setting

AllowEncodedSlashes NoDecode

in our Apache config, but that wasn’t working for me. For reasons I don’t understand, I needed to include this in both the :80 and :443 VirtualHost sections, not just the :443 which was serving all traffic.

Error contacting the Parsoid/RESTBase server (HTTP 500)

This was thankfully pretty obvious by looking in the php_errors log. As the VisualEditor Troubleshooting section calls out, our $wgTmpDirectory had the wrong write permissions.

Enabling for Namespaces

The documentation says that to change the namespaces that the VisualEditor will be used on, you use the English canonical names. To get this to work, we needed to use the namespace constants instead. Note that the MW code will include all content namespaces as enabled by default so you only need to include those if you want to disable them.

$wgVisualEditorAvailableNamespaces = [
// includes Content namespaces by default (Main)
NS_PROJECT => true,
NS_PROJECT_TALK => true,
NS_TALK => true,
NS_USER_TALK => true,
];

Fixing HP Scanner “malware” warning on macOS

It’s tax season here in the US and yesterday I plugged in our HP OfficeJet J4500 to scan in some documents that were mailed to us. I don’t use it much but imagine my consternation when I received a malware message “HPScanner.app will damage your computer” from macOS Big Sur and the scanner refusing to work.

I spent a good 30 minutes Googling, uninstalling drivers, reinstalling drivers, with zero success. I went to rant to my husband and surprisingly his Big Sur laptop was working just fine! And while usually I’m the stubborn technologist in the relationship, he was the one who solved this one.

The solution is to download and install the HP 5.1.1 Printer Software Update from Apple, uninstall the printer, and reinstall it.

Why is that so incredibly hard to figure out? Because HP’s website is an incompetent mess. The usual problem resolution path of going to HP’s Driver Download website fails because they don’t have macOS 11 drivers for the OfficeJet J4500. If you click through far enough you’ll arrive on their My operating system is not listed on HP Software and Driver Downloads page which routes you to the Apple website linked above. So there is a driver for the printer it’s just difficult to find. You might need to download and run the “HP Uninstaller” first. I had already done this in my troubleshooting attempt so it’s unclear if this is a prerequisite or not.

Other things that I discovered during my attempt to fix this:

  • Downloading the HP Easy Scan software from the App Store. This doesn’t include the OS-level driver and without it the app can’t communicate with the scanner.
  • Using the Generic PCL driver. While the printer will happily print with the Generic PCL driver (courtesy of Gutenprint) it doesn’t include the drivers for scanning.
  • The reason this likely worked for my husband and not me is that he updated his printer drivers while still on Catalina after HP screwed the pooch and mistakingly revoked certificates for their printer drivers in Oct 2020. I very very seldom print or scan so it’s not surprising that my system was out of date.

poetry auth via .netrc

poetry, the python package manager, provides several ways of authenticating against a repository. What isn’t explicitly documented, because it’s an implicit dependency, is that poetry can also use the ~/.netrc file for authentication when fetching packages.

poetry uses requests under the covers, and requests falls back to the ~/.netrc file. This is the same fallback method for pip for the same reason.

There are several (probably bad) reasons why someone would want to do this vs one of the explicit methods given by poetry. One that comes to mind is needing to install python packages from a private repository from inside a docker container by simply volume mapping the host’s ~/.netrc file to have poetry use the right creds.

This approach probably won’t work when publishing packages — caveat emptor.

While I’m not suggesting that this is a best practice, it’s good to know that it’s an available method in some extreme edge cases.

Sonos, Plex, and “Unable to browse music” on artist search

It’s taken me several months to connect the dots, but I’ve finally figured out why sometimes Sonos is unable to browse music from Plex artist search results. And it’s all about having multiple libraries in Plex.

The problematic behavior is this scenario:

  1. In the Sonos app, search for an artist on Plex
  2. Sonos shows the results successfully
  3. Click on the artist
  4. Sonos returns “Unable to browse music”

If, instead, you search for an album you can click on the results, see the tracks, and play the music. Sometimes this seems to work, sometimes it doesn’t, and it’s taken me months to find the pattern.

The problem is that my Plex system has two libraries: mine and my husband’s. Overall Sonos handles this well and we can play whatever music from whatever library.

But while artist searches are scoped to all libraries in Plex, browsing artist results is only scoped to the current Plex library. If the current Plex library contains the artist you can browse the search results. If not, you get the “Unable to browse music”. If you change the Plex source to the library with the artist it will work again.

I’m not certain if this is a limitation in the API that Sonos is using to interact with Plex, or if this is a bug in how Sonos is using that API, but knowing what the problem and how to work around it is a start.

Lemon Poppy Seed Bread

This recipe is from my Granny Dot who died last year. I’ve been missing her lately and decided tonight was the night for her Poppy Seed Bread.

I’ve annotated where my version differs from my Granny’s with *. See the bottom of the post for details on her original version.

Lemon Poppy Seed Bread

Makes 2x loaves (8.5″ x 4″ pans) or 1 bundt cake.

Bread

  • 3 c white flour
  • 2 1/4 c sugar
  • 1 1/2 tbsp poppyseeds
  • 1 1/2 tsp baking soda
  • 1 1/2 tsp salt
  • 3 eggs, lightly beaten
  • 1 1/4c milk*
  • 1/4 c lemon juice*
  • 1/2 c vegetable oil*
  • 1/2 c butter*
  • 1 1/2 tsp vanilla extract
  • 1 1/2 tsp almond extract

Preheat the oven to 350 degrees. Butter and flour the 2x loaf pans or the bundt pan, whichever you are using.

Mix dry ingredients (first 5) together in a mixing bowl. Add in remaining ingredients and mix well. Place the batter in the loaf or bundt pans (if using the loaf pans, split the batter evenly between them both).

Cook at 350 degrees for 60 to 65 minutes until a toothpick comes out clean. Let cool completely in the pan.

Note: non-dairy milk and butter works well in this recipe too.

Glaze

  • 3/4 c sugar
  • 1/4 c orange (or lemon) juice
  • 1/2 tsp vanilla extract
  • 1/2 tsp almond extract

In a sauce pan, bring all ingredients to a boil. Pour over bread in pans.

Cool for 5 minutes, remove from pans, and cool completely.

Changes from the original

My grandmother’s original bread recipe has no lemon or butter in it, somewhat surprisingly, and instead uses 1 1/2 c milk, 1 c vegetable oil, and 1 1/2 tsp butter extract.

The original glaze recipe also calls for an added 1/2 tsp butter extract.

Casey’s 2021 Playlist

Time dilation during the pandemic has been a very real thing for me. 2021 feels like it’s lasted both 4.3 days and 43 thousand years. And somehow this year’s playlist was crafted during that window and yet is only 43 minutes long.

  1. To Live – Norah Jones
  2. Perfect to Me – Anne-Marie
  3. Good as Hell – Lizzo
  4. I’ll Be There – Jess Glynne
  5. Juice – Lizzo
  6. Walk Me Home – Pink
  7. Cups – Anna Kendrick
  8. Farther We Go (A Capella) – Walk off the Earth
  9. Happy Now – Pentatonix
  10. Better Days – Ant Clemons feat. Justin Timberlake
  11. Epiphany – Trent Reznor and Atticus Ross (from the Soul Soundtrack)
  12. Just Us – Trent Reznor and Atticus Ross (from the Soul Soundtrack)
  13. Fate – Rui Fujishiro
  14. Happiness Does Not Wait (2021 Version) – Ólafur Arnalds

Every year’s playlist ends with an instrumental track, but this year features 4 of them. Sometimes when I get really stressed out — which has happened a lot during the pandemic — I’ll go into a dark room, sit on the floor, play those 4 tracks, close my eyes, and just breathe.

You can listen to the songs on Spotify. As always, the order of the songs has been carefully curated. You may not be able to listen to them in order with the Spotify free account.

You can find prior year playlists under the mix cd tag (yes, they’ve been going on that long).