This blog post talks about the general types of hurdles I’ve been encountering on the desktop side of the “upgrade treadmill” when running Linux. It is part two of a three-part blog post série. If you’re into servers, you may be interested in part one, the Linux server landscape’s post-2020 metamorphosis.
Modern software development happens at a breakneck pace, and while staying on ancient versions (hello, Debian Stable / Ubuntu LTS / Android users) is not really a safe and realistic option anymore (try reporting bugs without getting laughed out of the room by upstream maintainers), it is becoming a challenge for users to keep up. When it works, it works… but when something breaks down in the upgrade treadmill, the chain of dependencies to get back on track can become absolutely ludicrous and throw your digital life in turmoil. Just like needing to replace that one light bulb…
Case in point: I’m finally publishing this article in 2022, while I initially meant to blog about this way back in 2017… but more stuff kept breaking all the time, resetting my productivity and accidentally adding more potential content for this blog post. More value for you, dear reader! So grab your popcorn and read on.
As someone who has been running Linux for 19 years (as of 2022), I think I know my way around most hurdles you can possibly encounter. Undoubtedly, running Linux-based operating systems on desktop/laptop computers has overall gotten incredibly easier compared to 2003, but also, as one gradually becomes highly dependent on specific tools and committed to well-oiled workflows, the upgrade treadmill can become a real high-stakes pursuit.
I’ll skim over minor anecdotes like the fact that I once had to “dnf versionlock alsa” for a while because ALSA was misbehaving with my Sound Blaster™ Audigy™ (don’t laugh! That’s how you get analog surround sound when your motherboard doesn’t do 5.1) while testing on a live USB version of a new Fedora release but, as it turns out, it worked without problems after installation… or the fact that I just realized, in 2022, that nautilus-sendto doesn’t exist anymore and since most apps haven’t yet migrated to the Flatpak-compatible replacement “portal”, that means I can’t right-click a file in Nautilus to send it via email now… and I’ll instead dive into the rather more interesting reason why I ran an old insecure web browser—and operating system—for nearly two years.
A tale of foxes
In 2017, Mozilla unleashed Firefox Quantum, quite possibly the most awesome and groundbreaking release they did in that decade. Except there was just one little problem: it also completely changed the extensions system (for good technical reasons).
I happened to be critically dependent, for my day-to-day productivity, on the now incompatible TabGroups extension (which was, itself, a reimplementation of Mozilla’s previously deprecated “Panorama” feature from Firefox 4, previously known as “Tab Candy“), whose author had burned out and given up completely on porting it to the new WebExtensions API.
As a power user, I was utterly foxed.
There was an awfully long period of time where Firefox had no replacement and no migration path for users like me.
Because the modern web (a.k.a. the web obesity crisis) was unbearable, I needed to stay on an Electrolysis-capable-but-pre-Quantum version of Firefox, and that meant specifically Firefox 57, and no other. Not even Firefox ESR. I had to deep-freeze my Firefox version to prevent any upgrade. Thankfully, operating systems like Fedora allow you to do that easily, with DNF’s “versionlock” feature.
This is what “dnf versionlock firefox” feels like (soft warning: this photo may shock animal lovers):
Locking your Firefox version is something that you should never do from a security standpoint (yes I can hear your lamentations, infosec crowd), but I had no other choice and I was too busy in my day-to-day mercenary CMO job to be messing with my personal productivity tooling. So I kept waiting, checking and hoping that someday, someone would write a replacement WebExtension for it.
I therefore spent a painfully long period of time stuck on the same old (and eventually EoL‘ed) version of Fedora Workstation, because upgrading Fedora eventually became impossible due to dependency issues arising from the versionlocked Firefox. While Fedora 27 was released in 2017, I had to remain on that frozen-in-time, unmaintained, insecure version until the end of 2019.
This is what waiting in suspense, on an EoL’ed operating system, looks like:
Yes, I can hear all of you from the InfoSec crowd gasping for air, in shock, after having stared at the screen for a solid 30 seconds wondering how I could have subjected myself to such risk in this day and age. What can I say, I have NERVes of StEELE.
Thankfully, one day in 2019, Simple Tab Groups was recommended to me and, as I tested it extensively to see how solid it actually was, I determined it to be the sole viable, Nekohayo-Proof™, Enterprise-Grade™ replacement for Panorama/TabGroups. It provided a migration path and it was pretty much bullet-proof (especially compared to things like Conex, which I found to be unreliable for my needs). I’ve been happily using it since 2019 and it works better than anything else ever did. With a couple of tweaks in the settings, it has much better performance than its predecessors—especially when you disable thumbnails, use the tabs “discarding” (unloading) feature, and after my suggested search activation technique was implemented—so that’s neat.
A tale of pure hard drive
Eventually, Firefox being a solved problem at last, and Fedora 27 reaching end-of-life status, I really had to migrate, not just for security reasons, but also because my computer’s hard drive partitions were badly space-bound, as I had really outgrown the layout I had made all the way back in… 2010.
Repartitioning my workstation, and upgrading some SSDs in the process, was also blocking my ability to reassign the older/smaller SSDs to my server, which in turn was blocking my ability to migrate from Centos 7 to Centos 8, which was hurting my ability to upgrade PHP sustainably, which was blocking my ability to upgrade Matomo.
If your head hurts while you’re reading this, at this point, it’s not just because of the hard drives. But wait, I’m not done yet!
A tale of serpents
Fedora, as a fast-moving operating system, generally favors leanness and innovation over backwards compatibility (“Friends, Features, First“). This means that if you want to get rid of Python 2 for example, you hunt down and exile any application that still hasn’t ported to Python 3 (they were not alone by the way; OpenSUSE and Sabayon Linux, among others, did the same thing roughly a year later). This works well in an ideal world where application developers have the time and resources to quickly port to new technologies, but the Python 2 to 3 migration has been a notoriously long process in terms of ecosystem. And sometimes, developers have to shave two dozen massive yaks in lockstep in order to achieve that. I know, because I have lived through, as a co-maintainer, the multi-years effort that was porting Pitivi to Python 3, GTK 3, GObject Introspection and GStreamer 1.0 simultaneously.
As a result, when Fedora decided to cut the cord for Python 2 apps in version 31, I had a number off desktop applications simply cease to exist on Fedora. Among them were DisplayCAL (which I had just discovered as the only way to actually make my Colormunki spectrophotometer work for display calibration without requiring a PhD in quantum physics, as colord unfortunately never worked reliably for me), and… Getting Things GNOME.
I could live without DisplayCAL (“we had barely just met!”), but I could not live without GTG, so I had to go begging for a good samaritan to emergency-package it as a Flatpak. Thankfully, Bilal Elmoussaoui answered the call to package the old version. Whew, I had some breathing room and could finally upgrade from the EoL’ed Fedora 27 to latest stable version at the time, Fedora 30, while I figured out a way to put the GTG project back on track.
- I eventually did put the GTG project back onto the rails (with Diego’s admirable bugfixing frenzy, the much-awaited 0.4 version was released in 2020 from development hell). Two years down the road, GTG is now slated to re-enter Fedora (hopefully) 🤞
- DisplayCAL, on the other hand, is still a Python 2 application so… as Larry “Solo Wing Pixy” Foulke said in the introduction scene of The Belkan War, “It’s going to take a while.”
As you can see, even for an experienced Linux user used to troubleshooting cutting-edge technology, the upgrade treadmill can be a problem. I mean, just look at how absolutely ridiculous the “chain of dependencies” has been for me here just to solve the “critical path”:
- Objective: “run the latest Fedora version!”
- Blocked by Firefox version
- Blocked by needing to wait for a viable replacement for Panorama / Tab Groups
- Blocked by GTG
- Needs a temporary Flatpak package to live through the winter
- Needs a new Python 3 + GObject Introspection + GTK3-based release
- Needs a new community of coders to reform around the project
- Needs product management, including proper open-source processes and an opiniated direction
- Needs updating/rewriting the majority of contributors’ documentation
- Needs talking to previous maintainers and the public to ascertain what led to previous failure
- Needs remarketing & open-source tech evangelism, including outreach & community management
- Needs to evaluate the feasibility and available resources
- Need to ascertain the technical status of the previously stalled development efforts (by persistently tracking down and pressing previous maintainers for answers via personal email exchanges) and see if any of the onlookers of this ticket would be interested in embarking on that project
- Needs to survey the market to gauge interest, with a marketing piece that also provides historical context and a pitch on GTG’s unique value proposition, preceded by:
- Pre-market with a theory writeup explaining my worldview with a typology of workers
- Pre-market with a theory writeup explaining the unique value of the GTD methodology for personal productivity
- Needs a clear, actionable, lean, rapidly achievable “MVP” roadmap
- Needs heavy bug triaging
- Needs building & testing code that had been abandoned for 7 years
- Needs claiming administrative ownership of the multiple bug trackers
- Needs heavy bug triaging
- Needs to build a social media presence from scratch
- Needs to rewrite the project’s front page to market it to users (who may become potential contributors)
- Needs to evaluate the feasibility and available resources
- The things in bold above are done as a public service, but they also are what companies would hire me for, if they have enough Japanese Yen. I didn’t spend years training as a mercenary in Restricted Airspace B7R for nothin’!
- Needs product management, including proper open-source processes and an opiniated direction
- Needs a new community of coders to reform around the project
- Blocked by Firefox version
That’s not even counting the repartitioning of all my drives (which was another yak shave that had its own chain of dependencies), nor the years-long epic investigation to solve weird hardware issues some years prior to all this. Goodness gracious, it’s a miracle we get anything done around here.
You may think the story ends here. After all, with those issues resolved, I finally was able to thaw everything and upgrade Fedora… and it was a huge sigh of relief indeed; late 2019 to late 2020 was pretty great in terms of being able to run a normal up-to-date Fedora Workstation experience. But nothing is eternal (except Doom)…
Keep your popcorn nearby. Next, I will write about the Curious Case of Fedora and Unison, which warrants its own blog post. Keep your eyes open for a new article on Monday, January 31st!
You can check out my main website, find me on Twitter or Mastodon or subscribe to my YouTube channel.
Comments
8 responses to “The Software Upgrade Treadmill and Life’s crazy chain of dependencies — an epic tale about Firefox, GTG, Python, and Linux distros”
Sounds like PEBKAC
@YHVH: That is a disappointingly low-effort comment.
Are we… are we old now? (*￣▽￣)b
Funny you would write this a few days after I upgraded Fedora to 35. Apparently something about Python 3.10 didn’t sit well with Anki and it just gave me a blank screen. I gave up after a while of googling, and then remembered there was a flatpak version. Just copied the data over and bam, Anki is running again. This time from the comfort of its own sandbox with Python 3.8. (too bad the flatpak is now abandoned though).
I’m relatively optimistic that something like Silverblue or Nix will make all of this better one day
Shouldn’t you have reported them to your distro’s BTS?
Also, part of the problem is that _you_ ‘depend’ on a lot of software that got unmaintained for one reason or another. I’m not saying I’m better at that, maybe just that I’m luckier, maybe.
And I said ‘depend’ because nobody really forces you to calibrate the color of your screen (unless your day job depends on that?), but you already said that. Maybe you could have tried to keep your ‘legacy’ stuff in containers, but that also takes time to learn (and troubleshoot).
So in the end, yes, sometimes it’s hard to keep your system working through the years. Similar stuff would happen in other platforms, where OS’s drop support for X or Y and your old Z program cannot run anymore, or W’s manufacturer stopped supporting it and you have to throw it away 3y after purchase: https://www.vice.com/en_us/article/dypxpx/google-is-forcing-me-to-dump-a-perfectly-good-phone
Hi Marcos and thanks for commenting 🙂
Not only does this feel counterproductive to collectively divide our efforts by reporting software bugs downstream—each in our own little islands that distros are—but it is also my experience (from the last twenty years or so) that downstream/distro bug trackers are “where good bug reports go to die.” Rarely do things get fixed there, for a multitude of reasons that could amount to a dedicated blog post. Hence I don’t bother with distro bug trackers unless there’s a packaging issue rather than a bug in the software. The developers who maintain said software typically do not monitor/manage those, and it takes me enough hours to properly report and investigate bugs that I don’t want to do it more than once.
As a matter of fact yes, my job does depend on that. Whether doing graphic design as part of my visual branding work, or professional video editing, or professional photography work for clients in Montreal, I need to be able to trust the colors of my computer monitors somewhat. This is the same reason why I wear clear glasses, instead of photochromic lenses (i.e. Essilor Transitions™ or similar).
I totally understand that similar problems are found on other platforms though. Planned obsolescence is horrible, and one reason why I tend to dislike mobile devices. My blog post just tries to illustrate that users can also be exhausted by the FLOSS treadmill in sometimes comical ways.
I don’t know if it’s doable on Fedora, but I find Caja and more particularly caja-sendto works fine on ubuntu 20.04 lts.
Gives you spatial nautilus and sendto which coexist quite happily with gnome3 (or the classic version thereof).
So this is where an AppImage would have helped you. I create an AppImage for anything that needs deprecated dependencies to keep it unlinked from the system package manager. Another option is to create a new wine environment and run Firefox as a wine app.
Flatpak did indeed eventually end up saving my ass in the case of GTG and DisplayCAL, in fact! Although I still prefer to use distros’ native packages when available (and when they are equally up to date). The problem lies in the case where an application is hard to package thusly, or that nobody wants to package it.
There are applications where I’m unsure if it is actually possible to use them in sandboxed environments, such as peer-to-peer applications (like Unison, which is part of what the 3rd installment of this story will be about).