Last one I remember was related to the keychain update, you had to re-acknowledge all of the maintainers in the arch-keyring.
It was a lot of work to visit them all and get them to recite their key signatures while I held their hand to make sure they were really there. The blood tests took a while to come back and affirm their identities, but it was worth it for the piece of mind.
I think no rolling releases has gotten their act right. All eventually break with an update, and if you skipped several, the likelihood of reinstalling is high.
I mean, I’ve been running exclusively rolling release Linux distros (Gentoo and Arch) as my daily driver for the last 20 years and I don’t think I’ve ever had the thought that reinstalling from scratch would be simpler than just… fixing the issue. I’ve never backed up my OS files either. Only data files; the sort of files that aren’t put there by the install process or package manager. I’ve never used BTRFS or anything with snapshotting abilities either.
I have booted into a live system, chrooted, and fixed something. (Because my kernel or bootloader or something was borked.) But it’s been quite a long time since I’ve had to do that, and I’ve never reinstalled from scratch because I couldn’t fix something.
(The only times I’ve used non-rolling releases were pretty limited cases. A server here running Debian here, a Raspberry Pi running Raspian (though even Raspberry Pi’s, I’ve mostly used Arch Linux ARM) there. Also my work machine, though I’ve even installed rolling release distros on my work machine on a forgiveness-rather-than-permission basis at times. Lol.)
My record with Arch is longer than 6 months without updates, and it completed the updates fine.
My oldest install is 4 years old, and it’s true, it became unbootable once. But it was my fault, for generating an initramfs without free space in /tmp, which caused it to fail. It was a quick fix once I figured it out.
All of this isn’t to say rolling is as stable as Debian, of course. Rolling is inherently gonna be more unstable, and that’s by design.
But if you take some care, you can actually make these rolling distros pretty stable. Hell, my servers run Arch (I’m working on migrating them to Gentoo to get rid of systemd, but I haven’t yet commited to it). And I haven’t had any serious breakages, even on Arch Linux ARM.
But it was my fault, for generating an initramfs without free space in /tmp, which caused it to fail.
I really don’t think that was your fault. A critical build process that’s not checking for resources and not failing gracefully and leaving the machine unbootable is not the user’s fault.
A build+install can fail for many reasons. It’s actually annoying how many critical system updates on Linux don’t use a staging process and a sudden power-off at the wrong time can leave the machine completely crippled.
Some distros have taken to doing a BTRFS snapshot before they start the upgrade but (a) they’re far and few in between and (b) that still doesn’t fix the upgrade process, it’s just sweeping the problem under the carpet.
I manually triggered it, and didn’t read the logs when it said it failed. That was my fault. If it fails, it will tell you. You still have the machine working until you reboot, so if you read the logs, you can clear /tmp and retry again, and the machine should be bootable again.
Linux lets you do anything, including stupid things. I’d say doing stupid things is the fastest way to learn about it, though.
The same issue would have happened even on Ubuntu, as it uses the same tools as Arch for generating the initramfs. I don’t consider it an Arch or Linux issue, just a user mistake. It won’t happen normally, unless you manually trigger it.
I’m starting to sound like a broken record but since nobody else has mentioned it yet: atomic/immutable distros often use something called rpm-ostree, which stages updates for the next boot. Updates are “atomic”, so you can’t end up with a corrupt partial update, even if you pull the power mid-update (in which case it will just stay on the current version). Everybody gets the same base image, so your OS won’t diverge over time and end up like some frankenstein mess that. It’s all super cool tech.
Cuz it’s annoying to work with (it usually breaks stuff because it’s purposefully incompatible, and they have to force their way), it’s slower than alternatives, it is a mess, and I don’t like its philosophy.
SystemD is the Windows of Linux, and it has already brought enough problems. Sadly, all the major distros went all in on it, so a lot of software is developed with increasing dependency on SystemD’s non standard systems.
Also I don’t trust Poettering at all. This guy is antithetical to free, open and standard software, and he himself has said it.
And on top of all that, they went and folded so quickly to implement stuff like age verification when Linux wasn’t even tied by those laws. Their excuse was that they were doing it to set a good precedent and a safe standard for it on Linux, but their implementation was unsafe and flawed. When pointed out, Poettering himself said that why bother with security, because that data is “not important”.
TL;DR: I have many concerns with SystemD’s management and philosophy, and their software often breaks, causes issues, isn’t compliant with standards, has security holes and is slow.
Wouldn’t say that. OpenSuse Tumbleweed might got some tech debt with their backend and doesn’t come with fancy stuff such as Kernel modifications like Cachy, but their rolling distro is rock solid. And for the few cases where something really breaks the bootable btrfs Snapshots come by default.
It still is for Intermediate users for sure (the installer expects you to know what you want - yet is a GUI). But in terms of stability they really nailed it, I’d absolutely use it for work horses that require newest software. Or Leap if I require binary compatibility to SLED/SLES.
What specifically happened? Can you share the news item?
Last one I remember was related to the keychain update, you had to re-acknowledge all of the maintainers in the arch-keyring.
It was a lot of work to visit them all and get them to recite their key signatures while I held their hand to make sure they were really there. The blood tests took a while to come back and affirm their identities, but it was worth it for the piece of mind.
I think no rolling releases has gotten their act right. All eventually break with an update, and if you skipped several, the likelihood of reinstalling is high.
At least they teach you good backup practices.
I mean, I’ve been running exclusively rolling release Linux distros (Gentoo and Arch) as my daily driver for the last 20 years and I don’t think I’ve ever had the thought that reinstalling from scratch would be simpler than just… fixing the issue. I’ve never backed up my OS files either. Only data files; the sort of files that aren’t put there by the install process or package manager. I’ve never used BTRFS or anything with snapshotting abilities either.
I have booted into a live system, chrooted, and fixed something. (Because my kernel or bootloader or something was borked.) But it’s been quite a long time since I’ve had to do that, and I’ve never reinstalled from scratch because I couldn’t fix something.
(The only times I’ve used non-rolling releases were pretty limited cases. A server here running Debian here, a Raspberry Pi running Raspian (though even Raspberry Pi’s, I’ve mostly used Arch Linux ARM) there. Also my work machine, though I’ve even installed rolling release distros on my work machine on a forgiveness-rather-than-permission basis at times. Lol.)
At least with Arch, I don’t imagine needing to reinstall, worst case I should be able to reinstall all packages from a live image.
My record with Arch is longer than 6 months without updates, and it completed the updates fine.
My oldest install is 4 years old, and it’s true, it became unbootable once. But it was my fault, for generating an initramfs without free space in /tmp, which caused it to fail. It was a quick fix once I figured it out.
All of this isn’t to say rolling is as stable as Debian, of course. Rolling is inherently gonna be more unstable, and that’s by design.
But if you take some care, you can actually make these rolling distros pretty stable. Hell, my servers run Arch (I’m working on migrating them to Gentoo to get rid of systemd, but I haven’t yet commited to it). And I haven’t had any serious breakages, even on Arch Linux ARM.
I really don’t think that was your fault. A critical build process that’s not checking for resources and not failing gracefully and leaving the machine unbootable is not the user’s fault.
A build+install can fail for many reasons. It’s actually annoying how many critical system updates on Linux don’t use a staging process and a sudden power-off at the wrong time can leave the machine completely crippled.
Some distros have taken to doing a BTRFS snapshot before they start the upgrade but (a) they’re far and few in between and (b) that still doesn’t fix the upgrade process, it’s just sweeping the problem under the carpet.
I manually triggered it, and didn’t read the logs when it said it failed. That was my fault. If it fails, it will tell you. You still have the machine working until you reboot, so if you read the logs, you can clear /tmp and retry again, and the machine should be bootable again.
Linux lets you do anything, including stupid things. I’d say doing stupid things is the fastest way to learn about it, though.
The same issue would have happened even on Ubuntu, as it uses the same tools as Arch for generating the initramfs. I don’t consider it an Arch or Linux issue, just a user mistake. It won’t happen normally, unless you manually trigger it.
I’m starting to sound like a broken record but since nobody else has mentioned it yet: atomic/immutable distros often use something called rpm-ostree, which stages updates for the next boot. Updates are “atomic”, so you can’t end up with a corrupt partial update, even if you pull the power mid-update (in which case it will just stay on the current version). Everybody gets the same base image, so your OS won’t diverge over time and end up like some frankenstein mess that. It’s all super cool tech.
Why are you trying to get rid of systemd on your servers?
Cuz it’s annoying to work with (it usually breaks stuff because it’s purposefully incompatible, and they have to force their way), it’s slower than alternatives, it is a mess, and I don’t like its philosophy.
SystemD is the Windows of Linux, and it has already brought enough problems. Sadly, all the major distros went all in on it, so a lot of software is developed with increasing dependency on SystemD’s non standard systems.
Also I don’t trust Poettering at all. This guy is antithetical to free, open and standard software, and he himself has said it.
And on top of all that, they went and folded so quickly to implement stuff like age verification when Linux wasn’t even tied by those laws. Their excuse was that they were doing it to set a good precedent and a safe standard for it on Linux, but their implementation was unsafe and flawed. When pointed out, Poettering himself said that why bother with security, because that data is “not important”.
TL;DR: I have many concerns with SystemD’s management and philosophy, and their software often breaks, causes issues, isn’t compliant with standards, has security holes and is slow.
Why wouldn’t you? OpenRC FTW.
Wouldn’t say that. OpenSuse Tumbleweed might got some tech debt with their backend and doesn’t come with fancy stuff such as Kernel modifications like Cachy, but their rolling distro is rock solid. And for the few cases where something really breaks the bootable btrfs Snapshots come by default.
It still is for Intermediate users for sure (the installer expects you to know what you want - yet is a GUI). But in terms of stability they really nailed it, I’d absolutely use it for work horses that require newest software. Or Leap if I require binary compatibility to SLED/SLES.
Who needs research when you have backups?
My system creates automatic snapshots and that’s usually enough. Restoring a snapshots takes me literally 1 minute.
This should be built-in on every modern distro.
I don’t mean it happened right now. I mean it often happens. Probably when i tweak too much or more installs from AUR.
AUR is not the official distro. Complaining about arch because of the AUR is like complaining about Debian because you npm install some malware
Manual intervention announcements happen far from “often” though, tinkering with your system is a different thing