The joy of Open Source®™© software.
On Linux boxes, I've been taking full advantage of the logical volume manager drivers that come with recent versions of the kernel so I can do backups that remain logically consistant without having to take down the entire system for the duration of the backups (on FreeBSD and old versions of Linux, I have to use rsync to a different disk or system, but that takes long enough so that the contents of the offending filesystems can change during the rsync.) The lvm code was nice and stable, and worked across several versions of the system without fuss, muss, or bother, so I could just leave the code running and not worry about it.
At work, we've been using R*dh*t Linux (in particular, the hideously expensive RHE Linux), so we occasionally have to roll kernels when our hardware vendor comes out with a new something-or-another that they'll only support on the latest RH kernel of the week. A couple of months ago, we rolled up to the RHEL3 update 6 kernel (which has some features, like a malfunctioning USB stack, that I fixed by myself while fighting with what I only call "technical support" because the name I want to use is unprintable, even for me) and, during the excitement of the upgrade, managed to turn off the snapshotting.
On tuesday, we turned it back on, and the backup immediately failed. In the mount of the snapshotted filesystem. Not because the mount failed, but because it was in an UNINTERRUPTABLE DISK WAIT (complete with the obligatory kernel panic, which didn't manage to kill the machine, but which did manage to lock the superblock on the offending filesystem so that I couldn't sync and reboot.) Hmm, okay, perhaps it's time to roll up to a 2.6 kernel (because those are supported too, and since they're more recent we don't have to do any driver backports.) *Well*, apparently one of the features that Linux 2.6 has is that the LVM code has been completely rewritten (by R*dh*t, which bought Sistina [the prime maintainers of the LVM code for many years] a few years ago) and some of the features in it don't actually work any more. Like, um, snapshots.
WTF? WTF-FF? The stupid LVM code worked for us for TWO YEARS before we were encouraged to upgrade to the EL.37 (codename: TestlessinNC) kernel, and now it looks like the maintainers have promiscuously rewritten it so that it DOESN'T EVEN WORK anymore, so our only upgrade path is to abandon the whole damned thing altogether.
G-d, I love open source code. "OH NO, it looks like people are using our code to do production work!" "Well, we'd better rewrite it then. We can break all the interfaces and stop supporting half the features, and that will show them."
<wham!> <wham!> <wham!> <wham!> (pounds head against table)
Comments
Comments are closed
Open Source has no lock on: “…We can break all the interfaces and stop supporting half the features, and that will show them.”