Yeah, that’s probably a fair answer for some folks and systems. Depends on what your system is doing, your risk tolerance, and what money you’re willing to put into the thing.
If I didn’t have a backup system in place, say, that’d be much higher on my list than trying to ensure power isn’t cut off unexpectedly during power outages.
I agree that it’s less-critical than it was at one point. Any modern filesystem, including ext4 and btrfs, isn’t at risk of filesystem-level corruption, and a DBMS like PostgreSQL or MySQL should handle it at an application level. That being said, there is still other software out there that may take issue with being interrupted. Doing an apt upgrade is not guaranteed to handle power loss cleanly, for example. And I’m not too sanguine about hardware not being bricked if I lose power during an fwupd updating the firmware on attached hardware. Maybe a given piece of hardware has a safe, atomic upgrade procedure…and maybe it doesn’t.
That does also mean, if there’s no power backup at all, that one won’t have the system available for the duration of the outage. That may be no big deal, or might be a real pain.
Yeah, I listed it as one possibility, maybe the best I can think of, but also why I’ve got some issues with that route, why it wouldn’t be my preferred route. Maybe it is the best generally available right now.
The “just use a UPS plus a second system” route makes a lot of sense with diesel generator systems, because there the hardware physically cannot come up to speed in time. A generator cannot start in 10ms, so you need a flywheel or battery or some other kind of energy-storage system in place to bridge the gap…but that shouldn’t be a fundamental constraint on those home large-battery backup systems. They don’t have to be equipped with an inverter able to come online in 10ms…but they could. In the generator scenario, it’s simply not an option.
I’d like to, if possible, have the computer have a “unified” view of all of the backing storage systems. In the generator case, the “time remaining” is a function of the fuel in the tank, and I’m pretty sure that it’s not uncommon for someone to be able to have some kind of secondary storage that couldn’t be measured; I remember reading about a New Orleans employee in Hurricane Katrina that stayed behind to keep the datacenter functioning mostly hauling drums of diesel up the stairs to the generator. But that’s not really a fundamental issue with those battery backup systems, not unless someone is planning on hauling more batteries in.
If one gets a UPS and then backs it with a battery backup system, then there are two sets of batteries — one often lead-acid, with a shorter lifespan — and multiple inverters and battery charge controllers in multiple layers in the system. That’s not the end of the world, a “throw some extra money at it” issue, but one is having to get redundant hardware.
I’ll add one other point that might affect people running low-power servers, which I believe some people here are running for low-compute-load stuff like home automation: my past experience is that low-end, low power computers often have (inexpensive) power supplies that are especially intolerant of wall power issues. I have had multiple consumer broadband routers and switches that have gotten into a wonky, manual-reboot-requiring state after brownouts or power loss, even when other computers in the house continued to function without issue. I’d guess that those might be particularly-sensitive to a longer delay in changing over to a backup power source. I would guess that Raspberry Pi-class machines might have power supplies vulnerable to this. I suppose that for devices with standard barrel connectors and voltage levels, one could probably find a more-expensive power supply that can handle dirtier power.
If you run some form of backup power system that powers them, have you had issues with Raspberry Pis or consumer internet routers after power outages?
While laptop batteries may not have aged well, especially if they’re left discharged, one other nice perk is that laptops effectively have an integrated UPS.
I use gdb myself.
I don’t know exactly what you’re after. From the above, I see:
“easy to use”
" the mouse is faster, not slower"
You don’t specify a language, so I’m assuming you’re looking for something low-level.
You don’t specify an editor, so I’m assuming that you want something stand-alone, not integrated with an editor.
There are a number of packages that use gdb internally, but put some kind of visualization on it. I’ve used emacs’s before, though I’m not particularly married to it — mainly found it interesting as a way to rapidly move up and down frames in a stack — but I’m assuming that if you want something quick to learn, you’re not looking for emacs either.
Maybe seer? That’d be a stand-alone frontend on gdb with a GUI. Haven’t used it myself.
EDIT: WRT gdb, the major alternative that I can think of to gdb is dbx, and that’s also a CLI tool and looks dead these days. gdb is pretty dominant, so if you want something mouse-oriented, you’re probably going to have some form of frontend on gdb.
There are other important debugging tools out there, stuff like valgrind, but in terms of a tool to halt and step through a program, view variables, etc, you’re most-likely looking at gdb, one way or another, unless you’re working in some sort of high-level language that has its own debugger. If you want a GUI interface, it’s probably going to be some sort of frontend to gdb.
EDIT2: Huh. Apparently llvm has its own debugger, lldb. Haven’t used it, and it’s probably not what you want anyway, since it’s also a CLI-based debugger. I am also sure that it has far fewer users than gdb. But just for completeness…guess you already looked at that, mentioned it in your comment.
Reserved for future use: krex & krox
Poor “kryx”.
Three-letter words that can be typed with one hand, since I have to type them frequently.
$ egrep "^([qwertasdfgzxcvb]{3}|[yuiophjkllnm]{3})$" /usr/share/dict/words
Lemmy instances can have a maximum size-in-bytes limit placed that the pict-rs instance will accept. Your particular image may exceed the limit on the instance that you’re trying to post to.
If the image has a unique name or unique-combination of images sent to any one user and is hosted on a server that the sender controls — haven’t checked — it could be an attempt to deanonymize Threadiverse users by getting their IP addresses.
For example, say tal@lemmy.today gets https://imagehost.com/girl34720.jpg and https://imagehost.com/girl83710.jpg.
The server can see the IP address of the client that loaded the images. The attacker knows that only tal@lemmy.today received that combination. Now the attacker knows tal@lemmy.today’s IP address.
Nah, I wrote that when I thought that you just wanted content-based addressing, not that you specifically objected to hash trees being used for the addressing.
BitTorrent and Hyphanet have mechanisms that do this.
Magnet URIs are a standard way of encoding this.
EDIT: You typically want a slightly-more-elaborate approach than just handing the network a hash and then getting a file.
You typically want to be able to “chunk” a large file, so that you can pull it from multiple sources. The problem is that you can only validate that information is correct once you have the whole file. So, say you “chunk” the file, get part of it from one source and part from another. A malicious source could feed you incorrect data. You can validate that the end file does not hash to the right value, but then you have no idea what part of the file that some source fed you is invalid, so you don’t know who to re-fetch data from.
What’s more-common is a system where you have the hash of a hash tree of a file. That way, you can take the hash, request the hash tree from the network, validate that the hash tree hashes to the hash, and then start requesting chunks of the file, where a leaf node in the hash tree is the hash of a chunk. That way, you can validate data at a chunk level, and know that a chunk is invalid after requesting no more than one chunk from a given source.
See Merkle tree, which also mentions Tiger Tree Hash; TTH is typically used as a key in magnet URIs.
EDIT2:
Can’t think of a way to do it with a DHT
All of the DHTs that I can think of exist to implement this sort of thing.
EDIT3: Oh, skimmed over your concern, didn’t notice that you took issue with using a hash tree. I think that one normally does want a hash tree, that it’s a mistake to use a straight hash. I mean, you can generate the hash of a hash tree as easily as the hash of a file, if you have that file, which it sounds like you do. On Linux, rhash(1) can generate hashes of hash trees. So if you already have the file, that’s probably what you want.
Hypothetically, I guess you could go build some kind of index mapping hashes to hashes of hash trees. Don’t know whether you can pull the hash off BitTorrent or something, but I wouldn’t be surprised if it is. But…you’re probably better off with hash trees, unless you can’t see the file and already are committed to a straight hash of the file.
EDIT4:
I mean:
$ rhash --sha1 --hex pkgs
7d3a772009aacfe465cb44be414aaa6604ca1ef0 pkgs
$ rhash -T --hex pkgs
18cab20ffdc55614ed45c5620d85b0230951432cdae2303a pkgs
$
Either way, straight hash or hash of a hash tree, you’re getting a hex string that identifies your file uniquely. Just that in the hash tree case, you solve some significant problems related to the other thing that you want to do, fetch your file. Might be more compute-intensive to generate a hash of a hash tree, but unless you’re really compute-constrained…shrugs
Jia Tan was the username used by a group — probably a state intelligence agency — on GitHub to try to attack the xz open source package. The effort aimed at trying to take over the project, and lasted for years. They managed to get a compromised package briefly into the unstable versions of some major Linux distros that created a backdoor in the openssh daemon and came close to being widely deployed across Linux servers, which would have been a very severe compromise of a huge range of systems. The account vanished when the compromise was discovered.
The user here is, as a joke, using the same name.
EDIT:
note: i did NOT create the ASCII art. I’m not good enough to do that. I found on various ascii art archives as well as those image to ascii art converters for the continents I couldn’t find.
FYI:
$ telnet mapscii.me
Zoom is “a” and “z”, arrow keys pan, “q” quits.
e.g.:
Can produce a map of wherever at whatever zoom.
I’m not entirely enthusiastic about the fact that the maintaining GitHub account is using an avatar that appears to be a flaming Pepe.
and uses btrfs send/receive to create backups.
I’m not familiar with that, but if it permits for faster identification of modified data since a given time than scanning a filesystem for modified files, which a filesystem could potentially do, that could also be a useful backup enabler, since now your scan-for-changes time doesn’t need to be linear in the number of files in the filesystem. If you don’t do that, your next best bet on Linux – and this way would be filesystem-agnostic – is gonna require something like having a daemon that runs and uses inotify to build some kind of on-disk index of modifications since the last backup, and a backup system that can understand that.
looks at btrfs-send(1) man page
Ah, yeah, it does do that. Well, the man page doesn’t say what time it runs in, but I assume that it’s better than linear in file count on the filesystem.
You’re correct and probably the person you’re responding to is treating one as an alternative as another.
However, theoretically filesystem snapshotting can be used to enable backups, because they permit for an instantaneous, consistent view of a filesystem. I don’t know if there are backup systems that do this with btrfs today, but this would involve taking a snapshot and then having the backup system backing up the snapshot rather than the live view of the filesystem.
Otherwise, stuff like drive images and database files that are being written to while being backed up can just have a corrupted, inconsistent file in the backup.
Wouldnt the sync option also confirm that every write also arrived on the disk?
If you’re mounting with the NFS sync option, that’ll avoid the “wait until close and probably reorder writes at the NFS layer” issue I mentioned, so that’d address one of the two issues, and the one that’s specific to NFS.
That’ll force each write to go, in order, to the NFS server, which I’d expect would avoid problems with the network connection being lost while flushing deferred writes. I don’t think that it actually forces it to nonvolatile storage on the server at that time, so if the server loses power, that could still be an issue, but that’s the same problem one would get when running with a local filesystem image with the “less-safe” options for qemu and the client machine loses power.
NFS doesn’t do snapshotting, which is what I assumed that you meant and I’d guess ShortN0te also assumed.
If you’re talking about qcow2 snapshots, that happens at the qcow2 level. NFS doesn’t have any idea that qemu is doing a snapshot operation.
On a related note: if you are invoking a VM using a filesystem images stored on an NFS mount, I would be careful, unless you are absolutely certain that this is safe for the version of NFS and the specific caching options for both NFS and qemu that you are using.
I’ve tried to take a quick look. There’s a large stack involved, and I’m only looking at it quickly.
To avoid data loss via power loss, filesystems – and thus the filesystem images backing VMs using filesystems – require write ordering to be maintained. That is, they need to have the ability to do a write and have it go to actual, nonvolatile storage prior to any subsequent writes.
At a hard disk protocol level, like for SCSI, there are BARRIER operations. These don’t force something to disk immediately, but they do guarantee that all writes prior to the BARRIER are on nonvolatile storage prior to writes subsequent to it.
I don’t believe that Linux has any userspace way for an process to request a write barrier. There is not an fwritebarrier() call. This means that the only way to impose write ordering is to call fsync()/sync() or use similar-such operations. These force data to nonvolatile storage, and do not return until it is there. The downside is that this is slow. Programs that are frequently doing such synchronizations cannot issue writes very quickly, and are very sensitive to latency to their nonvolatile storage.
From the qemu(1) man page:
By default, the cache.writeback=on mode is used. It will report data writes as completed as soon as the data is present in the host page cache. This is safe as long as your guest OS makes sure to correctly flush disk caches where needed. If your guest OS does not handle volatile disk write caches correctly and your host crashes or loses power, then the guest may experience data corruption. For such guests, you should consider using cache.writeback=off. This means that the host page cache will be used to read and write data, but write notification will be sent to the guest only after QEMU has made sure to flush each write to the disk. Be aware that this has a major impact on performance.
I’m fairly sure that this is a rather larger red flag than it might appear, if one simply assumes that Linux must be doing things “correctly”.
Linux doesn’t guarantee that a write to position A goes to disk prior to a write to position B. That means that if your machine crashes or loses power, with the default settings, even for drive images sorted on a filesystem on a local host, with default you can potentially corrupt a filesystem image.
https://docs.kernel.org/block/blk-mq.html
Note
Neither the block layer nor the device protocols guarantee the order of completion of requests. This must be handled by higher layers, like the filesystem.
POSIX does not guarantee that write() operations to different locations in a file are ordered.
https://stackoverflow.com/questions/7463925/guarantees-of-order-of-the-operations-on-file
So by default – which is what you might be doing, wittingly or unwittingly – if you’re using a disk image on a filesystem, qemu simply doesn’t care about write ordering to nonvolatile storage. It does writes. it does not care about the order in which they hit the disk. It is not calling fsync() or using analogous functionality (like O_DIRECT).
NFS entering the picture complicates this further.
https://www.man7.org/linux/man-pages/man5/nfs.5.html
The sync mount option The NFS client treats the sync mount option differently than some other file systems (refer to mount(8) for a description of the generic sync and async mount options). If neither sync nor async is specified (or if the async option is specified), the NFS client delays sending application writes to the server until any of these events occur:
Memory pressure forces reclamation of system memory resources. An application flushes file data explicitly with sync(2), msync(2), or fsync(3). An application closes a file with close(2). The file is locked/unlocked via fcntl(2). In other words, under normal circumstances, data written by an application may not immediately appear on the server that hosts the file. If the sync option is specified on a mount point, any system call that writes data to files on that mount point causes that data to be flushed to the server before the system call returns control to user space. This provides greater data cache coherence among clients, but at a significant performance cost. Applications can use the O_SYNC open flag to force application writes to individual files to go to the server immediately without the use of the sync mount option.
So, strictly-speaking, this doesn’t make any guarantees about what NFS does. It says that it’s fine for the NFS client to send nothing to the server at all on write(). The only time a write() to a file makes it to the server, if you’re using the default NFS mount options. If it’s not going to the server, it definitely cannot be flushed to nonvolatile storage.
Now, I don’t know this for a fact – would have to go digging around in the NFS client you’re using. But it would be compatible with the guarantees listed, and I’d guess that probably, the NFS client isn’t keeping a log of all the write()s and then replaying them in order. If it did so, for it to meaningfully affect what’s on nonvolatile storage, the NFS server would have to fsync() the file after each write being flushed to nonvolatile storage. Instead, it’s probably just keeping a list of dirty data in the file, and then flushing it to the NFS server at close().
That is, say you have a program that opens a file filled with all ‘0’ characters, and does:
At close() time, the NFS client probably doesn’t flush “1” to position 1, then “1” to position 5000, then “2” to position 1, then “2” to position 5000. It’s probably just flushing “2” to position 1, and then “2” to position 5000, because when you close the file, that’s what’s in the list of dirty data in the file.
The thing is that unless the NFS client retains a log of all those write operations, there’s no way to send the writes to the server in a way that avoid putting the file into a corrupt state if power is lost. It doesn’t matter whether it writes the “2” at position 1 or the “2” at position 5000. In either case, it’s creating a situation where, for a moment, one of those two positions has a “0”, and the other has a “2”. If there’s a failure at that point – the server loses power, the network connection is severed – that’s the state in which the file winds up in. That’s a state that is inconsistent, should never have arisen. And if the file is a filesystem image, then the filesystem might be corrupt.
So I’d guess that at both of those two points in the stack – the NFS client writing data to the server, and the server block device scheduler, permit inconsistent state if there’s no fsync()/sync()/etc being issued, which appears to be the default behavior for qemu. And running on NFS probably creates a larger window for a failure to induce corruption.
It’s possible that using qemu’s iSCSI backend avoids this issue, assuming that the iSCSI target avoids reordering. That’d avoid qemu going through the NFS layer.
I’m not going to dig further into this at the moment. I might be incorrect. But I felt that I should at least mention it, since filesystem images on NFS sounded a bit worrying.
Ooh, I didn’t know that someone had developed a mechanism to move issues and PRs.
I remember commenting on the fact that while it’s easy to move the source repo itself from location to location, as git makes that easy and self-contained, issues and PRs didn’t enjoy that.