undefined | Better HN

0 pointspkal8mo ago0 comments

That is exactly what I do, and it works well enough because if the power-loss were to happen, I wouldn't have lost anything of crucial value. But that is admittedly a very instance-specific advantage I have.

0 comments

kragen8mo ago

There's a fsync/close/rename dance that ext4fs recognizes as a safe, durable atomic file replacement, which is often sufficient for preventing data loss in cases like this.

masklinn8mo ago

FWIW POSIX requires that rename(2) be atomic, it's not just ext4, any POSIX FS should work that way.

However this still requires a lockfile because while rename(2) is an atomic store it's not a full CAS, so you can have two processes reading the file concurrently, doing their internal update, writing to a temp file, then rename-ing to the target. There will be no torn version of the reference file, but the process finishing last will cancel out the changes of the other one.

The lockfile can be the "scratch" file as open(O_CREAT | O_EXCL) is also guaranteed to be atomic, however now you need a way to wait for that path to disappear before retrying.

kragen8mo ago

I forget the particular failure mode that POSIX arguably permitted in this case. I think it had to do with fsyncing the directory? And the file ending up existing but empty after the reboot? Anyway, it broke userspace, so the ext4fs maintainers killed it. I think there were some LWN articles.

I might have been misremembering: https://lwn.net/Articles/322823/

"With ext4's delayed allocation, the metadata changes can be journalled without writing out the blocks. So in case of a crash, the metadata changes (that were journalled) get replayed, but the data changes don't."

That was without fsync, though. But apparently it's a topic of discussion more recently: https://lwn.net/Articles/789600/

I agree that it still requires a lockfile if write conflicts are not acceptable.

j / k navigate · click thread line to collapse

0 comments

kragen8mo ago

There's a fsync/close/rename dance that ext4fs recognizes as a safe, durable atomic file replacement, which is often sufficient for preventing data loss in cases like this.

masklinn8mo ago

FWIW POSIX requires that rename(2) be atomic, it's not just ext4, any POSIX FS should work that way.

The lockfile can be the "scratch" file as open(O_CREAT | O_EXCL) is also guaranteed to be atomic, however now you need a way to wait for that path to disappear before retrying.

kragen8mo ago

I might have been misremembering: https://lwn.net/Articles/322823/

That was without fsync, though. But apparently it's a topic of discussion more recently: https://lwn.net/Articles/789600/

I agree that it still requires a lockfile if write conflicts are not acceptable.

j / k navigate · click thread line to collapse