undefined | Better HN

0 pointsmarbu3y ago0 comments

I don't have a negative experience with journald file format. That said, the format is documented here:

On the other hand I agree that journald seems to call open syscall more often than I would originally expect, which can be a problem for some edge cases[1], but I don't consider this to be a real problem.

[1] https://blog.marbu.eu/posts/2022-11-20-ebpf-journald/

0 comments

ilyt3y ago

The problem is:

If cache is hot, you have hundreds of MBs of systemd logs littering your buffer cache, while it could be used for something more useful

If cache is cold simple operations like systemctl status take up to tens of seconds, especially on loaded servers. Example:

    # time systemctl status influxdb >/dev/null

    real 0m11.913s
    user 0m0.013s
    sys 0m0.531s

It doesn't even keep index of "last file where app wrote logs" which causes above.

marbuOP3y ago

But this buffer cache usage is caused by readers/clients checking logs for some reason, not logging itself, right? I see no significant difference compared to syslog here: if I grep all logs for something, it will also place these logs into buffer cache.

Your example with dropping cache and running systemctl status is indeed interesting, 11s looks like too much. But it's a number without a context, and I wonder how big a problem this actually is. I haven't noticed it myself before.

> It doesn't even keep index of "last file where app wrote logs" which causes above.

While I definitely see some room for optimization, I'm not sure what you mean here: journald uses one active journald database file, there should not be a problem with figuring out where the file is, should it?

j / k navigate · click thread line to collapse