The systemd notification protocol could have been as simple as just writing a newline to a pipe, but instead you have to link to the libsystemd C library, so now security-critical daemons like openssh have additional dependencies like liblzma loaded into their address space (even if you don't use systemd as PID 1), increasing the risks of supply chain attacks. Thanks, systemd.
> These functions send a single datagram with the state string as payload to the socket referenced in the $NOTIFY_SOCKET environment variable.
The simplest implementation (pseudocode, no error handling, not guaranteed to compile), is something like:
const char *addrstr = getenv("NOTIFY_SOCKET");
if (addrstr) {
int fd = socket(AF_UNIX, SOCK_DGRAM, 0);
struct sockaddr_un addr = { .sun_family = AF_UNIX };
strncpy(addr.sun_path, sizeof(addr.sun_path), addrstr);
connect(fd, (struct sockaddr*) &addr);
write(fd, "READY=1");
close(fd);
}It basically is. libsystemd links to liblzma for other features not related to notifications.
(The protocol is that systemd passes the path to a unix socket in the `NOTIFY_SOCKET` env variable, and the daemon writes "READY=1" into it.)
It doesn't matter that libsystemd links to liblzma for other reasons. It's still in the address space of any daemon that is using libsystemd for the notification protocol.
For Slurm, I looked at what a PITA pulling libsystemd into our autoconf tooling would be, stumbled on the Golang implementation, and realized it's trivial to implement directly.
Which is pretty emblematic of systemd's primary architectural fault!
If that split had never happened, then liblzma wouldn't have ended up being linked into sshd...
[EDIT: this string gives cleaner results:]
lsof -w -P -T -p $(pgrep sshd)|grep mem
Anyway. I did not see lzma in the results on Devuan running a process check (just in case). I did see it on a Debian.
Systemd itself seems security-critical to me. Would removing other dependencies on libsystemd really make a secure system where systemd was compromised through its library?
2. You can run Debian systems without systemd as PID 1, but you're still stuck with libsystemd because so many daemons now link with it.
Socket activation and the NFS automounter appear to.
If I run "netstat -ap" I see pid 1 listening on enabled units.
Edit: tinysshd is specifically launched this way.
Edit2: there is also substantial criticism of xz on technical grounds.
Yes, there are things delivered with that complexity. However, as an example, sysvinit is maybe, oh, 20k lines of code including binaries, heck including all core init scripts.
What's systemd? 2M lines? It was >1M lines 4+ years ago.
For an init system, a thing that is to be the core of stability, security, and most importantly glacial, stable change -- that is absurdly complex. It's exceedingly over engineered.
And so you get cases like this. And cases like that, and that over there, and that case over there too. All which could not exist, if systemd didn't try to overengineer, over complicate everything.
Ah well. I'm still waiting for someone to basically fork systemd, remove all the fluff (udev, ntp, dns, timers, restart code, specialized logging, on and on and on), and just end up with systemd compatible service files.
But not yet. So... well, oh well.
systemd is a collection of tools, one of which is an init system. Nobody accused GNU yes of being bloated just because it's in a repository alongside 50 other tools.
But that's the very problem with systemd! As time goes on you're required, whether by systemd itself or by the ecosystem around it, to use more and more of it, until it's doing not only service management but also timezones, RTC, DNS resolution, providing getpwent/getgrent, inetd, VMs and containers, bootloader, udev (without adding literally any benefit over the existing implementations), ... oh and you also have to add significant complexity in other things (like the kernel!) to use it, like namespaces (which have been a frequent source of vulnerabilities)...
[1] https://github.com/coreutils/coreutils/blob/master/src/yes.c
Most of the things you named there are modular and can be easily disabled.
Furthermore, udev precedes systemd and systemd has in fact its own replacement for it (though the name escapes me).
Kind of a classic, people loving harping on systemd without properly understanding it.
I understand that people don't like the way it seems to work itself into the rest of Linux user space as a dependency but that's actually our own fault for not investing the man power that Red Hat invests. We have better things to do than make our own Linux user space and so they have occupied that niche. It's free software though, we always have the freedom to do whatever we want.
By the way, all the stuff you mentioned is not really part of the actual init system, namely PID 1. There's an actual service manager for example and it's entirely separate from init. It manages services really well too, it's measurably better than all that "portable" nonsense just by virtue of using cgroups to manage processes which means it can actually supervise poorly written double forking daemons.
Except it literally is. I once had a systemd system suddenly refuse to boot (kernel panic because PID1 crashed or so) after a Debian upgrade, which I was able to resolve by... wait for it... making /etc/localtime not be a symlink.
Why does a failure doing something with the timezone make you unable to boot your system? What is it even doing with the timezone? What is failing about it? Who knows, good luck strace'ing PID1!
Also, the more extensive the remit (of this init), the more complexly interconnected the interactions between the components; the fewer people understand the architecture, the fewer people understand the code, the fewer people read the code. This creates a situation where the codebase is getting larger and larger at a rate faster than the growth of the number of man-hours being put into reading it.
This has to make it easier for people who are systemd specialists to put in (intentionally or unintentionally) backdoors and exploitable bugs that will last for years.
People keep defending systemd by talking about its UI and its features, but that completely misses the point. If systemd were replaced by something comprehensible and less internally codependent, even if the systemd UI and features were preserved, most systemd complainers would be over the moon with happiness. Red Hat invests too much into completely replacing linux subsystems, they should take a break. Maybe fix the bugs in MATE.
Complexity that would otherwise be distributed to a sea of ad-hoc shell scripts? systemd is a win
this is and always has been such a dumb take.
if you'd like to implement an init (and friends) system that doesn't have "unnecessary complexity" and still provides all the functionality that people currently want, then go and do so and show us? otherwise it's just whinging about things not being like the terrible old days of init being a mass of buggy and racey shell scripts.
Zero of the major distros used System V init by default. Probably only distros like Slackware or Linux From Scratch even suggested it.
It's unfortunate that so many folks uncritically swallowed the Systemd Cabal's claims about how they were the first to do this, that, or the other.
(It's also darkly amusing to note that every service that has nontrivial pre-start or post-start configuration and/or verification requirements ends up using systemd to run at least one shell script... which is what would have often been inlined into their init script in other init systems.)
The problem? It's on the backburner because I don't think I could find a business model to make money from it.
I don't think offering support for a price would work, for example.
If you want you can just use systemd as PID1 for service management and enjoy a sane way to define and manage services – and do everything in archaic ways like 20 years ago.
Note that OP found this in Debian sid as well, which means it's highly unlikely this issue will find its way into any Debian stable systems.
My Arch system was not vulnerable because openssh was not linked to xz.
IMO every single commit from JiaT75 should be reviewed and maybe even rolled back, as they have obliterated their trust.
edit:
https://github.com/google/oss-fuzz/pull/10667
Even this might be nefarious.
But let me stress two other things:
Libselinux pulls in liblzma too and gets linked into tons more programs than libsystemd. And will end up in sshd too (at the very least via libpam/pam_selinux). And most of the really big distros tend do support selinux at least to some level. Hence systemd or not, sshd remains vulnerable by this specific attack.
With that in mind libsystemd git dropped the dep on liblzma actually, all compressors are now dlopen deps and thus only pulled in when needed.
Could you point out where the man page (https://www.freedesktop.org/software/systemd/man/latest/sd_n...) says this?
(I really wish there were a way to link such that the library isn't actually loaded but it still shows in the metadata, so you can get the performance benefits of doing less work but can still analyze the dependency DAG easily)
Dlopen has drawbacks but also major benefits. We decided the benefits relatively clearly outweigh the drawbacks, but of course people may disagree.
I have proposed a mechanism before, that would expose the list of libs we potentially load via dlopen into an ELF section or ELF note. This could be consumed by things such as packagae managers (for auto-dep generation) and ldd. However there was no interest in getting this landed from anyone else, so I dropped it.
Note that there are various cases where people use dlopen not on hardcoded lib names, but dynamically configured ones, where this would not help. I.e. things like glibc nss or pam or anything else plugin based. But in particular pam kinda matters since that tends to be loaded into almost any kind of security relavant software, including sshd.
That's what I think too. Do the relevant docs point this out too? Ages ago they didn't. I think we should try to avoid that people just google "implement systemd notify daemon" and end up on a page that says "link to libsystemd and call sd_notify()".
Inaccurate.
It's not pulled in on any sysvinit Debian system I run. It is on stable, oldstable, and oldoldstable systems via systemd.
Not systemd:
# ldd $(which sshd) linux-vdso.so.1 (0x00007ffcb57f5000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fbad13c9000)
libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007fbad13bd000)
libaudit.so.1 => /lib/x86_64-linux-gnu/libaudit.so.1 (0x00007fbad138c000)
libpam.so.0 => /lib/x86_64-linux-gnu/libpam.so.0 (0x00007fbad137a000)
libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007fbad12d5000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fbad12a5000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fbad1253000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fbad1179000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fbad1173000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007fbad0c00000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fbad1154000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbad0a1f000)
libnsl.so.2 => /lib/x86_64-linux-gnu/libnsl.so.2 (0x00007fbad1137000)
libcap-ng.so.0 => /lib/x86_64-linux-gnu/libcap-ng.so.0 (0x00007fbad112f000)
libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007fbad1123000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbad156a000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007fbad1089000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fbad09f2000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fbad09e4000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fbad09dd000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fbad09cc000)
libtirpc.so.3 => /lib/x86_64-linux-gnu/libtirpc.so.3 (0x00007fbad099e000)
# ldd $(which sshd) linux-vdso.so.1 (0x00007ffc4d3eb000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007feb8aa35000)
libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007feb8aa29000)
libaudit.so.1 => /lib/x86_64-linux-gnu/libaudit.so.1 (0x00007feb8a9f8000)
libpam.so.0 => /lib/x86_64-linux-gnu/libpam.so.0 (0x00007feb8a9e6000)
libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007feb8a916000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007feb8a8e6000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007feb8a894000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007feb8a7ba000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007feb8a7b4000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007feb8a200000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007feb8a795000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007feb8a01f000)
libnsl.so.2 => /lib/x86_64-linux-gnu/libnsl.so.2 (0x00007feb8a778000)
libcap-ng.so.0 => /lib/x86_64-linux-gnu/libcap-ng.so.0 (0x00007feb8a770000)
libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007feb8a764000)
libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007feb89ed8000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007feb8a735000)
libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007feb89e1c000)
liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007feb8a70d000)
/lib64/ld-linux-x86-64.so.2 (0x00007feb8abb5000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007feb89d82000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007feb8a6e0000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007feb8a6d2000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007feb8a6c9000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007feb8a6b8000)
libtirpc.so.3 => /lib/x86_64-linux-gnu/libtirpc.so.3 (0x00007feb8a68a000)
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007feb89d5a000)
# ldd $(which sshd) | grep liblz
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fd1e647a000)
liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007fd1e6398000)Looking at most popular projects these days they are a mass of dependencies and I think very few of them can be properly audited and verified by the projects that use them. Rust and Go might be more memory safe than C but look at the number of cargo or go modules in most projects. I have mostly stopped using node/npm on my systems.
I've been learning NixOS for a few years now, and it would have been impossible without systemd. It's one heck of a learning curve, but when you get to the other side, you know something of great power and value. Certain kinds of complexity adds 'land' (eg. systemd) that can become 'real estate' (eg. NixOS), which in turn hopes to become 'land' for the next innovation, and so forth.
Whether this happens or not (whether it's the right kind of complexity) is really hard to assess up-front, and probably impossible without knowing the complex new technology in question very well. (And by then you have the bias of depending, in part, yourself on the success of the new tech, as you've committed significant resources to mastering it, so good luck on convincing skeptical newcomers!)
It's almost like a sort of event horizon -- once you know a complex new technology well enough to see whether or not it's useful, the conflict-of-interest makes your opinion unreliable to outsiders!
Nevertheless, the assessment process itself, while difficult to get right, is worth getting better at.
It's easy for impatience and the sensation of what I've taken to calling 'daunt' -- that intrinsic recoil that the mind has from absorbing a large amounts of information whose use case is not immediately relevant -- to dissuade one from exploring. But then, one never discovers new 'land', and one never builds new real estate!
[ Aside: This is why I'm a little skeptical of the current rebellion against frontend frameworks. Certainly some of them, like tailwind, are clearly adding fetters to an otherwise powerful browser stack. But others, like Svelte, and to some extent, even React, bring significant benefits.
The rebellion has this vibe like, well, users _should_ prefer more simply-built interfaces, and if they don't, well, they just have bad taste. What would be more humble would be to let the marketplace (e.g. consumers) decide what is preferable, and then build that. ]