However, there are a few aspects of it that are inconvenient.
Automount units use an unintuitive naming scheme, and you are not free to name them as you wish (as you might for a socket unit). If you are mounting an NFS volume immediately below the root directory, you don't see the problem, but if the mount is several directories deep and/or uses ASCII symbols (non-alphanumeric), it is not pretty.
Socket units require two files per port. When I am moving complex inetd.conf setups to Linux, it's far easier to implement them with busybox inetd than convert dozens/hundreds of services to unit files, despite the increased functionality. Somebody has probably written some scripting to do this.
I am not aware of any include directive for my own directories, so I don't have to place everything in /etc/systemd/system. There probably is a way to do this, and I am betraying my ignorance.
And my, things can get messy in a hurry in /etc/systemd/system.
I don't know how to configure users to be able to maintain their own (personal) units.
And lastly, it's so seductive that I have no idea how to do many things in other operating systems that I easily do in Linux. I wish this itself was not a walled garden (but I'm not leaving).
Systemd by default looks at a bunch of directories depending on the context.
$ systemd-analyze unit-paths (or --user / --global)
https://www.freedesktop.org/software/systemd/man/latest/syst...
This is something I use all the time! Just put unit files in ~/.config/systemd/user and use systemctl --user to start/enable them
The point about there existing no competition and no respect for glibc alternatives is very valid and to that I can only say; if you build it, they will come.
Systemd had the advantage of Red Hat backing so they got a great headstart. But in general people will use whatever works. So roll your sleeves up and get cracking.
(In case you don't believe me: it is now impossible to run a desktop without a half dozen Systemd shims, because the entire ecosystem is now hard coded to use Systemd. I know because my desktop is Alpine Linux, a musl distro)
I find that hard to believe, unless what you mean is "it is impossible to run a desktop with all of the features that I want, many of which are coincidentally easier to implement/maintain using systemd ecosystem features"
The article is such a good take it makes me want to try out the author's Adélie project.
So for example, Red Hat backing meant that Canonical would have to be pulled in kicking and screaming to use SystemD and Ubuntu is one of, if not the most influential distro in the Linux world.
Ubuntu created and used upstart until it basically lost out to SystemD and had to move over.
DNS and all the additions seem to be really not easy to handle.
Also, NIH is a hell of a drug.
After one particularly deep rabbit hole where I never actually solved a problem where local network mDNS lookups always took exactly 6 seconds, I wound up disabling it entirely in favor of unbound, which works great but revealed that a handful of other things seem to expect resolved to be present and won’t work properly without it.
I can see how people who want to run their own tight ship on their machine would balk at it. I also sort of hate the systemd-resolv and the fact that it instantly rendered worthless 99% of online guides of how to unfuck your local DNS resolution.
I have a lot of network shares and some of them caused real trouble when they failed for whatever reason.
Now I really appreciate it.
I disagree whole hearty. With 15 years of system administratorship, me and systemd don't get on.
Services timing out on start-up/shutdown, "waiting x/y -- x/y retries and hijacking resolv.conf are my common two.
What would you prefer happen instead? Having seen plenty of downtime caused by not having retries or hanging indefinitely weren’t exactly improvements in my experience.
In all seriousness this is a wonderful article. Has the author/op seen this talk[0] by a FreeBSD developer on systemd?
What I like about the talk is asking about what is going to happen when we need to move on to newer ways of doing things that will have advantages and disadvantages over the old way. We can see a similar story with X11 and Wayland. Systemd does do some things better than the old way but does have flaws.
Reimplementing such a massive piece of infrastructure is pretty daunting. I don't think many people who are not being paid to would embark on it and then keep up the momentum to actually cover all those edge cases and maintain a community around it.
Rewriting things in rust seems to be pretty motivating though, so maybe it could be a force for good here
https://github.com/KillingSpark/rustysd
Rustysd is a reimplantation of the core features of the systemd service manager with a deliberate focus on less features so it can work on more platforms (e.g. BSD)
It's been a while since I actively progressed on the project but it works and would probably be a good starting point for anyone wanting to do further work in this direction.
dinit may lack many things - I don't know - I haven't had any reason to be unhappy with it and it seems to work in a fairly understandable way. I defined a new service (for minidlnad) and that appears to be straightforward.
I'm not overly found of timer units, either. They seem feature rich but much more complicated to setup than a single line in a crontab.
On the other hand, real dependency management between daemons, mounts, and sockets is a huge win.
I can't quite remember the right terminology but afaik systemd-resolvd supports routing different domains to different DNS servers on different network interfaces which can be auto configured via DHCP. The practical implication being, you can connect to a split tunnel VPN and domains accessed over the VPN get routed to the VPN DNS server.
Edit-
I think split dns more commonly refers to serving different internal/private and external/public zones.
In this case, the DNS servers may return the same results but if I'm connecting to a VPN running some distance away, I don't want to route all DNS requests to that server--only the ones for domains on that network. The Domains directive allows configuring this https://man.archlinux.org/man/systemd.network.5#%5BNETWORK%5...
You can add a dhclient hook to reconfigure dnsmasq with `server=` directives to achieve the same thing, but, as mentioned, that still requires restarting the dnsmasq daemon. That usually manifests as: the VPN connection times out, the hook runs, dnsmasq is restarted, but a random web page or request will fail while the daemon is restarting. Sure, you could do SO_REUSEADDR or something to prevent the disruption, but, once again, this all just works out of the box with systemd-*
I’ve twice tried to use them. Closest I got to having one work was that it reported it ran, and didn’t error, but it did not run. Extremely simple cases that were basically copy-paste an example and barely modify it, too.
Oh well, add a line in cron and forget about it.
For example, all of my boxes spew many logs per minute about "Failed to set up mount unit". This is apparently a bug with the generated name of some internal unit related to mountpoints (too long). This one is not as bad as others, because it does have a bug reported for it, but is still something I'll have to deal with until I upgrade the distro on all the servers. Many similar bugs I can't track down at all.
Systemd makes a lot of things easy, but not simple, and that is a big problem in practice.
https://serverfault.com/a/1143851/92104
There's no reason for this not to work! Taking a very straightforward and traditional approach to running processes (as in my demo code in that question), you don't have this problem. But rather than that, they did something clever which doesn't work properly.
If the first answer is correct, then the unbuffer works because the process introspection as the source of the output is unified to be the unbuffer process. But why bother introspecting the exact process that originated a write to the output? Did we have a problem with file descriptors being highjacked? And even if we did, losing the output (or not associating it with the parent process) is worse.
But the author is especially right on the need for competition. So I am working on it.
I am building a build system, and I am making it usable as a library. When I have made it so, I will implement an near drop-in [1] replacement for systemd.
You'll be able to use systemd unit files, and there will be an option to have binaries with the same name (off by default to not interfere, though) so that users don't have to learn new stuff right away.
[1]: I am not going to implement journald for example; logs will be text, even if they are structured internally.
Besides, implementing things to be perfectly compatible would only strengthen the monoculture, not weaken it. The plan is to be compatible on the things that matter for distro integration, so that distros can ship both with little work, but have other things different, so that users can choose what works best for them between the two.
The author is very wrong on the systemd-resolved bit.
Not understanding the ability to have per-interface specific zones is ok; it is a thing for desktops with multiple interfaces that come and go (like VPNs, for example). There is no other resolver on Linux capable of doing it and integrating with NetworkManager. You can kinda-sorta make dnsmasq do it, but with some limitations.
But the crown is taken by the "mDNS is better taken care of by a dedicated package like Avahi." Uh, oh. It is like saying, that you don't need Chrome, it's duties are better taken care by nginx. Similarly, you can use Avahi for _advertising_ mDNS services, but not for _resolving_. Which, as an user, you probably are interested in.
There's also nss-mdns at https://github.com/avahi/nss-mdns but it is mostly unmaintained.
The actual "customers" of linux now - the ones providing the money that employs people - aren't Linux enthusiasts but big companies. So it's absolutely obvious that their needs are "what matter".
It's ok. The rest of us can fork and "be irrelevant" in the same way Linux was irrelevant before it became popular.
Something like supervisord, but you use systemd unit files, and it tries to do as much as it can within limitations (process tracking sure can be wonky, no dbus everywhere, no cgroups everywhere, no absolute freedom in resource limitations if you’re not PID 1).
Here are some of my criticisms, although I do still use systemd daily on my personal devices and servers.
* Bad security
Systemd is architected in a way that has a lot of code running as root, it's also written in a language that isn't memory safe. This means it has a large attack surface (a lot of code you need to make sure is bug free to be secure) and it's harder to make sure your code doesn't have really severe security related bugs (The [NSA recommends](https://www.nsa.gov/Press-Room/Press-Releases-Statements/Pre...) using memory safe languages for critical stuff like this).
There are certainly other critical projects (like this linux kernel) that are similarly important and written in memory unsafe languages, but systemd has had some [pretty critical vulnerabilities](https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=systemd) discovered that do not inspire confidence in their ability to use these kinds of dangerous languages safely, and they don't have nearly the same budget as the linux kernel for detecting and preventing these kinds of issues.
This is less of a problem if you're a large enterprise customer running up to date SELinux, but it should still have been possible to write systemd in a way that limited the pid1 attack surface while retaining all current functionality.
* Journalctl is a pain for desktop users and smaller teams
Journalctl is how systemd manages logs. By default it saves logs in a binary format with additional metadata. This makes it easier for large teams to ingest the logs into a centralized log-collection daemon, like the ones offered by redhat for enterprise deployments, but it breaks a lot of workflows that older sysadmins probably used. Things like just rsync-ing a bunch of logs to one place, or using tools like grep and find to inspect logs. Systemd does of course provide replacements for those tools, instead of using grep you can use journalctl to search through your logs, but you could use grep to search through any text file. Config files, source code, or logs. Now I need to memorize all the flags for one more tool, and change a bunch of stuff about how I collect logs.
This also presents a challenge for people doing embedded work, as you can't just grab the SD card out of a system can look at it's logs. You need a working journalctl CLI on your host machine. They've add in a --directory flag, but in the past this was much harder, requiring you to actually chroot into the embedded system (that may be broken in strange ways) in order to read logs.
Journalctl has advantages, but they're mostly enjoyed by large enterprises.
Yes I know it's actually systemd-journal or just "journal" or whatever they call it. Journalctl is the command most users will be familiar with though.
* Poor [Locality of behavior](https://htmx.org/essays/locality-of-behaviour/) makes it harder to reason about
When you're trying to understand how a system works it's nice to be able to see everything in one places. There are like 9 different places a systemd unit file can live, you can apply an over-ride to a unit file, unit files can depend on other files like socket files.
This is good for large teams as it makes it easier for specific groups in a company to claim ownership over parts of the system, but it means as a desktop user or sysadmin working with a small business you've added a lot of complexity. You can't just type `ls /etc/init.d` to get a rough overview of what services exist, you need to memorize more systemd-specific commands. If you want to edit a service you can't just edit a service, you need to create an over-ride using another systemd specific command, make sure you have the EDITOR environment variable set up, and then open the original service in another editor so you can compare the two.
It creates some more work and complexity and encourages you to use a bunch of systemd-specific tools (and presumably get red-hat certified training).
* People use systemd stuff before it's ready
I'm not sure this is something I can blame systemd or redhat for, but the official stance of redhat is that systemd-resolved is not ready for production, and yet it's used all over the place. That can give people a poor impression of systemd after the 9th time they try to do something even slightly different with their networking setups and systemd-resolvd breaks, not to mention the numerous security issues in systemd-resolvd.
* Unix philosophy
A lot of people say that either unix philosophy doesn't matter, or that systemd does embrace unix philosophy. That stuff about logging and systemd-specific tools I mentioned above? That's what people actually mean when they talk about unix philosophy, they mean being able to grep through their logs and rsync logs to a remote server. They mean using standard text files and not needing to have a special command that wraps your text editor to "properly" edit a unit file.
* Doesn't run in a chroot
As someone who splits my time between embedded linux and server linux this is just a personal pet peeve of mine. It makes it very hard to debug embedded systems that use systemd if you're not using systemd on your workstation. It does feel like I'm being forced to use systemd sometimes, and while I've largely gotten over it I'm still a bit bitter. It's also made some small personal projects, like getting a full linux distro running on a KoBo e-reader, much much more difficult than they had to be.
It's a mess under docker, and why I need to use alternative OCI-runtimes like nestybox to do a bunch of testing for embedded systems. Thankfully I wasn't an early adopter to docker and didn't have those problems until there were already mature solutions. But there's really no reason why it should have to run as pid1, other than them wanting you to use docker-alternatives that are deeply integrated with systemd, like their podman tool or systemd-nspawn. This was just such a blatant attempt to abuse their near-monopoly position that it bears some extra whining.
* OpenRc does everything systemd does, but better
Unfortunately other red-hat influenced projects like Gnome won't support or test on non-systemd init systems, let alone providing default services files for them, meaning that any distro that wants to be compatible with gnome will need to do a bunch of extra work to write and test service files. For one project that's potentially reasonable, and certainly there are distros that do that extra work, but for projects like Arch who have the explicit goal of sticking as close to upstream sources as possible it makes it more or less impossible.
Systemd survives not because it's a good solution to the problem, but because it has a large corporate backer, is widely deployed, and is a safe thing to code against. There's a very old IT saying, "No one ever got fired for buying IBM". If you pick a safe industry-standard options no one can blame you if it goes wrong, even if it's the technologically inferior option.
As much as I'm a systemd-hater I still do use it on my personal devices and servers, because it's by far the path of least resistance.
I hope we see a similar situation like with pulseaudio and pipewire, where the pulseaudio rewrite was much much nicer than the original. I don't think that's going to happen until systemd slows down though, right now if you tried to re-implement systemd I suspect you'd get the rug pulled out from under you as they changed standards and behaviors (I've seriously thought about doing it myself, at least for relatively simple unit files). I'd still prefer to be using OpenRc, as I don't know how a rewrite would deal with the locality-of-behavior issues, but systemd has been getting better and more reliable over time.
I don't agree with this. As a single administrator, being able to do `journalctl -u <service>` and get all output from a unit (and its subprocesses), including anything that it erroneously send to stdout or stderr instead of its logfile, is a godsend. Back in the SysV days I had to manually step through initscripts to figure out what's wrong when a daemon immediately exited without logging anything.
I'm not comparing it to sysvinit here, I'm comparing it to any other modern init system.
You never really got that on svinit systems either, since there was still inetd, and cron had at least three different places you had to look as well. With systemd, you have systemctl list-units and you're looking at everything -- to say nothing of being able to filter by status. rc files make you implement service control yourself from scratch every time (save for a small set of primitive utility functions), and every one is a snowflake.
What would you want to move to a different user specifically?
Systemd is the first one that actually makes modern security features first-class. Most big services run with NoNewPrivileges set, with specific paths allowed, with PrivateTmp, and a bunch of other features. Then there's DynamicUser and other fun bits.
Not only did systemd make it simpler and more common to run bits as different users, it also makes the root ones more restricted.
That's just objectively wrong.
systemd uses less privileged accounts for many parts, for example systemd-networkd runs as the systemd-network user. This is different from the classic UNIX way of running everything as root (ISC DHCP Client, NetworkManager, ifupdown, ... all just run as root).
I sincerely believe that systemd solved a problem that nobody was willing to solve, and it has every right to solve those problems any way it wants (when you ask for help you don't get to choose how you are helped after all) - my concerns boil down to the fact that the adoption it has seen has lead indirectly (or, directly) into a monoculture; and a monoculture that almost certainly will stifle innovation since a replacement will need to be bug-for-bug compatible. Prior init's were actually quite easy to replace and alternatives were often used, but these days most software assumes systemd and this situation gets worse every year.
That said; and with the knowledge that while I am afraid of systemd as monoculture (and a relatively opaque one); this line "systemd, as a service manager, is not actually a bad piece of software by itself. The fact it can act as both a service manager and an inetd(8) replacement is really cool."
Is something I vehemently disagree with.
For starters, (x)inetd is an anti-pattern, everything I understand about systems development indicates we should be seperating concerns as much as possible, having one super-server that launches everything under one daemon is directly opposed to this.
"But", I hear you thinking already: "systemd runs services as independent users, it solves that!", and I would agree with you, except now pid 0 is listening to the network instead.. That doesn't strike me as much better.
If there's a bug/backdoor in your binary distributed version of systemd then you are SOL and your whole system is owned as root, but at least a bug in your application might not expose your entire inetd user. :\
It's also a common issue that inetd's architecture can lend itself to getting DoS'd harder than other more-standard daemons, except now it's your pid 0 being DoS'd; not sure how you recover from that honestly.
EDIT: if you are going to downvote, please provide reasons. Sick of this holy war, lets just end it reasonably please.
While I do agree that a "super server" listening to the network could be a new (or old, with (x)inetd) concern, a well-written and well-designed pid1 doing this isn't much more risky than having the services manage themselves. The listener could/should be just as unprivileged as the target daemon, nominally using the same uid/groups and root directory as the target as well. And indeed, systemd's .socket files support the same environment control variables (User=, Group=, SupplementaryGroups=, RootDirectory=, WorkingDirectory=, et al) that services do.
So that boils down to "are people writing socket activation units correctly", which is probably "no", but could be "yes".
The issue is that the previous "multi-culture" really sucked real bad. It was a shit show, really.
systemd is rising the bar by A LOT in terms of systems management, and most alternatives are simply not keeping up.
But can you really blame it on systemd making a stellar job on its own?
If anything, we could blame it on the "alternative projects" doing a fairly poor job and delivering very little.
edit: systemd is so good that FreeBSD people have already started pondering if they should build something similar for themselves: https://www.youtube.com/watch?v=o_AIw9bGogo
Not really. Sysvinit scripts are... scripts full of sysvinitisms (double forking and PID files, anyone?). Idiomatic systemd daemons are, comparatively speaking, very straightforward. Of course, there are a lot of nice-to-have features, but to get a running system, you should largely be able to get away with parsing Wants=, After=, and ExecStart=.
... well, a bunch of scripts are something that's relatively easy to replace. It's not as though other system components have reliance of these scripts and their sysvinit'isms baked in.
From what I've heard (though not verified) - non-systemd distributions seem to manage to work with upstart, or openrc, instead of sysvinit, without much hassle.
What problem was there which nobody was willing to solve?
Also, "solving" a problem by creating a mechanism that in itself highly problematic in other ways does not necessarily count as a solution; it is a shifting-around of problems.
-----
The systemd "holy war", such as it is (mostly complaints on forums and in blog posts and in personal chats; don't remember any violence in this war) - is due to four reasons:
* The significant problems which systemd introduces.
* The way systemd has been developed and managed as project (including some grievances with individuals).
* The fact that distributions have not only adopted systemd, but made it effectively impossible to opt out of.
* Faults with the process in which systemd was adopted as a required default by all of the main distributions - bypassing wide opposition without addressing its criticism. So, the adoption of systemd exposed technical-governance/power-dynamics problems in the Linux distro world, with systemd serving as the symbol for those.
The fourth item is what I find to be the most interesting. I really hadn't thought about it that way before. I remember the discussion in Fedora mailing lists around systemd and iirc the criticisms were mostly actually answered and handled. But in Debian it really wasn't and it fractured the community a lot.
It is perhaps the case that the "anti-systemd" crowd wouldn't be "anti-systemd" if they felt like they had been heard. Perhaps that's the bigger lesson we should all be learning: to listen closely and respond respectfully.
The job of a distro is to decide what to ship and integrate that in the best way possible. Would a distro let you choose between glibc and musl for example? If you want either, use a distro that chooses that.
It means that this server had to be carefully secured but the benefit is that you have exactly one bit of heavily-audited code listening to the network, logging activity & problems, starting processes, changing users / dropping privileges, setting up namespaces, etc. I’ve seen a lot of code get various combinations of those wrong so I think that’s a far more nuanced problem than in your portrayal.
Those are all things people should know how to do but I’ve seen Java or PHP running as root in production because someone couldn’t figure out how to drop privileges needed only at startup enough times to appreciate the benefits from systemd making it so much easier to do things right.
It's in a terrible way, too. Not sure it improved but I recall somebody once, at a company I worked at, trying to introduce per-unix-user systemd services (e.g. `graphite` user for running Graphite), and there were some atrocious steps required like `loginctl --enable-linger` and God knows what.
We moved to running everything systemd as root, it's easier (and you can specify which unixuser the actual systemd unit runs at, which is "close enough").
Were there multiple implementations of sysv-init being used by different distros on Linux before systemd came along?
Pid 1, surely.
The shortcomings of systemd seem likely the overall shortcomings of open source: none of the $PROJECT maintainers experience $PAIN_POINT, so it is simply not a priority for $PROJECT.
Somehow this seems a variation on the tragedy of the commons => https://en.m.wikipedia.org/wiki/Tragedy_of_the_commons
Without some capitalist skin in the game, the $PAIN_POINTS become difficult to prioritize.
So we need to start with 'init'.
'init' -- even in its absolute first, simplest incarnation -- is still too complex to understand correctly!
You see, we need to shift perspectives!
We need to shift perspectives from a longtime System Administrator -- to that of a new barebone OS programmer.
What is 'init'?
Is 'init' a program that handles runlevels, starts and stops services, that mounts filesystems, that processes messages, that captures dead processes, that waits for hardware to become available, that logs and maintains informational/database/etc files, that starts audio, that starts X11, that stars the GUI, that acts as a proxy for sockets, or does anything else with the system?
No!
From the point of view of a new barebone OS programmer (as Dennis Richie and Ken Thompson were when they invented Unix and invented 'init') -- 'init' is NONE of these things!
'init' is only THE FIRST PROGRAM, THE FIRST COMPUTER CODE THAT RUNS IN USER SPACE.
And that's it!
That is all that 'init' ever is, or ever was!
(User space, to recap, is the unprotected AKA "unprivileged" AKA "non-supervisor" memory running unprotected (AKA "user-land") code: https://en.wikipedia.org/wiki/User_space_and_kernel_space)
'init' (and every single 'init' successor program, i.e., OpenRC, systemd, etc.) -- are the first program, the first set of computer code OUTSIDE OF KERNEL CODE (which has been running and is currently still running) to be run by the system.
Now, what should that first program do?
See, that's the magic question -- which gives rise to all that is to follow!
In theory you could have an OS where the 'init' program, or its equivalent -- did absolutely nothing! But that wouldn't be very productive!
If the 'init' program isn't itself a shell program (i.e., sh, bash, etc.) -- then (because there's no GUI at this point) the computer will not be able to accept typed command-line commands -- which is the first thing that you want a new OS to do!
So now our 'init' expands in scope (and lines of code)!
Our 'init' could be hardcoded to launch 'sh' or 'bash' (or whatever shell program exists) -- but what if the user wants to change that?
OK, so now we need our first configuration file. Where to put that exactly?
Oh, it's on a filesystem that hasn't been mounted yet?
Well, maybe init should mount that filesystem!
Point is, there's a set of problems (and sub-problems!) -- which give rise to increasing and increasing init's functionality over time!
init, as the first user-space program for an OS to run, on whatever OS it is ran on, in whatever form it is in -- could simply be written to run and 'outsource' all of its functionality to other programs...
But init (as it evolved into its very large LOC complex descendants) -- became a "dumping ground" -- for functionality that was inconvenient to go in other places and/or to be outsourced to other programs.
See, all of the code in all userland Linux utilities -- could in theory be grafted together into one big super program in userspace.
It would have the same functionality as all of the individual Unix/Linux command-line programs put together (and maybe that would be desirable to some people). But from a Software Engineering "separation of concerns" AKA dependency reduction AKA modularity AKA "do one thing and do it right" AKA loose-coupling perspective -- doing that might not be so desirable!
And yet, with the complexity brought about by 'init' descendants -- it seems like we're going down that exact route!
Which leads us full circle (because history always repeats itself!) -- back to the reason why Unix was created -- because of the complexity and problems brought about by the complexity of its predecessor, Multics!
https://en.wikipedia.org/wiki/Multics
Point is -- 'init' in whatever form it takes -- is by no means obligated to do anything -- although if it is to do nothing, then it should at least launch one other program which will do something! If that's the case, then why not put that under user control? But wait, if we're doing that, why not make it launch multiple other programs! OK, now we need a file to tell init where that should be! But what if the filesystem for that file is not mounted?
Anyway, you see how the "rabbit hole" of problems (and increasing LOC complexity) forms!
Related: https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...
Except it is not. The "init" process seen by users as PID 1 on the usual Linux distributions is far from the first program started by the kernel. There are usually hundreds of processes started before that PID 1 process gets started.
By the time PID 1 is started, the OS has run OS code, or more specifically, prior to PID 1 being ran, the OS has run OS code -- that much is true!
Loosely, you are calling that OS code -- "processes"
We could call that OS code "processes" -- however, there is a difference between this loose definition of what a "process" is, and what a "process" is defined to be by Unix/Linux standards and conventions.
The OS code that the OS runs, the kernel code -- typically does not have a 'PID' (Process ID) associated with any of it.
To understand this, let's remember what Unix/Linux (or any OS) is -- it's software abstraction layer over hardware.
It's goal is to act as a hardware abstraction layer, but also as a moderator, a resource arbiter for other pieces of software, other code -- to run on the same hardware, the same machine, at the same time.
This other code -- is typically the user space (AKA non-privileged) software, the user-space code.
In order to manage that user-space code, AKA user programs, AKA user-installed software and because several (or more) of these programs might be and probably will be running simultaneously, a unique postive integer handle (functionally the equivalent of a key in key-value programming).
That unique positive integer handle assigned to each user-space program (unique ones can also be assigned to threads of programs) -- is typically the PID, or Process IDentifier.
That's WHY they exist.
PID 1 is the Process Identifier for the first user-space program to be run by the operating system, typically 'init' -- or a descendant.
If "There are usually hundreds of processes started before that PID 1 process gets started", which is what you're calling "processes" (kindly disambiguate exactly what you mean when you say "process" please!), then WHY is it that they are not given ascending unique positive integer numbers and WHY does this numbering, should it exist, force PID 1 to be some other higher-numbered PID?
If "hundreds of [user-space] processes [are] started before that PID 1 process gets started", then WHY exactly, is the first ran user-space program PID 1 -- and not some higher numbered PID?
?
???
In other words, it has historically been the program/process used to manage other programs/processes.
Consider the simplest possible Unix/Linux system with only two user-space processes. For a Kiosk, let's say.
The first user-space process/program is 'init'.
The second (and last!) user-space process/program is the kiosk software itself. Call it KIOSK.EXE (or /usr/bin/kiosk -- since this is Unix/Linux, or what-have-you)
OK, so now here's the problem!
What should happen if KIOSK.EXE or /usr/bin/kiosk -- should fail for some unexpected reason?
Some stack-heap collision, some segmentation fault, some core dump, some "blue screen of death", some abort, some failure...
OK, so now how do we:
a) Monitor that kiosk software to know if its up and running;
b) Bring it back up and running if it has crashed?
Well, if there's only 'init', aka the PID 1 program/process in the system (other than the kiosk software, and the kiosk software is crashed, no longer functional) -- then it must be 'init' (or PID 1's) responsibility to bring 'kiosk' back up!
To get 'kiosk' back up and running!
That's WHY the "process management stuff" functionality -- historically has been placed into 'init' (PID 1) -- simply because there was no other easy/simple/straightforward/obvious/user-friendly place to put it!
But 'init' ultimately is just a user-space program.
It could be programmed to do nothing, it could be programmed to do everything (its code could in theory be fused with the kiosk program, although then it becomes the OS's responsibility to restart that program if it crashes!)
But ultimately 'init' -- is just a program!
It could, if programmed to, do whatever any other user-space program could do.
It could also be programmed not to do anything -- or offer the barest minimal functionality.
What 'init' does or does not do, or is supposed to do or not supposed to do -- is truly at the behest of any programmer who wish to work with the source code of 'init', in whatever form that may take...
Nothing "HAS to" be any particular way for any OS or user program if you have the source code, if you know what it does, and if you know what you are doing...
There are no "HAS to's" -- for those that have source code and understand it.
