systemd reference

What is systemd?

systemd is an init and system manager that is used on most Linux distros. Once the Linux kernel has booted, systemd runs as PID 1 and initializes user-space processes.

For example, log in as root, run ps -ax | head, and you should see PID 1 is /sbin/init:

 PID TTY      STAT   TIME COMMAND
      1 ?        Ss     1:55 /sbin/init
      2 ?        S      0:00 [kthreadd]
      3 ?        I<     0:00 [rcu_gp]
      4 ?        I<     0:00 [rcu_par_gp]
      6 ?        I<     0:00 [kworker/0:0H-kblockd]
      9 ?        I<     0:00 [mm_percpu_wq]
     10 ?        S      0:12 [ksoftirqd/0]
     11 ?        I      0:37 [rcu_sched]
     12 ?        S      0:11 [migration/0]

Run ls -l /sbin/init and you should see that it’s a symlink to /lib/systemd/systemd:

lrwxrwxrwx 1 root root 20 Jan 10 04:56 /sbin/init -> /lib/systemd/systemd

systemd manages your processes as “units”

systemd operates on “units”, which are processes / resources that it knows how to manage. It classifies units into several types such as a service, socket, device, timer, and more.

Each unit is configured in a configuration file that has its type suffix (e.g., nginx.service, ssh.socket, apt-daily.timer, etc).

Where are the units?

On Ubuntu 20.04 (any many other distros), the unit files can be found in lib/systemd/system. For example, running:

ls -l /lib/systemd/system | head

prints something like:

-rw-r--r-- 1 root root  741 Nov  9 12:23 accounts-daemon.service
-rw-r--r-- 1 root root 1162 Apr 10  2020 apparmor.service
-rw-r--r-- 1 root root  212 Dec  4  2019 apport-autoreport.path
-rw-r--r-- 1 root root  242 Dec  4  2019 apport-autoreport.service
-rw-r--r-- 1 root root  246 Dec  4  2019 apport-forward.socket
-rw-r--r-- 1 root root  142 Dec  4  2019 apport-forward@.service
-rw-r--r-- 1 root root  389 Jun 15  2021 apt-daily-upgrade.service
-rw-r--r-- 1 root root  184 Jun 15  2021 apt-daily-upgrade.timer
-rw-r--r-- 1 root root  326 Jun 15  2021 apt-daily.service

What does a unit configuration file look like?

Here’s the configuration file for the Nginx service:

[Unit]
Description=nginx - high performance web server
Documentation=https://nginx.org/en/docs/
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/var/run/nginx.pid
ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf
ExecReload=/bin/sh -c "/bin/kill -s HUP $(/bin/cat /var/run/nginx.pid)"
ExecStop=/bin/sh -c "/bin/kill -s TERM $(/bin/cat /var/run/nginx.pid)"

[Install]
WantedBy=multi-user.target

Manage units in current session

Most of the time, you can leave out the unit suffix when running systemctl commands. So, instead of:

sudo systemctl status nginx.service

You can use:

sudo systemctl status nginx

Start unit

sudo systemctl start <unit>

Stop unit

sudo systemctl stop <unit>

Restart unit

sudo systemctl restart <unit>

Reload unit

Only works if the resource supports reloading.

sudo systemctl reload <unit>

Monitor units

Check status of unit

sudo systemctl status <unit>

Example output:

● nginx.service - nginx - high performance web server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-04-08 16:41:28 UTC; 3 weeks 5 days ago
       Docs: https://nginx.org/en/docs/
   Main PID: 32812 (nginx)
      Tasks: 2 (limit: 2339)
     Memory: 19.5M
     CGroup: /system.slice/nginx.service
             ├─32812 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
             └─32813 nginx: worker process

Apr 08 16:41:28 prod systemd[1]: Starting nginx - high performance web server...
Apr 08 16:41:28 prod systemd[1]: nginx.service: Can't open PID file /run/nginx.pid (yet?) after start: Operation not permitted
Apr 08 16:41:28 prod systemd[1]: Started nginx - high performance web server.

Check if unit is active

sudo systemctl is-active nginx

Check if unit is enabled

sudo systemctl is-enabled nginx

Check if a unit is in a failed state

sudo systemctl is-failed nginx

Filter units by state

sudo systemctl list-units --all --state=<active|inactive|failed>

Filter units by type

systemctl list-units --type=<service|socket|timer,etc>

View unit file as known to the current running unit process

systemctl cat <unit>

List unit dependencies

systemctl list-dependencies <unit>

List dependences that must start before unit

systemctl list-dependencies <unit> --before

List dependences that must start before unit

systemctl list-dependencies <unit> --after

Show properties of unit

systemctl show <unit>

Example output for systemctl show nginx | head :

Type=forking
Restart=no
PIDFile=/run/nginx.pid
NotifyAccess=none
RestartUSec=100ms
TimeoutStartUSec=1min 30s
TimeoutStopUSec=1min 30s
TimeoutAbortUSec=1min 30s
RuntimeMaxUSec=infinity
WatchdogUSec=0

Manage units on boot

As well as managing units in the current session, you can instruct systemd to start a unit at boot by enabling the unit.

Enable unit

sudo systemctl enable <unit>

This will create a symlink from the system’s copy of the unit file (usually in lib/systemd/system) to the location on disk where systemd searches for autostart files (usually /etc/systemd/system/some_target.target.wants).

Note: enabling a unit doesn’t start it in the current session. You’ll have to use the start command or reboot the system.

Disable unit

sudo systemctl disable <unit>

This will remove the symlink created after the previous enable command.

Mask unit (make it impossible to start)

systemctl mask <unit>

This will symlink the unit file to /dev/null, making it impossible to start them either manually (via systemctl start or automatically (via scripts or as dependencies to other units). Use the --now option to also stop the units now.

You can check masked units with:

systemctl list-unit-files --state=masked

Unmask unit (re-enable for start)

sudo systemctl unmask <unit>

Edit unit files

Merge custom snippet config with full unit file

sudo systemctl edit <unit>

This will create a file at /etc/systemd/system/<unit.d>/override.conf

Edit the full unit file

sudo systemctl edit --full <unit>

This will load the existing unit file into the editor. When you save and exit, the file will be written to /etc/systemd/system, which will take precedence over the system’s unit definition (usually found in /lib/systemd/system).

Other notes

• systemd keeps track of processes using Linux control groups (cgroups).
• systemd manages unit dependencies.

Sources

• systemd – Wikipedia
• Rethinking PID 1
• systemd.io
• How To Use Systemctl to Manage Systemd Services and Units – DigitalOcean
• Managing services with systemd – RedHat