Finding Why SSH Logins Fail at the systemd-logind Session Limit

When new SSH or SFTP logins to a Linux server fail and the journal shows "pam_systemd(sshd:session): Failed to create session: Maximum number of sessions (8192) reached", the server isn't out of CPU, memory, or disk. systemd-logind has reached its session limit. Every session logind still tracks counts toward that limit, including sessions that already logged out. This post covers how to count the sessions by state, find what keeps the closing ones open, clear them, and keep the count down.

A recent Ubuntu Community Hub thread describes an Ubuntu 24.04 SFTP server in this state, with thousands of sessions in the closing state, no leader process, and no session scope. A reload of systemd-logind didn't change the counts.

What the limit counts

SessionsMax= in logind.conf sets the maximum number of concurrent user sessions logind manages. The Ubuntu 24.04 man page gives the default as 8192.

The systemd documentation for sd_session_get_state defines three session states. An online session is logged in but not in the foreground. An active session is logged in and in the foreground. A closing session is "nominally logged out, but some processes belonging to it are still around".

Ubuntu sets KillUserProcesses= to no by default. With that setting, logind marks the session's scope as abandoned at logout and leaves its processes running. Any process started from an SSH login keeps that session in the closing state until the process exits.

Confirm the error

Count the refused logins in the last hour.

journalctl --since "1 hour ago" --no-pager | grep -c "Maximum number of sessions"

A count above 0 confirms that the session limit is refusing the logins.

Count the sessions by state

List every session ID, read its state, and count each state.

loginctl list-sessions --no-legend | awk '{print $1}' | xargs -n1 loginctl show-session -p State --value | sort | uniq -c

Run it again an hour later and compare the closing count. A closing count that rises between runs while the active count stays flat shows sessions that are never removed.

Find what holds a closing session

Pick one session ID in the closing state and read its leader process, scope unit, and user.

loginctl show-session <id> -p Leader -p Scope -p Name -p State

Then check whether the scope unit still exists and which processes are in it.

systemctl status session-<id>.scope --no-pager

When the scope still exists, the CGroup section of that output lists the processes left behind. A terminal multiplexer, a job started in the background, or an sftp-server process that never exited each keep the session in the closing state. Stop that process, or let it finish, and the session should be removed.

When the command prints "Unit session-<id>.scope could not be found." and the session has no leader process, nothing is left running in it. logind is still tracking a session it should have removed. Launchpad bug 2020177 reports the same symptom on Ubuntu 20.04, with the same error at 8192 sessions, and the reporter found that only a reboot cleared it.

Clear the closing sessions

The loginctl man page says terminate-session kills all processes of the session and deallocates all resources attached to it. Try it on one session first.

sudo loginctl terminate-session <id>
loginctl list-sessions --no-legend | awk '{print $1}' | grep -cx "<id>"

A count of 0 means logind released the session. When it does, loop over every session in the closing state. This also kills any process still left in those sessions, so check their scopes first for anything a user still needs.

loginctl list-sessions --no-legend | awk '{print $1}' | while read -r s
do
  if test "$(loginctl show-session "$s" -p State --value)" = closing
  then
    sudo loginctl terminate-session "$s"
  fi
done

Run the count by state again afterward to check the closing count went down.

When terminate-session doesn't clear them

Raise SessionsMax= in a drop-in file and restart systemd-logind. A higher limit delays the next failure, but the closing sessions keep adding up until they're removed.

sudo mkdir -p /etc/systemd/logind.conf.d
printf '[Login]\nSessionsMax=16384\n' | sudo tee /etc/systemd/logind.conf.d/sessions.conf
sudo systemctl restart systemd-logind

On Ubuntu 20.04 and earlier, restarting systemd-logind dropped the existing login sessions (Launchpad bug 1944711). The bug report lists Ubuntu 21.04 and later as already fixed. I'd still restart it in a quiet window on a production SFTP server. A reboot clears every session and is the fallback when neither step works.

Report the leak with ubuntu-bug systemd and attach the show-session output for one closing session.

Keep the count down

When terminate-session clears them, save the loop as a script and run it once a day from a systemd timer or a cron job until an update fixes the leak. Keep running the count by state on the same schedule, and compare the closing count with the SessionsMax= value.

Where ET Ducky shows it

I build ET Ducky, and its Linux agent reads process exec and exit events from eBPF tracepoints on every managed server. The start and exit of the sshd and sftp-server processes behind each login show up in the same event stream. You can open a Live Session shell from the browser without the need for a VPN and run the loginctl commands above on the server.

You'd start a Multi-Agent Session across the server that builds up closing sessions and one that doesn't. It compares the kernel events from both to find the outlier, and the raw events stay on the hosts.