In this article I will share the steps to enable persistent logging in
systemd-journald without rebooting the node or restarting the
systemd-journald service. The systemd journal is configured by default to store logs only in a small ring-buffer in
The default storage type in
journald.conf is "auto". In auto storage type the journal logs will not be persistent and will not survive reboots.
Different Storage Type supported with journald.conf
- The storage type value is controlled using /etc/systemd/journald.conf file. Under
[Journal]you can modify the storage type.
- The supported values are “volatile”, “persistent”, “auto” and “none”
- Default storage type is configured as “auto”
- If “volatile“, journal log data will be stored only in memory, i.e. below the /run/log/journal hierarchy (which is created if needed)
- If “persistent“, data will be stored preferably on disk, i.e. below the
/var/log/journalhierarchy, with a fallback to /run/log/journal during early boot stage and if the disk is not writable
- The “auto” value will configure journald to store journal log data in the
/var/log/journal/directory. However, the directory must already exist and have the proper permissions set. If it does not exist, then journal data is stored in the volatile
/run/log/journal/directory, and the data is erased when the system shuts down.
- “none” turns off all storage, all log data received will be dropped.
Enable persistent logging in systemd-journald
Now if your current storage type is auto in
/etc/systemd/journald.conf, in that case the “auto” value will configure
journald to store journal log data in the
/var/log/journal/ directory. However, the directory must already exist and have the proper permissions set. If it does not exist, then journal data is stored in the volatile
/run/log/journal/ directory, and the data is erased when the system shuts down.
Now in this case to enable persistent logging in systemd-journald you just need to create a directory
systemd-journald will start writing journal log files into the disk rather than memory.
With this change the logs will become persistent and will not be erased after reboot
Single step to be followed
# mkdir -p /var/log/journal
No need to restart the systemd-journald service or reboot the node. As soon as this directory is accessible, systemd-journald will start writing logs under this path
# ls -l /var/log/journal/3a0d751560f045428773cbf4c1769a5c/ total 8192 -rw-r----- 1 root root 8388608 Sep 4 08:15 system.journal
There is no problem in using the steps from Method 1 but again to enable persistent logging in systemd-journal properly we must change the "Storage" type to "persistent". In which case you need not manually create
Replace storage type with
# sed -i 's/#Storage.*/Storage=persistent/' /etc/systemd/journald.conf
Next you can restart
# systemctl restart systemd-journald.service
killall -USR1 systemd-journald. With this the memory log gets copied to the new disk location as well as all subsequent events.
With storage option "persistent", data will be stored preferably on disk, i.e. below the
/var/log/journal hierarchy (which is created if needed), with a fallback to
/run/log/journal (which is created if needed), during early boot and if the disk is not writable.
You can always use flush to move the journal log files from
# journalctl --flush
here this command asks the Journal daemon to flush any log data stored in
/var/log/journal, if persistent storage is enabled. This call does not return until the operation is complete
Lastly I hope the steps from the article to enable persistent logging in
systemd-journald on Linux was helpful. So, let me know your suggestions and feedback using the comment section.