You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pages/learn/manage/device-logs.md
+49-3
Original file line number
Diff line number
Diff line change
@@ -10,15 +10,61 @@ Device logging and the storage of device logs in {{ $names.cloud.lower }} is des
10
10
11
11
## Device logs on the balenaCloud dashboard
12
12
13
-
Device logs on the balenaCloud dashboard contains anything written to stdout and stderr of app services, and the system logs of the device. The dashboard allows logs to be cleared, filtered, searched and viewed according to the browser timezone (or UTC).
13
+
Device logs on the balenaCloud dashboard contains anything written to stdout and stderr of app services, and the system logs of the device. The dashboard allows logs to be cleared, filtered, searched and viewed according to the browser timezone (or UTC).
14
14
15
15
The maximum limit of logs that can be displayed on the dashboard is 1000 lines. That is also the amount of logs stored in the API, and available for download. Device logging in balenaCloud isn't meant for long-term, reliable storage of logs. It's instead designed to provide the latest logs from the device for debugging purposes. We do plan on expanding our logging solution to offer long-term storage and search.
16
16
17
17
## Persistent logging
18
18
19
-
The ability to read logs from the different system services running in balenaOS is vital in tracking issues. However, on reboot these logs will be cleared. To alleviate this, balenaOS allows persistent logs to be enabled by customers. This provides insight into the cause of reboots and helps make debugging easier.
19
+
The ability to read logs from the different system services running in balenaOS is vital in tracking issues. However, on reboot these journal logs are cleared and so examining them will not, for example, give any insight as to why the reboot may have occurred (or which services may have failed causing a reboot).
20
+
To alleviate this, balenaOS allows persistent journals (logs). This provides vital insight into when checking why a reboot occured and helps make debugging easier.
20
21
21
-
Persistent logging can be enabled using the Configuration tab on the sidebar for either a specific device or fleet wide. Select 'Activate' to enable persistent logging on your device/fleet. The device(s) will reboot once persistent logging is activated to ensure that the settings are applied. Once enabled, the logs are stored in `/var/log/journal/<uuid>` where the UUID is the device UUID.
22
+
Persistent logging can be enabled using the Configuration tab on the sidebar for either a specific device or fleet wide. Select 'Activate' to enable persistent logging on a specific device or on all devices in a fleet.
23
+
Since logs are stored in the data partition of the hostOS, the device(s) will reboot once persistent logging is activated to ensure that the settings are applied.
24
+
Once persistent logging is enabled, the logs are stored as part of the data partition on the device (either on SD card, eMMC, harddisk, etc.). This is located on-device at `/var/log/journal/<uuid>` where the UUID is device UUID.
25
+
26
+
Journals can be read like those for any unit file, using journalctl, although the switches passed to the command are slightly different. Here's an example of how to read persistent journals:
27
+
```
28
+
root@dee2945:~# cd /var/log/journal/b9ccd869194e4f1381c06967f99b0265/
29
+
root@dee2945:/var/log/journal/b9ccd869194e4f1381c06967f99b0265# ls -l
30
+
total 2051
31
+
-rw-r----- 1 root root 1048576 Jan 13 11:05 system.journal
32
+
-rw-r----- 1 root root 1048576 Jan 13 11:05 system@2ad94f188fb64c2da9803557662b57b2-0000000000000001-00058b3468ac9625.journal
33
+
root@dee2945:/var/log/journal/b9ccd869194e4f1381c06967f99b0265# journalctl -a --file system.journal
34
+
-- Logs begin at Mon 2020-01-13 11:05:06 UTC, end at Mon 2020-01-13 11:05:37 UTC. --
35
+
Jan 13 11:05:06 dee2945 systemd-journald[490]: Time spent on flushing to /var is 65.151ms for 795 entries.
36
+
Jan 13 11:05:06 dee2945 systemd-journald[490]: System journal (/var/log/journal/b9ccd869194e4f1381c06967f99b0265) is 2.0M, max 8.0M, 5.9M free.
37
+
Jan 13 11:05:07 dee2945 systemd[1]: Started Resin persistent logs.
38
+
Jan 13 11:05:07 dee2945 resin-persistent-logs[670]: resin-persistent-logs: Persistent logging activated.
39
+
Jan 13 11:05:06 dee2945 kernel[664]: [ 14.553592] systemd-journald[490]: Received request to flush runtime journal from PID 1
40
+
Jan 13 11:05:07 dee2945 systemd[1]: Started Modem Manager.
41
+
Jan 13 11:05:07 dee2945 NetworkManager[740]: <info> [1578913507.2694] NetworkManager (version 1.18.0) is starting... (for the first time)
Depending on the OS version, the size of persistent logs can also be increased to store more logs than the default size (32 MB currently). Do keep in mind persistent logging increases the wear on the storage medium due to increased writes. Refer to [long term storage of device logs](#long-term-device-logs-storage) for ways to offset this.
0 commit comments