Skip to main content

​
View logs

Kernel messages can be retrieved with talosctl dmesg command:
Service logs can be retrieved with talosctl logs command:
Kernel log is also mirrored as talosctl logs kernel. Container logs for Kubernetes pods can be retrieved with talosctl logs -k command: If some host workloads (e.g. system extensions) send syslog messages, they can be retrieved with talosctl logs syslogd command.

​
Forward logs for aggregation

Talos writes logs to files in /var/log directory. A pod running in Kubernetes can mount this directory and forward logs to a log aggregation system.

​
Send logs over network

​
Service logs

You can enable log sending in machine configuration:
Several destinations can be specified. Supported protocols are UDP and TCP. The only currently supported format is json_lines:
Messages are newline-separated when sent over TCP. Over UDP messages are sent with one message per packet. msg, talos-level, talos-service, and talos-time fields are always present; there may be additional fields. Every message sent can be enhanced with additional fields by using the extraTags field in the machine configuration:
The specified extraTags are added to every message sent to the destination verbatim.
syslog is considered a service in Talos, and so messages/logs sent to syslog (e.g., by system extensions) are considered service logs and will be sent to any configured remote receivers without further configuration.

​
Kernel logs

Kernel log delivery is configured with the KmsgLogConfig document in the machine configuration:
It can also be enabled with the talos.logging.kernel=tcp://host:5044/ kernel command line argument, which is useful to capture logs from the early boot: add it to the boot asset with Image Factory or pass it on the command line of the boot loader. Kernel log destination is specified in the same way as service log endpoint. The only supported format is json_lines. Sample message:
KmsgLogConfig is applied to a running node, while a change of the kernel command line only takes effect on the next boot with the updated boot asset.

​
Receive logs

If you have configured remote service logs or kernel logs on a Talos system and want to collect the logs centrally for debugging purposes you can temporarily run the netcat nc command to receive logs. On a Linux host run the following command:
This will print the logs to standard out and also save them to a logs.txt file. On the Talos machines make sure to configure logs to send to your machine’s IP address with a logging configuration as described above. Central logging configuration will allow you to collect logs even when the Talos API is not available (e.g. during installation).