Differences

This shows you the differences between two versions of the page.

--- realtime:documentation:howto:tools:hwlat [2018/07/11 14:52]
anna-maria created
+++ realtime:documentation:howto:tools:hwlat [2018/07/13 10:59]
ebugden [Analysis] Improve todo phrasing
@@ Line 1: / Line 1: @@
-====== HWLAT ======
+====== Ftrace - hwlat tracer ======
+hwlat is an Ftrace tracer designed to detect latencies caused by a system's firmware or hardware. The tracer was originally intended for [[realtime:documentation:howto:debugging:smi-latency:start|detecting SMIs]], but it can detect other low level latencies as well. More details about this tracer can be found in the kernel documentation for Ftrace and for the hardware latency detector.
+===== Setup =====
+==== Configuring the kernel ====
+If the kernel is already configured correctly and the hwlat tracer is available, it will be listed in the available_tracers file in tracefs. Otherwise, the hwlat tracer can be included in the kernel by enabling the CONFIG_HWLAT_TRACER option.
+==== Mounting tracefs ====
+Before any Ftrace tracers can be used, the tracefs file system must be mounted. If the file system is already mounted, then the mount location can be displayed using the following command:
+<code bash>
+$ mount | grep tracefs
+</code>
+The tracefs file system is usually mounted at /sys/kernel/tracing, but this is not necessarily the case for all systems. If tracefs is not mounted and the previous command did not display anything, then the file system will need to be manually mounted with this command:
+<code bash>
+$ mount -t tracefs nodev /sys/kernel/tracing
+</code>
+===== Tracing =====
+If the tracer is available, hwlat can be activated manually via tracefs in the same way as any other Ftrace tracer. The '[tracing]' before the following commands is to indicate that they are being run from the root of tracefs (the tracing folder). The '#' symbol is used to indicate that the commands must be run as root or with sudo. Here is an example of how to select the hwlat tracer:
+<code>
+[tracing]# echo hwlat > current_tracer
+</code>
+If this is the first time that the user selects a tracer using the current_tracer file since the system was booted, then tracing will automatically be enabled afterwords. Tracing can also be enabled by writing 1 to the tracing_on file or disabled by writing a 0 to the tracing_on file. The files current_tracer and tracing_on can be read to verify that the tracer was selected correctly and that tracing is enabled.
+Before consulting the trace results for analysis, make sure that only one instance of hwlat is running. The program //top// can be used for this. If more than one instance is running, restart the system, and trace the latencies again with only one instance of hwlat.
+<WRAP center round alert 60%>
+**Bug:** Do not write 1 to the tracing_on file when tracing is already enabled. This will start another instance of hwlat.
+</WRAP>
+Because of a bug, an additional instance of hwlat is started every time 1 is written to tracing_on regardless of whether or not tracing with hwlat is already enabled. When tracing is disabled, only one instance of hwlat will be stopped and the only way to stop the additional instances is to restart the system. Running multiple instances of hwlat can quickly cripple a system so it is best to avoid this.
+===== Results =====
+After the situation that was to be traced has occurred, tracing can be disabled, and the trace results can be viewed in the trace file:
+<code>
+[tracing]# cat trace
+</code>
+If another trace is to be taken, the previous trace results can be cleared by selecting the nop tracer:
+<code>
+[tracing]# echo nop > current_tracer
+</code>
+===== Analysis =====
+The kernel documentation for Ftrace and for the hardware latency detector contain details about how to interpret the resulting traces.
+<WRAP center round todo 60%>
+Explain how to read the results because the kernel documentation is confusing and/or improve kernel documentation?
+</WRAP>

Wiki

User Tools

Site Tools

Differences

Page Tools