Ftrace is a tracer that is internal to the kernel. Tracing can help reveal important details about a system's execution and so it can be very useful when debugging latencies in a real-time system.

This page gives a brief overview of how to configure and use Ftrace.

More detailed information about Ftrace can be obtained via the following sources:

• Ftrace kernel documentation
• tracefs documentation: Setup, file interface explanation
• README at the root of tracefs
• LWN Articles: Debugging with Ftrace - part 1, Debugging with Ftrace - part 2, Ftrace function tracer

Not all of the following setup steps will be necessary on all systems. Certain steps have been included for the sake of completeness.

If the kernel is already configured correctly for using Ftrace, then tracefs, the filesystem used by Ftrace to communicate with userspace, will be listed in /proc/filesystems as one of the available filesystems. If tracefs is available, then the following command:

$ cat /proc/filesystems | grep tracefs

should return something like:

nodev    tracefs

If the tracefs filesystem is not available, the configuration of the kernel will need to be changed.

To include tracefs, at least one of the Ftrace configuration options must be selected. This could be a configuration option with FTRACE in the name, such as CONFIG_FTRACE or CONFIG_DYNAMIC_FTRACE. It could also be a configuration option that enables a specific tracer, such as CONFIG_FUNCTION_TRACER or CONFIG_STACK_TRACER. If Ftrace is going to be used for debugging, then several configuration options will probably need to be enabled.

When configuring the kernel, it is also important to select the configuration options that will include the desired tracers in the kernel. There is one configuration option per tracer. For example, if the function tracer will be used then the CONFIG_FUNCTION_TRACER option should be selected.

The exact configuration options required for different tracing setups are outside the scope of this page. However, when configuring the kernel with make menuconfig the option dependencies are clearly indicated and each configuration option has a description.

After modifying the kernel configuration, the kernel can be built and installed on the system.

Before any Ftrace tracers can be used, the tracefs file system must be mounted. If the file system is already mounted, then the mount location can be displayed using the following command:

$ mount | grep tracefs

The tracefs file system is usually mounted at /sys/kernel/tracing, but this is not necessarily the case for all systems. If tracefs is not mounted and the previous command did not display anything, then the file system will need to be manually mounted with this command:

$ mount -t tracefs nodev /sys/kernel/tracing

Ftrace can collect information using several different kinds of instrumentation and tracers. The tracers that can be used by Ftrace depend on the configuration used to build the kernel. Before proceeding, it is a good idea to check that the kernel has been configured correctly and all the desired tracers have been included in the kernel.

The '[tracing]' before the following commands is to indicate that they are being run from the root of tracefs (the tracing folder). The '#' symbol is used to indicate that the commands must be run as root or with sudo. The available tracers are listed in the available_tracers file in tracefs:

[tracing]# cat available_tracers

If the desired tracers are not all listed, then the kernel configuration will need to be modified.

If a tracer is available, it can be activated manually via tracefs. Here is an example of how to select the function tracer:

[tracing]# echo function > current_tracer

By default, tracing is already enabled when the system starts. So, if the user has not disabled tracing since booting they just need to select a tracer (other than “nop”) to start collecting data. Tracing can also be enabled by writing 1 to the tracing_on file or disabled by writing a 0 to the tracing_on file. The files current_tracer and tracing_on can be read to verify that the tracer was selected correctly and that tracing is enabled.

After the situation that was to be traced has occurred, tracing can be disabled, and the trace results can be viewed in the trace file:

[tracing]# cat trace

If another trace is to be taken, the previous results can be cleared by writing into the trace file:

[tracing]# echo > trace