Table of Contents

RTEval

Rteval is a python program written to evaluate the performance of a realtime Linux kernel on a particular hardware platform. The program unpacks source code for two loads: hackbench and a Linux kernel compile, then loops running hackbench and a parallel build of the Linux kernel. While the loads are running, the cyclictest program is run to measure realtime performance under load. When the specified run duration is met, the loads are stopped and cyclictest outputs measured timer latency values in histogram format, which is analyzed by rteval. Rteval then writes an XML file to disk with information about the system (gotten through DMI tables), the raw histogram data collected during the run and the statistical analysis of the run. (Reference: https://git.kernel.org/pub/scm/utils/rteval/rteval.git/tree/README)

More information about rteval: https://git.kernel.org/pub/scm/utils/rteval/rteval.git/tree/doc/rteval.txt

Installation

rteval requires Python3 with the following dependencies:

Most of the above Python3 dependencies should be available in package management systems for most operating systems.

The user must manually download and compile rt-tests. Compiling rt-tests additionally requires glibc and libnuma headers.

Following are quick instructions to install and run once all the dependencies have been installed:

$> git clone https://git.kernel.org/pub/scm/utils/rteval/rteval.git
$> cd rteval
# The following kernel will be used by the kcompile workload. A different kernel version can also
# used here. But, corresponding changes will have to be made in KLOAD in Makefile and kernel_prefix
# in ./rteval/kcompile.py before installation.
$> wget -P loadsource https://www.kernel.org/pub/linux/kernel/v6.x/linux-6.6.1.tar.xz
$> sudo make install
$> sudo make runit

More information at: https://git.kernel.org/pub/scm/utils/rteval/rteval.git/tree/README

Results

Following is a sample result for the command:

Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance.

  ===================================================================                                                                                                                                     [86/1801]
   rteval (v1.38) report
  -------------------------------------------------------------------
   Test run:     2017-02-13 12:49:21
   Run time:     0 days 0h 0m 11s


   Tested node:  vpatel-dev2
   Model:         - 
   BIOS version:  (ver: , rev :, release date: )

   CPU cores:    8 (online: 8)
   NUMA Nodes:   1
   Memory:       31.292 GB
   Kernel:       4.4.2
   Base OS:      unknown
   Architecture: x86_64
   Clocksource:  tsc
   Available:    tsc hpet acpi_pm

   Load commands:
       Load average: 4.07
       Commands:
         - kcompile: make -C /home/vpatel/anon_repos/rteval/run/rteval-build/linux-2.6.39 -j16
         - hackbench: hackbench -P -g 16.0 -l 100 -s 100

   Latency test
      Command:cyclictest -qm -i 100 -d 0 -h 2000 -p 95 --smp

      System:  (8 cores) Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
      Statistics: 
          Samples:           869190
          Mean:              2.12836433921us
          Median:            2us
          Mode:              2us
          Range:             731us
          Min:               0us
          Max:               731us
          Mean Absolute Dev: 0.937310248954us
          Variance:          35.8090162273us
          Std.dev:           5.98406352133us

      CPU core 0   Priority: 95
      Statistics: 
          Samples:           109626
          Mean:              2.61308448726us
          Median:            2us
          Mode:              2us
          Range:             645us
          Min:               0us
          Max:               645us
          Mean Absolute Dev: 1.67699461734us
          Variance:          90.2609495244us
          Std.dev:           9.50057627328us
          
      CPU core 1   Priority: 95
      Statistics: 
          Samples:           109870
          Mean:              1.86766178211us
          Median:            1us
          Mode:              1us
          Range:             710us
          Min:               1us
          Max:               711us
          Mean Absolute Dev: 0.921409903422us
          Variance:          34.2588155195us
          Std.dev:           5.85310306756us

      CPU core 2   Priority: 95
      Statistics: 
          Samples:           109463
          Mean:              2.37517700045us
          Median:            2us
          Mode:              2us
          Range:             621us
          Min:               0us
          Max:               621us
          Mean Absolute Dev: 0.944711592396us
          Variance:          31.7869418697us
          Std.dev:           5.63799094268us

      CPU core 3   Priority: 95
      Statistics: 
          Samples:           107860
          Mean:              2.1190246616us
          Median:            2us
          Mode:              2us
          Range:             652us
          Min:               0us
          Max:               652us
          Mean Absolute Dev: 0.876297487018us
          Variance:          27.0425459294us
          Std.dev:           5.20024479514us

      CPU core 4   Priority: 95
      Statistics: 
          Samples:           108567
          Mean:              2.1447309035us
          Median:            2us
          Mode:              2us
          Range:             684us
          Min:               0us
          Max:               684us
          Mean Absolute Dev: 0.879960259054us
          Variance:          29.0171586253us
          Std.dev:           5.38675770991us

      CPU core 5   Priority: 95
      Statistics: 
          Samples:           108366
          Mean:              1.66483029733us
          Median:            1us
          Mode:              1us
          Range:             571us
          Min:               0us
          Max:               571us
          Mean Absolute Dev: 0.755909668094us
          Variance:          6.99546256844us
          Std.dev:           2.6448936781us

      CPU core 6   Priority: 95
      Statistics: 
          Samples:           107590
          Mean:              2.14418626266us
          Median:            2us
          Mode:              2us
          Range:             731us
          Min:               0us
          Max:               731us
          Mean Absolute Dev: 0.797404828348us
          Variance:          37.2396828533us
          Std.dev:           6.10243253575us

      CPU core 7   Priority: 95
      Statistics: 
          Samples:           107848
          Mean:              2.0935761442us
          Median:            2us
          Mode:              2us
          Range:             693us
          Min:               0us
          Max:               693us
          Mean Absolute Dev: 0.844485575396us
          Variance:          28.6649385662us
          Std.dev:           5.35396475205us

  ===================================================================
rteval: exiting with exit code: 0

Most of the information is self explanatory. The statistics printed in the above example is just the output from cyclictest.