The Linux Foundation

 
LSB Distribution Testkit Getting Started

From The Linux Foundation

This document describes the general steps on getting started with the LSB Distribution Testkit (LSB DTK).

Contents

Installation

  1. Download the LSB Distribution Testkit package (tar.gz) for your architecture from the Downloads page.
  2. Unpack the package:
     tar xzf lsb-dtk-<version>.<architecture>.tar.gz 
  3. Go to the subdirectory ./lsb-dtk created when unpacking, run the installation script:
     ./install.sh 
    and follow the instructions. You may need to enter the root password.
  4. Run the LSB DTK Manager:
    • Run the script:
       /opt/lsb/test/manager/bin/lsb-dtk-start.sh [port-number] 
      The port-number argument is optional (by default 8888 is taken). This script will start a simple web server embedded in the LSB DTK Manager and will try to locate and open your web-browser with the proper address to this server.
    • Note: If the script could not find your web-browser and you wish to use Web-UI, please start the browser manually and specify the address http://localhost:8888/ (the port number should be the same as specified on the previous step).
    • Note: You may also connect to the LSB DTK Manager from any remote computer by specifying the URL http://<test-machine-name>:8888/ in your web-browser.

Using Web-UI

  1. In the browser you will see the main page of the LSB DTK Manager. It allows you to choose whether to perform certification tests or custom set of tests.
  2. Depending on your choice either Get Certified or Custom Tests page will be opened.
    • On the Get Certified page you can see the following options:
      Error creating thumbnail: File seems to be missing: /var/www/mediawiki/images/5/59/LDTKM_Certify.png

      You should specify your name, organization and (if you wish) E-mail for sending the report. Also you may change the architecture, LSB version and some other settings for the tests. Then just press the Run the Tests button.
    • The Custom Tests page contains the extended set of options:
      Error creating thumbnail: File seems to be missing: /var/www/mediawiki/images/c/ca/LDTKM_CustomTests.png

      At the first time, the list contains the most essential test suites only. You can press the Refresh List button to update it from the LF's FTP server.
      Specify necessary information in the edit fields, select the test suites you wish to execute (check boxes to the left), specify the source for every test suite selected. Localhost, FTP or FTP (beta) stand for the packages already downloaded and/or installed, located on the LF's FTP server or in the beta subdirectory of the LF's FTP server, respectively. See the detailed description on the Help page of the LSB DTK Manager.
      After that press the Run Selected Tests button at the bottom of the page.
    • Note: There are some prerequisites for the test suites to be able to run (listed in the /opt/lsb/test/manager/README file). If they are not met, an error message will be shown at the top of the page when you try to run the tests. In this case you will have to eliminate the problems and run the tests again.
  3. You will be redirected to the next page — Execution:
    Error creating thumbnail: File seems to be missing: /var/www/mediawiki/images/1/12/LDTKM_Execution.png

    On this page, you will see the general status of the execution and the terminal emulation that shows on-the-fly output of the tests.
  4. After the automated tests are finished you are redirected to the Summary Report page. It contains just the general information about the status of the automatic and manual tests:
    Error creating thumbnail: File seems to be missing: /var/www/mediawiki/images/8/83/LDTKM_SummaryReport.png

    If the manual tests need to be performed, a link is provided to the Manual Tests page. Depending on the tests selected and their status some additional information is shown on this page: what to do with failing tests, how to apply for certification, where to find the journal files and how to return back to this page after closing it. Also you can open the detailed report by clicking the corresponding link.
    For a certification test run this page contains the Apply for certification link which redirects you to the Certification System where you will be able to use the test results for certifying your distribution.
    For custom test run you will see the Upload the test results link instead. You cannot start certification with these results, but you can use them as additional data for the product already being certified.
  5. The Detailed Report page contains the exhaustive information about every test suite. Any tests you have failed that have already been waived will be marked as such in the report.
    Error creating thumbnail: File seems to be missing: /var/www/mediawiki/images/d/d9/LDTKM_DetailedReport.png
  6. On the Manual Tests page you can perform the tests which cannot be automated.
    Error creating thumbnail: File seems to be missing: /var/www/mediawiki/images/2/24/LDTKM_ManualTests.png

    You need to go through each of the tests shown in the left table (that is for each test the result should become either Passed or Failed). Then just press the Finish button and you will be redirected to the updated Summary Report page with new status of the manual test.
    Note: You do not need to perform all the manual tests at once. You can go through just some of them (or even none), close the browser and return to these tests later. The link for continuing the manual tests can be found on the Results page (see below).
  7. The list of all the reports generated for every test run is shown on the Results page:
    Error creating thumbnail: File seems to be missing: /var/www/mediawiki/images/6/65/LDTKM_Results.png

    To view any particular summary report, just click the corresponding link in the Date/Time column. Additional icons in the right bottom corner allow you to quickly open the Summary Report page, Detailed Report page or, if you have not yet finished with the manual tests, to continue with them. To remove some of the reports, just select them with the check boxes and press the Remove Selected Entries button.

Using Command Line UI

Instead of using the web-based interface, you may use the command-line interface provided by the LSB DTK extensions for lsb-autotest.

  1. Go to the directory /opt/lsb/test/manager/autotest-ext.
    • To run the certification tests, execute the command (with root privileges):
       # ./lsb-tef.pl --cert 
    • For saving the whole console log the `tee' utility can be used:
       # ./lsb-tef.pl --cert | tee /tmp/tests_output.8888 
      If /tmp/tests_output.8888 file is used for saving, then this log file will be automatically copied to the result directory after the tests are finished.
    • To run all installed tests, execute the command (with root privileges):
       # ./lsb-tef.pl all 
    • To run the custom set of tests, execute the command:
       # ./lsb-tef.pl test1 test2 ... 
      test1, test2 etc. are the specific test suites you wish to run. To obtain the list of supported test suites, use ./lsb-tef.pl --help.
  2. In addition, the following options may be useful:
    • -D: Automatically download the test packages needed.
    • --lsb=<LSB-version>: Allows to explicitly specify the LSB version to test against.
    • -T<profile>: Allows to specify the LSB profile to test against. Possible values are: core,c++ and core,c++,desktop (default).
    • -r <results-dir>: Specify the directory for the results.
    • -v[0-2]: Specify verbose mode.
  3. You can use the Command Line UI for nightly test runs. There is a detailed instruction on this topic.
  4. Please, see
     ./lsb-tef.pl --help 
    for the full list of the options available in the current version. All the options are also described here: LSB DTK Manager Command Line Options.

Test Results Repository

The results for test runs are stored in the directory /var/opt/lsb/test/manager/results. All the files from each test run are stored in separate subdirectory named <arch>-<machine>-<date>-<time>. For the result to be visible on the Results page of the Web-UI it should be listed in the file /var/opt/lsb/test/manager/results/HISTORY.

If you are using Web-UI or Command Line UI for running the tests, the HISTORY file is managed automatically. If you copy the test results from another machine or another directory by hand, you will need to write the directory names into this file manually.

Finalizing

  1. To stop the web-server, you may either run the script
     /opt/lsb/test/manager/bin/lsb-dtk-stop.sh 
    or go to the Administration page in the web-interface and press the Stop server button.
  2. You can uninstall all packages installed by LSB DTK Manager using the command
     /opt/lsb/test/manager/autotest-ext/lsb-tef.pl --cleanup all 
  3. If you wish to deinstall the LSB DTK Manager at all, use your package manager (rpm or dpkg) to remove the package lsb-dtk-manager. After that you may also wish to delete the package cache, old results and other data from the /var/opt/lsb/test/manager directory.

[Article] [Discussion] [View source] [History]