The Linux Foundation

 
LSB F2F November 2007 Testing Discussion

From The Linux Foundation

Q1: Precise definition of test coverage measurement method

Which interfaces have to be covered by run-time tests and which not?

We agreed that the following interfaces does not require to be separetly targeted by run-time tests:

  • data interfaces;
  • C++ thunks;
  • C++ guard variables;
  • C++ typeinfo;
  • C++ private methods.

Correspondingly the interfaces, which have to be covered by run-time tests includes:

  • functional interfaces;
  • C++ public and protected methods.

Q2: Processing of bugs detected by newly developed tests

What shall we do, when newly developed tests detect inconsistencies between requirements of the standard and implementation?

The policy of processing such problems agreed is as follows.

Each problem is considered separately. If the problem is a consequence of incorrect specification we just fix it.

If it is an important bugs in implementation we

  • require all distributions to be certified have to fix it;
  • push already certified distributions to fix it;
  • add a link to description of the problem as a comment at the interface homepage of the LSB Navigator.

For minor problems in implementation we

  • report it to upstream component developers;
  • push them to fix it;
  • grant the corresponding waiver for the current certification program;
  • add a link to description of the problem as a comment at the interface homepage of the LSB Navigator.

Q3: Fixing of problems in frozen version of documentation referenced by the standard

How to incorporate the correct changes in documentation to its frozen version referenced by the standard?

All problems detected have to be reported to the upstream component developers as well as to the LSB Bugzilla. They will be fixed in documentation, but also they will be fixed in LSB by adding correct description of the corresponding interface to the LSB itself. After uplifting version of the upstream component the description in LSB will be removed if the new version of documentation have the problem fixed.


[Article] [Discussion] [View source] [History]