The Linux Foundation

 
Testing

From The Linux Foundation

(Difference between revisions)
(Reorganization to reflect up-to-date situation)
(LSB Testing: LSB ATK Manager -> Linux Application Checker)
 
(3 intermediate revisions by one user not shown)
Line 1: Line 1:
 +
__NOTOC__
 
== LSB Testing ==
 
== LSB Testing ==
  
Even though Linux is developed in a highly decentralized manner, in order to be attractive to the ISV community, Linux must provide the same long-term compatibility guarantees and comprehensive compatibility testing as proprietary platforms such as Microsoft Windows. That's why the Linux Foundation and the [http://ispras.linux-foundation.org/index.php/ISP_RAS_Company_Profile Institute for System Programming of the Russian Academy of Sciences] are jointly putting huge resources for developing new tools and technologies to break through Linux Standard Base (LSB) testing challenges. These tools and technologies are collectively known as ''LSB Testing Framework'', which includes the following components:
+
Even though Linux is developed in a highly decentralized manner, in order to be attractive to the Independent Software Vendors (ISV) community, Linux must provide the same long-term compatibility guarantees and comprehensive compatibility testing as proprietary platforms such as Microsoft Windows. That's why the Linux Foundation and the [http://ispras.linux-foundation.org/index.php/ISP_RAS_Company_Profile Institute for System Programming of the Russian Academy of Sciences] are jointly putting huge resources for developing new tools and technologies to break through Linux Standard Base (LSB) testing challenges. These tools and technologies are collectively known as ''LSB Testing Framework'', which includes the following components:
 
* '''Test Execution Frameworks''' to ease test execution and results analysis:
 
* '''Test Execution Frameworks''' to ease test execution and results analysis:
** [http://ispras.linux-foundation.org/index.php/About_LSB_Application_Testkit_Manager Application Testkit (ATK) Manager] - a web-based tool for checking applications for LSB compliance and analyzing external interfaces required by applications to run correctly.
+
** [http://ispras.linuxfoundation.org/index.php/About_Linux_Application_Checker Linux Application Checker] - a web-based tool for checking applications for LSB compliance and analyzing external interfaces required by applications to run correctly.
 
** [http://ispras.linux-foundation.org/index.php/About_LSB_Distribution_Testkit_Manager Distribution Testkit (DTK) Manager] - a web-based tool for testing Linux distributions - selecting which tests to run, execute them and finally analyze colorful HTML reports.
 
** [http://ispras.linux-foundation.org/index.php/About_LSB_Distribution_Testkit_Manager Distribution Testkit (DTK) Manager] - a web-based tool for testing Linux distributions - selecting which tests to run, execute them and finally analyze colorful HTML reports.
 
* '''Test Development Frameworks''' to ease development of new tests in different value & cost segments:
 
* '''Test Development Frameworks''' to ease development of new tests in different value & cost segments:
Line 14: Line 15:
 
== LSB Navigator ==
 
== LSB Navigator ==
  
The Testing Framework is leveraged by [http://linux-foundation.org/navigator/ LSB Navigator] - a web-portal for LSB community, which interlinks the various moving parts that make up the Linux platform to an unprecedented degree. This provides upstream package developers and downstream distribution vendors with a powerful set of tools for coordinating their work and improving the quality of the platform, as well as giving Independent Software Vendors (ISVs) a more effective way to provide feedback to both parties.  
+
The Testing Framework is leveraged by [http://linux-foundation.org/navigator/ LSB Navigator] - a web-portal for LSB community, which interlinks the various moving parts that make up the Linux platform to an unprecedented degree. This provides upstream package developers and downstream distribution vendors with a powerful set of tools for coordinating their work and improving the quality of the platform, as well as giving ISVs a more effective way to provide feedback to both parties.  
  
 
One of the most challenging aspects of maintaining a standard for the Linux community is tracking all the moving parts. A typical Linux distribution is made up of hundreds of independent “upstream” open source components, each of which evolves at its own pace. The LSB tracks the independently evolving Linux distributions to guarantee cross-distribution portability, which adds a third dimension to an already complex task. This coordination is crucial to ensure the Linux platform’s long term success and LSB Navigator is to play an important role here.
 
One of the most challenging aspects of maintaining a standard for the Linux community is tracking all the moving parts. A typical Linux distribution is made up of hundreds of independent “upstream” open source components, each of which evolves at its own pace. The LSB tracks the independently evolving Linux distributions to guarantee cross-distribution portability, which adds a third dimension to an already complex task. This coordination is crucial to ensure the Linux platform’s long term success and LSB Navigator is to play an important role here.
Line 20: Line 21:
 
== LSB Infrastructure Program ==
 
== LSB Infrastructure Program ==
  
LSB Testing Framework and LSB Navigator are developed under '''[http://ispras.linux-foundation.org/ LSB Infrastructure Program]''' jointly run by the Linux Foundation and the Institute for System Programming of the Russian Academy of Sciences (ISP RAS). This multi-million dollar cooperation takes advantage of hundreds of man years of existing ISP RAS developments and plans for dozens of man years to be done. The Russian Academy of Sciences has a wealth of knowledge and experience in developing testing frameworks and informational systems, including work with HP, Intel, Microsoft, Nortel and others, and the development of the Open Linux Verification Project at the Linux Verification Center ([http://linuxtesting.org http://linuxtesting.org]).
+
LSB Testing Framework and LSB Navigator are developed under [http://ispras.linux-foundation.org/ LSB Infrastructure Program] jointly run by the Linux Foundation and the Institute for System Programming of the Russian Academy of Sciences (ISP RAS). This multi-million dollar cooperation takes advantage of hundreds of man years of existing ISP RAS developments and plans for dozens of man years to be added. The Russian Academy of Sciences has a wealth of knowledge and experience in developing testing frameworks and informational systems, including work with HP, Intel, Microsoft, Nortel and others, and the development of the Open Linux Verification Project at the Linux Verification Center ([http://linuxtesting.org http://linuxtesting.org]).
  
LSB Infrastructure Program includes the following areas:
+
The Testing Framewok and LSB Navigator are just selected parts of the LSB Infrastructure Program. The full program includes the following areas:
* Infrastructure Software Systems:
+
* Development of new Infrastructure Software Systems:
 
** Central [http://ispras.linux-foundation.org/index.php/LSB_Database_Home LSB Database] & [http://ispras.linux-foundation.org/index.php/LSB_DB_Tools Scripts]
 
** Central [http://ispras.linux-foundation.org/index.php/LSB_Database_Home LSB Database] & [http://ispras.linux-foundation.org/index.php/LSB_DB_Tools Scripts]
 
** [http://ispras.linux-foundation.org/index.php/LSB_DB_Navigator LSB Navigator]
 
** [http://ispras.linux-foundation.org/index.php/LSB_DB_Navigator LSB Navigator]
 
** LSB Certification System
 
** LSB Certification System
* Testing Frameworks and New Tests:
+
* Development of new Testing Frameworks and Tests:
** Test Execution Frameworks ([http://ispras.linux-foundation.org/index.php/About_LSB_Application_Testkit_Manager ATK] and [http://ispras.linux-foundation.org/index.php/About_LSB_Distribution_Testkit_Manager DTK] Managers)
+
** Test Execution Frameworks ([http://ispras.linux-foundation.org/index.php/About_LSB_Application_Testkit_Manager ATK] and [http://ispras.linux-foundation.org/index.php/About_LSB_Distribution_Testkit_Manager DTK] Managers).
 
** 3 Test Development Frameworks for 3 grades of value&cost – shallow, normal, deep.
 
** 3 Test Development Frameworks for 3 grades of value&cost – shallow, normal, deep.
** New Tests for LSB Interfaces (with corresponding quality depending on the interface "importance")
+
** New Tests for LSB Interfaces (with corresponding quality depending on the interface "importance").
 
* LSB Analytical and Promotional Tasks
 
* LSB Analytical and Promotional Tasks
  
All results of LSB Infrastructure program will be free for anyone to use under the Gnu Public License (GPL).
+
All results of LSB Infrastructure program are free for anyone to use under the Gnu Public License (GPL).

Latest revision as of 04:14, 25 September 2009

[edit] LSB Testing

Even though Linux is developed in a highly decentralized manner, in order to be attractive to the Independent Software Vendors (ISV) community, Linux must provide the same long-term compatibility guarantees and comprehensive compatibility testing as proprietary platforms such as Microsoft Windows. That's why the Linux Foundation and the Institute for System Programming of the Russian Academy of Sciences are jointly putting huge resources for developing new tools and technologies to break through Linux Standard Base (LSB) testing challenges. These tools and technologies are collectively known as LSB Testing Framework, which includes the following components:

  • Test Execution Frameworks to ease test execution and results analysis:
    • Linux Application Checker - a web-based tool for checking applications for LSB compliance and analyzing external interfaces required by applications to run correctly.
    • Distribution Testkit (DTK) Manager - a web-based tool for testing Linux distributions - selecting which tests to run, execute them and finally analyze colorful HTML reports.
  • Test Development Frameworks to ease development of new tests in different value & cost segments:
    • AZOV Shallow Test Development Framework - allows semi-automatic construction of shallow tests by using information about interfaces from the central LSB Database (low value & low cost).
    • T2C Normal Test Development Framework - allows automated development of medium quality tests (medium value & medium cost).
    • UniTESK Deep Test Development Framework - allows automated development of sophisticated deep tests for critical interfaces (high value & high cost).

Tests produced under T2C and UniTESK frameworks include explicit linkage of test failures to specific requirements of the source standard. So when a test fails it says which specific assertion from the text of the standard is violated and provides details about specific inconsistencies. This gives unprecedented comfort for test users.

[edit] LSB Navigator

The Testing Framework is leveraged by LSB Navigator - a web-portal for LSB community, which interlinks the various moving parts that make up the Linux platform to an unprecedented degree. This provides upstream package developers and downstream distribution vendors with a powerful set of tools for coordinating their work and improving the quality of the platform, as well as giving ISVs a more effective way to provide feedback to both parties.

One of the most challenging aspects of maintaining a standard for the Linux community is tracking all the moving parts. A typical Linux distribution is made up of hundreds of independent “upstream” open source components, each of which evolves at its own pace. The LSB tracks the independently evolving Linux distributions to guarantee cross-distribution portability, which adds a third dimension to an already complex task. This coordination is crucial to ensure the Linux platform’s long term success and LSB Navigator is to play an important role here.

[edit] LSB Infrastructure Program

LSB Testing Framework and LSB Navigator are developed under LSB Infrastructure Program jointly run by the Linux Foundation and the Institute for System Programming of the Russian Academy of Sciences (ISP RAS). This multi-million dollar cooperation takes advantage of hundreds of man years of existing ISP RAS developments and plans for dozens of man years to be added. The Russian Academy of Sciences has a wealth of knowledge and experience in developing testing frameworks and informational systems, including work with HP, Intel, Microsoft, Nortel and others, and the development of the Open Linux Verification Project at the Linux Verification Center (http://linuxtesting.org).

The Testing Framewok and LSB Navigator are just selected parts of the LSB Infrastructure Program. The full program includes the following areas:

  • Development of new Infrastructure Software Systems:
  • Development of new Testing Frameworks and Tests:
    • Test Execution Frameworks (ATK and DTK Managers).
    • 3 Test Development Frameworks for 3 grades of value&cost – shallow, normal, deep.
    • New Tests for LSB Interfaces (with corresponding quality depending on the interface "importance").
  • LSB Analytical and Promotional Tasks

All results of LSB Infrastructure program are free for anyone to use under the Gnu Public License (GPL).


[Article] [Discussion] [View source] [History]