- 1 AT-SPI LSB testing
AT-SPI LSB testing
AT-SPI is an interface for application accessibility. It is defined in CORBA IDL and has been translated into language-specific interfaces. More information about AT-SPI is available at: http://www.linuxfoundation.org/en/Accessibility/ATK/AT-SPI.
The LSB specification currently contains a single accessibility interface, ATK - This is the GNOME accessibility interface. ATK is fully implemented by the GTK toolkit, as well as others such as Windows forms on Mono. ATK is however not generally used by Assistive Technologies (ATs). ATs are the applications that use accessibility interfaces to provide extra functionality to people with disabilities. Thhe goal of LSB certification with regards to accessibility should be to ensure that they function correctly across distributions. As Linux Desktop ATs commonly use AT-SPI libraries, standardization of these would be most useful.
AT-SPI has a number of language-specific interfaces as well as multiple implementations. For example AT-SPI may be implemented by the ATK bridge for GTK applications or by the Java bridge for Swing based applications. Client libraries include Pyatspi for python based ATs and Cspi for 'C' language ATs. Testing correct functionality of AT-SPI will be required for each of its implementations. It would be possible to perform functional testing of each implementation separately using each client library. This would lead to N X M tests being performed, and would probably be wasteful.
LSB AT-SPI standardization should be for two client libraries: cspi and pyatspi. "Medium" LSB test should be performed for different AT-SPI implementations using the pyatspi library. "Shallow" LSB tests should be performed on cspi perhaps using a dummy AT-SPI implementation. Initially the AT-SPI implementations tested should be the Atk bridge, although this may be extended to QT and Java Swing.
The LSB specifications are sometimes based on upstream documentation. It is from these specifications that test requirements are derived. LSB specifications must be created for the AT-SPI client libraries cspi and pyatspi. Both of these interfaces are closely derived from the AT-SPI CORBA IDL definition. Because of this it should be possible to use the IDL definition to finish incomplete documentation of language-specific bindings.
Agree with LSB and Linux Accessibility community on LSB specifications for AT-SPI. This is a difficult question as there are multiple client interfaces that need to be standardized. The cspi library documentation is very complete, but pyatspi documentation is not. Work may be required to complete this documentation based on the IDL.
The "Medium" LSB tests require that test failures be linked back to the particular specification violation. This seems sensible, and is ok as long as the test requirements are closely derived from the specifications. An example of some requirement XML taken from the ATK test suite:
Gets the value of this object.
</req> <req id="app.atk_value_get_current_value.02">
obj: a GObject instance that implements AtkValueIface.
Generate test requirements from the pyatspi / IDL documentation. As only "Shallow" tests will be performed on cspi, such requirements may not be necessary.
ATs operate in a separate process space to the Applications they serve. AT-SPI uses an IPC mechanism to access the applications. To test an AT-SPI interface an application is required to test against. This application will provide Accessible objects, with a certain set of interfaces, that is backed by the AT-SPI implementation in question. Test scenarios will be the applications used to test the AT-SPI / pyatspi interfaces. These scenarios should be well documented. The reason for this is that the same tests will need to be implemented on multiple AT-SPI implementations. This means multiple test applications, based on different widget toolkits, that must have the same abstract structure exposed via AT-SPI.
Some example XML describing a test application.
<accessible name="Main" role="4" description="Test Application for interface Text"> <implements name="Accessible::Accessible/> <implements name="Accessible:Text"> <method name="get_character_count" result="24"/> </implements> </accessible>
Perhaps an XML description is overkill, but decent documentation of the test scenarios would go a long way to helping in the testing of new AT-SPI implementations.
Translate the test requirements into some test scenarios. This will involve creating abstract test applications that expose AT-SPI interfaces. The test scenarios should be chosen to each cover a small section of the AT-SPI interface. Its likely in reality that these scenarios would be created alongside an initial test implementation.
GTK AT-SPI Implementation
The first test should probably investigate the GTK implemenation of AT-SPI provided by GAIL and the Atk-bridge. Choice of test framework should be made to fit in with LSB, I believe that there are some TET bindings for python, so these would be a good choice for invoking the pyatspi interface.
Translate test scenarios into test applications written using the GTK toolkit.
Write pyatspi tests using the TET framework. These will need to load the correct test application for each tests. They will need to link failures back to the specification. Collaboration with the Institute for System Programming of the Russian Academy of Sciences (ISP RAS) will be necessary. They have extensive experience of working with the TET framework and developed the LSB desktop test suite that included ATK.
Create some shallow tests to ensure that the CSPI interface remains API compatible between versions.
Some input from ISP RAS would be helpful as they have previously performed this task for other libraries and interfaces.
Very rough guesses:
- LSB Specification - Variable depending on community agreement, 1 man month.
- Test requirements - 1 man month.
- Test Scenarios - 2 man weeks.
- GTK implementation - 2 man months.
CORBA & D-Bus AT-SPI
AT-SPI is in the process of changing the IPC mechanism used by its libraries. This is a large change to the infrastructure. Although the intention is to keep libraries API compatible CSPI and Pyatspi LSB breaks will probably occur. LSB standardization should not take place for these libraries until after this rather large change has been completed.
Work on creating the LSB test suite may run at the same time as the D-Bus work, as API compatibility is a target.
Using the test scenarios more implementations of cspi and pyatspi could be tested. These currently include OpenOffice/UNO and Java/Swing. In the future the QT toolkit may support a compatible version of AT-SPI. Choice of implementations to test is largely made by the widget toolkit libraries that are standardized by LSB. Currently the only sensible implementations to test would be GTK and QT.
There is no road-map for the inclusion of AT-SPI library specifications into LSB. Any inclusion will take place post LSB 4.0.