Note: These minutes have not yet been finalized.
Note: These minutes have not yet been finalized. [I apologize for likely confusing people and mis-attributing quotes. - MG]
JS: The LSB has a set of tests. ATK is part of the LSB. There is some level of testing already. We could do better with testing that what the user did will translate into meaningful data that will be translated into the right accessibility response. Looking to enhance the testing to test AT-SPI and have this be part of LSB. Have had brief conversations with Aaron Leventhal about having the Mozilla Foundation to write these tests. Aaron agreed that this would be worth funding, but now Willie Walker is the contact person.
WW: Aaron was an advisor to Frank Heckor (sp?) Marco Zehe and I are both sharing a role that Aaron left. In general, this seems like the right thing to do, to ensure accessibility is done correctly across multiple platforms. Want to support that.
JS: Could ask for Aaron's blessing if that seems useful to do. Question for us is to figure out how we want to proceed with this. Have some early prototype tests that GK created a year ago. What we would like to do and how this scopes into a job description, how much time it needs to get done, so that we can write a grant. Then there is the process of sending to LSB, but won't get into that today.
GK: LSB has a group in Russia that has categories of depe testing, normal testing, shallow tests. They say that we have 100% normal tests for ATK, but nobody has looked at them in detail. We need to differentiate ourselves / be able to explain why we did those tests.
JS: First action item is to look at what's tested
MG: Test ATK but not AT-SPI
GK: IBM prototype tests ATK but with a pseudo assistive technology, but this indirectly tests AT-SPI. Approach now is to say that libATK is normative. Libatspi probably isn't normative, but we need to utilize to ensure that things are properly tested.
PB: Makes a lot of sense. Ultimately boils down to whether an assistive technology works in this environment
GK: If I had to do it all over again, would use pyatspi instead of Cspi
WW: Not trying to push the Orca test harness on anybody, but we have facilities where we synthesize keyboard events, then, on the Orca side, get the AT-SPI events and output speech/braille, use the output of Orca to do our regression testing, general idea is to have a framework to play things back. We shouldn't use Orca for the ATK/AT-SPI tests, though, since Orca is always going through constant improvement.
AR: Could use Accerciser; has a lot of things that can be reused for making scripts/tests
PB?: Want something stable, that someone won't change from underneath you and your tests are broken
JS: Need to be able to work relatively easy in the dbus environment
JS: Should we break this down into a job description?
MD offering to do this
JS: Are all tests automated? We might need some run by users
GK: All tests are automated. Have POSIX test results. Have six types of passes/fails.
GK: We might look at what LSB has already done and expand it / make it deeper by using a pseudo-at. The GTK tests had a bit of ATK testing, but it just went to gail and never went across the wire to AT-SPI. The IBM tests had an application and a pseudo-at to exercise the whole system.
MD: Are those tests still available?
GK: Will send a url. If I had my team do this over again, I'd have them use Python
JS: The URL is in the agenda and/or the meeting announcement
GK?: What are trying to test? ATK itself, or the ATK+AT-SPI infrastructure?
JS: Last year we decided to focus on libATK to test that site of things, to leave room to test QT separately. Our moving to dbus might change things.
GK?: We also have the Java side of things, that doesn't use ATK at all. Would be nice to have one test suite that would ensure that the info given to the assistive technology is correct.
GK?: The event model isn't specified very well. If we have gail using ATK, the events coming from it may be from a different order and different types than the events from the OpenOffice toolkit, and same with QT.
MD: What events?
GK?: focus events, caret movement, text inserted, what order they'll come in, are the object model and menus going to be the same, that kind of thing. You're at the mercy of the toolkit.
MD: Tests have to be generic enough to take that into account.
GK?: We get some of these events in a set. Maybe have a larger set of events from a single input event. They may not be in a given order, but at least we get them.
MD: Perhaps test what the state is when things settle down.
GK?: Make sure we at least get a focus event when you press tab. Going to be a hard problem. Focusing on GNOME alone for the start may be a bad thing to do; may end up missing these kind of assumptions. Try to use other toolkits along the way. OO exists and works. Java works with Corba. When we move to dbus, we'll have all of those plus QT
WW?: If we test ATK alone, we don't have this toolkit problem, but do we test everything we want to test?
GK?: Might be two steps: one suite for ATK and another suite for AT-SPI. If bugs came up, you can be certain that the ATK implementation is valid first, the ncheck AT-SPI second.
MD: If we move to dbus, it shouldn't invalidate any of the tests. AT-SPI has to be tested as well, but adding more ATK tests shouldn't change anything.
WW?: GK, did your tests have pseudo-applications that would generate events as well as pseudo-ATs?
GK: Yes. Wasn't dependent on gedit or gtk-demo or anything like that. Just writing any test cases.
WW: Raw ATK objects, or gail/GTK objects?
GK: Don't recall; might have been little GTK applications
JS: Mark going to give us a job description that outlines what needs to happen, and we'll come back. Can anyone look at the ATK tests?
AR: I could probably look at it
JS: When should we come back to this?
MD: Will have something written up in two weeks
PB: A proposed interface for accessing revisions. Worked out through the ODF sub-committee; WW has looked at it. We think that it's stable and want to know if the same thing would work on the Linux side of the equation. Want to minimize differences between platform to help apps supporting multiple platforms. A little concerned; hasn't seen any activity on the Linux side since Bill left.
WW: It's a valid concern. Nobody has stepped up. Should probably bring in Li Yuan. Probably the better person to step up for the spec discussion; he took over maintainership from Bill and is the person who would have to implement it.
JS: This call is at a difficult time of day in China.
WW: Li is a really capable engineer, but time zone would be an issue. Expect a 24-hour turn-around.
PB: Willing to chat with him late at night.
PB: IAccessible2 spec is just about ready to go out to review. Working on a text attribute set, which I think is done. Working on an object attribute set spec. Getting input from Firefox and Symphony teams. Will send a zip file of docs to Gregory.
MG starting to work on events/signals
Would be good for the Mono team to check in once in a while; perhaps invite everyone on once a month for the first few minutes of the call at least. Will talk to Calvin.