[Accessibility] Accessibility Conference Wednesday 1/25/2005 AM

john goldthwaite jgoldthwaite at yahoo.com
Wed Feb 16 14:49:28 PST 2005

Accessibility Conference Wednesday, January 26, 2005 
- morning session

Bill Haneman
Will Walker – speech, Sun
Mark Mulcahy – gnome speech, pda for blind
George Kraft – IBM
Gunnar Schmidt – KDE Access
Olaf Schmidt – KDE Access
Hararld Fernengel– Trolltech
Peter Korn
Marco Gambra – Suse, screen reader
Ed Price 
John Goldthwaite
Doug Beatty
Saban Tibo – braill
Sebastian  Hinderer
Sebastian Thibault
Jonathan Blanford, red hat
Al Gilman, W3C WAI
Larry Weiss, IBM accessibility
Hynek Hanke  Braillecom
Tomas Cerha	
Milan Zamazal
Jan Buchal Braillecom

Kirk Reiser, Toronto
? , Australia 


Bill Haneman - AT/SPI  

Overview of an Accessible Desktop

          o Meet functional criteria
          o Export/expose information programmatically
          o Give notifications
          o Allow programmatic interaction 
          o Provides location and registration
services for a11y
          o Provides access to device events and
platform I/O
          o Provides communication infrastructure 
      Assistive Technologies:
          o Mediate and adapt the user experience
                + Mediate -> must be listening
                + Adapt -> must proactively manipulate

Applications have responsibilities
Desktop provides services
AT provide functions
Applications have to export information

Long-term goal

      All applications on the desktop are accessible
          o Apps must all meet functional criteria
          o Apps must all export a functionally
consistent a11y ABI
          o AT must have all the platform
services/access that it needs
                + e.g. Keyboard snooping, display
hooks, input devices 
          o Must be possible to write an AT to a known
interface (SPI)
          o Awareness and testing!

The Role of AT-SPI

      AT-SPI provides:
          o Service model
          o Device event requests and notification
          o Application event notifications
          o Programmatic access to application user
          o Simplifies or eliminates AT-maintained off
screen model 
      AT-SPI gives AT a uniform interface within a
highly heterogeneous desktop
          o Makes it possible to write one AT that
gives unified access to GNOME, KDE, Mozilla, Java,
          o Allows AT to concentrate on improving user
                + Simplifies logic in
application-specific enhancements

AT-SPI Architecture
·	'Registry':
o	Central registry of apps which implement a11y API 
o	Central agent handles service event requests and
§	Actual implementation details may vary 
o	Dispatches of application events 
·	AT-SPI 'Accessible' and interfaces:
o	Programmatic access to application user interfaces 
§	UI structure 
§	Content 
§	Visual representation and attributes 

Desktop needs to provide location and registration
services so that AT can find application and register
for services,  provide notification of device events
and I/O, provide communications infrastructure

AT – mediate, must listen to events
Adapt – must proactively manipulate interface

Long term goal- all applications are accessible
Requires: all apps meet functional criteria, apps must
export functionality consistent with AT ABI
-	At must have access to services it needs, keyboard
snooping, display hooks, input device
-	Must be able to write an AT to a known interface SPI
-	Awareness among programmers of services needed,

The application functional comes after we have a
functional consistent ABI – don’t have to speak the
same ABI as long as they can be mapped onto the ABI. 
We want them to export their information in a know
format and we can deal with problems through bridging.

In Linux there are multiple toolkits, heterogeneous
AT-SPI provides the service model, provides device
event notifications, provides application event
notifications, programmatic access to application user
Simplifies or eliminates need for AT maintained off
screen model.

AT can maintain an OSM but it can do so with less
effort without snooping on features in the desktop 

AT-SPI is broader and more general than MSAA.  MSAA
doesn’t quite provide enough information about what is
going on in the screen.  AT-SPI gives AT a uniform
interface within a highly heterogeneous desktop. 
Makes it possible to write one AT that gives unified
access to GNOME, KDE, Mozilla, JAVA and OpenOffice
Allows the AT concentrate on improving user
effectiveness.  Events from AT-SPI still requires
complex logic but it makes it more tractable.

Registry daemon
-	central registry of application which implement AT
-	Central agent handles service event request and
notification (still talk directly to application using
structures provided by registry)
-	Dispatches of application events

UI structure – current there is a one-to-one
correspondence between AT-spi and UI objects,  this is
not required by the architecture, the application
could provide an alternate interface for the AT device
and simplify

It is not possible to register a specific listener for
an object, all go through the registry.  In Java API,
you have to setup a listener for each object you want
to monitor(Java has utilities to set up these
listeners).  AT-SPI requires you to go through the

Doesn’t give you direct access to the document model,
it gives access to a view of the document.

Protocol and ABI
·	Protocol 
o	RPC on objects 
o	CORBA-based 
o	IP or sockets transport 
o	Network-transparent 

o	Based on BonoboUnknown (trivial IDL) 
o	Multiple language bindings exist 
o	C bindings most common, ugliest 
o	Java and Python bindings are far superior 

Remote procedure call model- objects you can sent call
AT lives in its own process space so everything is
done via interprocess communication.
Use IP or sockets transport
CORBA based
Network transparent

Life span of a CORBA object reference is unlimited. 
In reality the IPC may go away at some point.   Can
talk to an accessible app as long as it is running.

Based on Bonobo Unknown – simple IDL
Multiple Language binds for the ABI: Java, C, Python
Java and Python bindings are easier to use than C
bindings.  They don’t expose the CORBA architecture. 
The C binding expose the CORBA, doesn’t handle

Performance- haven’t done comparison on relative
performance of these binds.  Think that Python
bindings maybe better but maybe inherent efficiency of

Desired Characteristics
·	Protocol 
o	Object-oriented 
o	Extensible 
o	RPC 
o	Network-transparent 
o	Standards-based 
o	performant 
o	Multiple language bindings 
o	Readily available to managed runtimes 
o	Based on existing model/ABI 
o	Supports interface query 
o	Controlled dependencies 

Desired characteristics of Protocol
-	needs to be object-oriented
-	needs to be extensible, be able add interfaces
-	support RPC
-	Network transparent
-	Standards based
-	Performant

-Multiple language bindings
-Readily available to managed runtimes
-Based on an existing model/ ABI – base on something
that is widely used
( don’t create a new one and therefore a new
-Supports interface query

Will- provisions for handling failure?
CORBA doesn’t do that very well, Orbit does.  We’ve
avoided tying to Orbit.

Existing Clients and Implementors
·	Clients
o	Dasher (C++, custom toolkit) 
o	Gnopernicus (C, GNOME) 
o	Orca (Python, none) 
·	Implementors
o	Java (via java-access-bridge for at-spi) 
o	Mozilla (ATK) 
o	OpenOffice (java-access-bridge, UNO-a11y) 
o	Adobe Reader (ATK) 
o	KDE (QtAccessible->ATK) [WIP] 
Current clients
-Dasher(C++, )
Gnopernicus (C, Gnome)
Orca (Python, no toolkit

JAVA- via bridge
OpenOffice via java-Access-bridge and UNO
Adobe Reader (ATK)
KDE ( QtAccessible to ATK bridge)
Harald – also writing a Qt to AT-SPI bridge 

ATK – from application standpoint you don’t want to
see all the client server detail. ATK maps to AT-SPI,
uses easy to use in process calls so you don’t have to
do all the RPC 
Thin wrapper  

Client Bindings
·	C:
o	C Corba bindings, defined by the CORBA spec 
o	Cspi candy-coating 
§	Not designed to be normative interface 
§	Implementation requires standard C bindings for ORB 
§	Cspi could be bundled with AT if desired 
·	C++:
o	C++ bindings defined by CORBA spec 
o	Unfortunately, no C++ bindings for ORBit2 
o	Use C bindings instead 
·	Java:
o	Java bindings, defined by CORBA spec 
·	Python:
o	Via PyOrbit 
o	Lifecycle issues become transparent 
Client Bindings:
C – C CORBA bindings, defined by the CORBA spec
Using IDL so its not dependent on a particular
Cspi – wrapper to hid some of the ugliness of the
Corba C bindings

These can make writing AT easier but not designed to
be a normative interface
Implementation requires standard C bindings for ORB
AT can bundle Cspi if desktop doesn’t support it.

No C++ bindings for ORBit2 compiler,  Use C bindings
If you want to use an alternate ORB that has CORBA
bindings, can use it
Java- Java bindings as defined Corba spec
Python – Via PyOrbit,  lifecycle issues become

Barriers to Adoption
·	Dependencies
o	ORBit2 or other CORBA-compliant ORB 
o	Bonobo (remediable) 
·	Performance concerns
o	Protocol-based concerns are addressed by ORBit2's
sockets-based protocol 
·	Competing RPC protocols
Barriers to adoptions:
-	Dependencies:  ORBit2 and other CORBA compliant ORB
& Bonobo
( concern that there are too many library dependencies
– some are pretty soft
lots of libraries.  ORBit2 has some library
The Bonobo dependency is small, could fix that. 

Mark- Registry activation is done through Bonobo so
its major.  Can’t connect to anything if I don’t have
Bonobo installed.

The AT just need to make a call to get the start the

Mark- CORBA doesn’t give you a way to find stuff.
-	it has a way to find things, but people have found
it difficult to use.

The applications that the user wants are all mapping
things on to the X display, all are talk to Xserver. 
Argument that the registry should be defined by the X
display.   Can do registry can be placed on the root

Peter- issues have been discussed many times,  it
would be useful to have a document that enumerates the
problems and potential solutions. 

Olaf- what about console applications.  Need to have a
way to handle console, ss
The xserver solution that would work at console.
-	the API don’t presume that there is a screen, there
are different ways to store the registry and keys.  If
you ssh into a text mode application, the interface is
being graphically though the x virtual terminal so
xserver is there.  There are some ugly problems with

Performance concerns. – If you need to marshall and
de-marshall a lot of data, that is going to take some
time.  If you’re not using shared memory, you
Marshalling, demarshalling over TCP/Ip, serialization
process is going to take time.

Harald- if I want to access a widget tree, that is a
lot of data.  
-	opening and closing a connection for each request, 
need to optimize

Lots of ways of doing things in Linux distributions:
UNO, Jeanie, d..
People have vested interest in these protocols.  We
can’t say we’re going to choose a RPC protocol for the
desktop.  We need to identify one and find what are
the substantive issues.  Longer term- we can look at
ways to improve IPC.

What is the mechanism to go from the proposed
implementation to develop a 2.0 version that might be
based on Dbus or other protocols.

Potential Alternatives
·	Alternatives
o	Promote cspi and provide alternate back-ends 
o	Replace CORBA with DBUS 
o	UNO 
o	Make the IDL normative, and allow alternate ABI
conformance vehicles 
·	Pros/Cons of each
o	Cspi cons: bad for managed runtimes, handles
internal cacheing, not designed to be universal. 
o	Cspi pros: few dependencies in the ABI, already
exists and is used by some AT 
o	DBUS cons: does yet meet requirements, not yet
sufficiently complete. Designed for simpler localhost
communications. So-so for managed runtimes. No IDL
compilers available. 
o	DBUS pros: very few dependencies 

Potential Alternatives (2)
·	Alternatives (cont)
o	UNO 
o	Make the IDL normative, and allow alternate ABI
conformance vehicles 
·	Pros/Cons of each
o	UNO cons: Specific to OpenOffice/StarOffice. 
o	UNO pros: shipped by most distros, mature, meets
criteria. Performant, and supports IDL compilation.
Could be drop-in replacement. 
o	'Many worlds' cons: Novel approach. Short-term path
would retain at-spi dependencies. 
o	'Many worlds' pros: allows short-term adoption of
existing implementations, while being future-proof 
·	Recommend one or several strategies
·	Give a summary of the expected results
·	Name the next steps to be taken
·	Delegate the various tasks 

Do you Yahoo!? 
Yahoo! Mail - You care about security. So do we. 

More information about the Accessibility mailing list