[Accessibility-ia2] next changes to IAccessible2

Pete Brunet pete at a11ysoft.com
Tue Jun 28 12:37:10 PDT 2011


Rich, There is one MSAA method, accLocation, and three IA2 methods,
IAText::characterExtents, IAImage::imageLocation, IAImage::imageSize. 
All these are getters.

Are you asking if IAccessibleImage should include a means for returning
a series of points? 

How would this information be used by AT?

A side note: IAccessibleImage::imageLocation provides for fetching
either screen or parent relative offsets.  MSAA's accLocation only
provides screen relative offsets.

Thanks, Pete

On 6/28/2011 2:04 PM, Richard Schwerdtfeger wrote:
>
> So, we are discussing making canvas accessible. Currently accessible
> objects take bounding rectangles yet drawing objects can be more
> complex. Should we introduce a drawing path to the API vs doing a best
> fit to a rectangle?
>
> Rich
>
>
> Rich Schwerdtfeger
> CTO Accessibility Software Group
>
> Inactive hide details for Pete Brunet ---06/28/2011 01:57:10 PM---AT
> devs, do you use IAccessibleEditableText? Is there a kind Pete Brunet
> ---06/28/2011 01:57:10 PM---AT devs, do you use
> IAccessibleEditableText? Is there a kind of AT that can't use the GUI
> for clipbo
>
> From: Pete Brunet <pete at a11ysoft.com>
> To:
> Cc: IA2 List <Accessibility-ia2 at lists.linux-foundation.org>
> Date: 06/28/2011 01:57 PM
> Subject: Re: [Accessibility-ia2] next changes to IAccessible2
> Sent by: accessibility-ia2-bounces at lists.linuxfoundation.org
>
> ------------------------------------------------------------------------
>
>
>
> AT devs, do you use IAccessibleEditableText?
>
> Is there a kind of AT that can't use the GUI for clipboard operations?
> The same question often comes up regarding IAccessibleAction, i.e. why
> have actions if the same thing can be done via the GUI. My
> understanding is that this is for those who can't use the mouse and
> keyboard. Do ATs for those users not have the ability to use GUI menus?
>
> Pete
>
> On 6/28/2011 2:29 AM, Malte Timmermann wrote:
> > Actually, it's also not clear to me why the clipboard stuff exists
> at all.
> >
> > Maybe it was introduced with JAA/ATK, because there is no "standard
> > platform mechanism" on all supported platforms?
> >
> > I don't know - how would an AT tell any application on GNOME or Mac to
> > copy/paste, w/o faking some key events?
> >
> > The clipboard API is incomplete anyway. For example, I don't have an
> > equivalent API for copy/paste on a drawing document or a bitmap editor.
> > Only for objects supporting accessible text.
> >
> > Malte.
> >
> > Carolyn MacLeod wrote, On 27.06.2011 21:06:
> >> An excellent point, Peter.
> >>
> >> In fact, I just read Malte's discussion on the same API, and something
> >> he said - "If [the user] really is interested in selecting a certain
> >> format, he can use the dialog in the application." - reminded me of our
> >> initial reaction when we were asked to implement
> >> IAccessibleEditableText::copyText, cutText, and pasteText in our
> >> StyledText control.... namely, "Why??".
> >>
> >> Why do AT need a separate way of doing something that every editable
> >> text control already provides by a multitude of standard platform
> >> mechanisms?
> >>
> >> I still don't have the answer. However, since we implemented
> >> IAccessibleEditableText in order to provide insert/replace/delete and
> >> allow setting of attributes, we then implemented cut/copy/paste anyway,
> >> but only because they were in the same interface.
> >>
> >> Our second question was "What are the offsets for? Why not just use the
> >> selection?". The answer to that was that since IAccessibleText::copy
> >> needed offsets because there might not be a selection, someone decided
> >> that IAccessibleEditableText cut/copy/paste needed offsets, too, for
> >> consistency. This was an unfortunate decision, because it gives these
> >> methods a very strange side effect: the server is forced to change the
> >> selection to match the offsets before doing the clipboard operation. We
> >> grudgingly implemented these methods, and hoped that no AT would ever
> >> send us any offsets other than those for the already existing
> selection.
> >> :)
> >>
> >> (Epilogue: I cannot find a definition of IAccessibleText::copy
> anywhere,
> >> so the original reason that there are offsets for the clipboard methods
> >> has been completely lost in the mists of time).
> >>
> >> Now, since multi-selection makes the strange semantics even stranger, I
> >> am happy to revert back to our initial impression that
> >> IAccessibleEditibleText[2] doesn't need clipboard operations at all. :)
> >>
> >> Carolyn
> >>
> >>
> >> From: Peter Korn <peter.korn at oracle.com>
> >> To: Carolyn MacLeod/Ottawa/IBM at IBMCA
> >> Cc: Alexander Surkov <surkov.alexander at gmail.com>,
> >> accessibility-ia2-bounces at lists.linuxfoundation.org, IA2 List
> >> <Accessibility-ia2 at lists.freestandards.org>
> >> Date: 27/06/2011 02:22 PM
> >> Subject: Re: [Accessibility-ia2] next changes to IAccessible2
> >>
> >>
> >>
> ------------------------------------------------------------------------
> >>
> >>
> >>
> >> Carolyn, gang,
> >>
> >> Before we get too deep into (re-) designing this API... Do we have a
> >> clear set of use cases? What AT do we expect to call this/these API
> >> call(s), supporting which use cases?
> >>
> >>
> >> Regards,
> >>
> >> Peter
> >>
> >> On 6/27/2011 11:13 AM, Carolyn MacLeod wrote:
> >>
> >> Hi, all.
> >>
> >> Regarding *IAccessibleEditableText2::pasteText(startOffset, endOffset,
> >> mimeType)*:
> >>
> >> 1) If we are going to use "mime type" to specify the clipboard format,
> >> then I think we need an explicit mapping from mime type to
> >> platform-specific clipboard format so that AT and server are both
> >> speaking the same language. For example, something like:
> >> MIME type Windows GTK/ATK
> >> text/plain CF_TEXT (predefined) gdk_atom_intern("COMPOUND_TEXT")
> >> gdk_atom_intern("UTF8_STRING")
> >> gdk_atom_intern("STRING")
> >> text/rtf CF_RTF = RegisterClipboardFormat("Rich Text Format")
> >> gdk_atom_intern("text/rtf")
> >> gdk_atom_intern("TEXT/RTF")
> >> gdk_atom_intern("application/rtf")
> >> text/html CF_HTML = RegisterClipboardFormat("HTML Format")
> >> gdk_atom_intern("text/html")
> >> gdk_atom_intern("TEXT/HTML")
> >>
> >>
> >>
> >>
> >> 2) Do we allow only text/* MIME types? Or do we allow all mime types?
> >> For example, is *pasteText(10, 20, "image/jpeg")* allowed? Probably
> >> should be, given that most word processors can paste an image (among
> >> other things) at a specific location, but the name pasteText is wrong
> >> for that context (unless it means "paste the *into *the text"...).
> >>
> >> For the curious, here is a typical list of text/* mime types, taken
> from
> >> Java AWT data transfer's "best text flavor" method:_
> >>
> __http://download.oracle.com/javase/6/docs/api/java/awt/datatransfer/DataFlavor.html#selectBestTextFlavor(java.awt.datatransfer.DataFlavor_
> >>
> <http://download.oracle.com/javase/6/docs/api/java/awt/datatransfer/DataFlavor.html#selectBestTextFlavor%28java.awt.datatransfer.DataFlavor>[])
> >> •"text/sgml"
> >> •"text/xml"
> >> •"text/html"
> >> •"text/rtf"
> >> •"text/enriched"
> >> •"text/richtext"
> >> •"text/uri-list"
> >> •"text/tab-separated-values"
> >> •"text/t140"
> >> •"text/rfc822-headers"
> >> •"text/parityfec"
> >> •"text/directory"
> >> •"text/css"
> >> •"text/calendar"
> >> •"application/x-java-serialized-object"
> >> •"text/plain"
> >> •"text/<other>"
> >>
> >> 3) We need to more fully specify: "If mime type is missed then
> >> application choose more appropriate one depending on insertion
> context."
> >> i.e. If mime_type is empty string or NULL? Also, if mime_type is
> >> unsupported by the server, I assume we would return E_INVALIDARG and
> >> paste nothing?
> >>
> >> 4) Paste gets a bit complicated in the context of multiple selections,
> >> so the following sentence may need revision:
> >> "If both start and end offsets are equal to IA2_TEXT_OFFSET_SELECTION
> >> (value -3) then they point to start and end offset of active selection
> >> (i.e. start or end of selection has a caret)."
> >>
> >> If we truly mean that pasteText only pastes onto the selection that has
> >> the caret, then perhaps we can spec it as:*
> >> pasteText(IA2_TEXT_OFFSET_CARET, IA2_TEXT_OFFSET_SELECTION, mime_type)*
> >> rather than:*
> >> pasteText(IA2_TEXT_OFFSET_SELECTION, IA2_TEXT_OFFSET_SELECTION,
> mime_type)*
> >> This would make it clearer that it is the selection with the caret that
> >> will be operated on.
> >>
> >> However, it probably makes more sense to spec that the paste should
> >> happen on all selections. Consider the following semantics of clipboard
> >> operations on multiple selections in MS Word (which is not the defining
> >> app for multiple selection clipboard operations, however their
> semantics
> >> mostly make sense... and when I tried FF, I got odd behavior for
> >> everything except Copy).
> >>
> >> If we have this line of text:
> >> One fish two fish red fish blue fish
> >> Now select One two red blue by double-clicking while holding down Ctrl
> >> (I don't know how to do this with only the keyboard), then Copy,
> and the
> >> clipboard contains:
> >> One
> >> two
> >> red
> >> blue
> >> (I am not sure why line breaks are inserted between words... that is a
> >> bit odd. Multi-select copy in FF concatenates the words without line
> >> breaks).
> >>
> >> Start again with the original text, select One two red blue as before,
> >> and Cut, and we are left with:
> >> fish fish fish fish
> >> The clipboard contains the same data as for Copy.
> >>
> >> Start again with the original text, select One two red blue, and Paste
> >> "the ", and we get:
> >> the fish the fish the fish the fish
> >>
> >> Finally, start again with the original text, select One two red blue,
> >> and type "green", and we have:
> >> One fish two fish red fish green fish
> >>
> >> In other words, clipboard commands operate on all selections, but
> insert
> >> (typing) only operates on the selection that contains the caret.
> >>
> >> What I am getting at by all of this is that maybe pasteText should not
> >> have any offset parameters at all, because startOffset and
> endOffset are
> >> inadequate to define multiple selections).
> >> Perhaps pasteText should simply be specified to always operate on the
> >> selection, whatever that may be.
> >>
> >> And if that is the case, then it only makes sense to remove the
> >> parameters from copyText and cutText as well, for the same reason.
> >>
> >> Carolyn
> >>
> >>
> >> From: Alexander Surkov _<surkov.alexander at gmail.com>_
> >> <mailto:surkov.alexander at gmail.com>
> >> To: Brian Cragun _<cragun at us.ibm.com>_ <mailto:cragun at us.ibm.com>
> >> Cc: _accessibility-ia2-bounces at lists.linuxfoundation.org_
> >> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>, IA2 List
> >> _<Accessibility-ia2 at lists.freestandards.org>_
> >> <mailto:Accessibility-ia2 at lists.freestandards.org>
> >> Date: 08/06/2011 06:45 AM
> >> Subject: Re: [Accessibility-ia2] next changes to IAccessible2
> >> Sent by: _accessibility-ia2-bounces at lists.linuxfoundation.org_
> >> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>
> >>
> >>
> >>
> >>
> ------------------------------------------------------------------------
> >>
> >>
> >>
> >> Hi, Brian.
> >>
> >> I added this suggestion to wiki -_
> >>
> __https://wiki.mozilla.org/Accessibility/IA2_1.3#IAccessibleEditableText2_interface_.
> >>
> >> Thank you.
> >> Alex.
> >>
> >>
> >> On Tue, Jun 7, 2011 at 10:51 PM, Brian Cragun _<cragun at us.ibm.com>_
> >> <mailto:cragun at us.ibm.com>wrote:
> >>> I propose we add a way for IA2 Paste Text to provide both a Start and
> >> an End
> >>> Offset parameter. Also to provide an Attributes parameter.
> >>> Copy-with-parameters should be implemented as an additional method in
> >>> IAccessibleText.
> >>>
> >>> See previous exchanges on this topic to the list:
> >>>
> >>
> _https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-September/001219.html_
> >>
> _https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-October/001223.html_
> >>> Previously this was not changed because it required a new interface.
> >> Now we
> >>> are making new interfaces. Good time to add.
> >>>
> >>> Regards,
> >>>
> >>> Brian
> >>>
> >>> Brian Cragun
> >>> IBM AbilityLab Consultant
> >>> Human Ability & Accessibility Center
> >>> _www.ibm.com/able_& w3.ibm.com/able
> >>> W:(720)-663-2801 H:(507)288-2437
> >>>
> >>>
> >>>
> >>>
> >>> From: Pete Brunet _<pete at a11ysoft.com>_ <mailto:pete at a11ysoft.com>
> >>> To: IA2 List _<Accessibility-ia2 at lists.freestandards.org>_
> >> <mailto:Accessibility-ia2 at lists.freestandards.org>
> >>> Date: 06/06/2011 11:12 PM
> >>> Subject: Re: [Accessibility-ia2] next changes to IAccessible2
> >>> Sent by: _accessibility-ia2-bounces at lists.linuxfoundation.org_
> >> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>
> >>> ________________________________
> >>>
> >>>
> >>> Hi all, Please take a look at this and provide your feedback:
> >>>
> >>> _https://wiki.mozilla.org/Accessibility/IA2_1.3_
> >>>
> >>> Thanks, Pete
> >>> --
> >>> Pete Brunet
> >>>
> >>> a11ysoft - Accessibility Architecture and Development
> >>> (512) 238-6967 (work), (512) 689-4155 (cell)
> >>> Skype: pete.brunet
> >>> IM: ptbrunet (AOL, Google), _ptbrunet at live.com_
> >> <mailto:ptbrunet at live.com>(MSN)
> >>> _http://www.a11ysoft.com/about/_
> >>> Ionosphere: WS4G
> >>>
> >>> On 3/11/2011 11:10 PM, Alexander Surkov wrote:
> >>> Hi, Jamie. I missed Mick suggestion on the list. It's sounds
> >> reasonable and
> >>> I agree we should try it before getting new API for this since the
> >> issue is
> >>> mostly about events.
> >>>
> >>> Thank you.
> >>> Alex.
> >>>
> >>>
> >>> On Sat, Mar 12, 2011 at 11:43 AM, James Teh _<jamie at nvaccess.org>_
> >> <mailto:jamie at nvaccess.org>wrote:
> >>> Hi.
> >>>
> >>> Nice work; good to get the discussion going. :)
> >>>
> >>> I still don't see a need for this registry API. Why not just use
> >>> IsWinEventHookInstalled(), as Mick suggested on the IA2 list?
> >>>
> >>> Thanks.
> >>>
> >>> Jamie
> >>>
> >>>
> >>> On 12/03/2011 3:48 AM, Alexander Surkov wrote:
> >>> Hi.
> >>>
> >>> I gathered ideas into one doc -
> >>> _https://wiki.mozilla.org/Accessibility/IA2_1.3_. Please give feedback
> >>> here and feel free to edit the wiki.
> >>>
> >>> Thank you.
> >>> Alex.
> >>>
> >>> --
> >>> James Teh
> >>> Vice President, Developer
> >>> NV Access Inc, ABN 61773362390
> >>> Email: _jamie at nvaccess.org_ <mailto:jamie at nvaccess.org>
> >>> Web site: _http://www.nvaccess.org/_
> >>>
> >>> _______________________________________________
> >>> Accessibility-ia2 mailing list
> >>> _Accessibility-ia2 at lists.linuxfoundation.org_
> >> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
> >>>
> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
> >>>
> >>>
> >>> _______________________________________________
> >>> Accessibility-ia2 mailing list
> >>> _Accessibility-ia2 at lists.linuxfoundation.org_
> >> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
> >>>
> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
> >>>
> >>>
> >> _______________________________________________
> >> Accessibility-ia2 mailing list_
> >> __Accessibility-ia2 at lists.linuxfoundation.org_
> >> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>_
> >>
> __https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Accessibility-ia2 mailing list
> >> _Accessibility-ia2 at lists.linuxfoundation.org_
> >> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
> >> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
> >>
> >>
> >> -- _
> >> _Oracle <http://www.oracle.com/>
> >> Peter Korn | Accessibility Principal
> >> Phone: _+1 650 5069522_ <tel:+1%20650%205069522>
> >> 500 Oracle Parkway | Redwood City, CA 94065_
> >> _Green Oracle <http://www.oracle.com/commitment>Oracle is committed to
> >> developing practices and products that help protect the environment
> >>
> >>
> >>
> >> _______________________________________________
> >> Accessibility-ia2 mailing list
> >> Accessibility-ia2 at lists.linuxfoundation.org
> >> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2
> > _______________________________________________
> > Accessibility-ia2 mailing list
> > Accessibility-ia2 at lists.linuxfoundation.org
> > https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2
> >
>
> _______________________________________________
> Accessibility-ia2 mailing list
> Accessibility-ia2 at lists.linuxfoundation.org
> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2
>

-- 
*Pete Brunet*
                                                                
a11ysoft - Accessibility Architecture and Development
(512) 689-4155 (cell)
Skype: pete.brunet
IM: ptbrunet (AOL, Google), ptbrunet at live.com (MSN)
http://www.a11ysoft.com/about/
Ionosphere: WS4G
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.linux-foundation.org/pipermail/accessibility-ia2/attachments/20110628/df301960/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 105 bytes
Desc: not available
Url : http://lists.linux-foundation.org/pipermail/accessibility-ia2/attachments/20110628/df301960/attachment-0001.gif 


More information about the Accessibility-ia2 mailing list