[Accessibility-ia2] next changes to IAccessible2

Malte Timmermann malte.timmermann at oracle.com
Tue Jun 28 00:29:21 PDT 2011


Actually, it's also not clear to me why the clipboard stuff exists at all.

Maybe it was introduced with JAA/ATK, because there is no "standard
platform mechanism" on all supported platforms?

I don't know - how would an AT tell any application on GNOME or Mac to
copy/paste, w/o faking some key events?

The clipboard API is incomplete anyway. For example, I don't have an
equivalent API for copy/paste on a drawing document or a bitmap editor.
Only for objects supporting accessible text.

Malte.

Carolyn MacLeod wrote, On 27.06.2011 21:06:
> 
> An excellent point, Peter.
> 
> In fact, I just read Malte's discussion on the same API, and something
> he said - "If [the user] really is interested in selecting a certain
> format, he can use the dialog in the application." - reminded me of our
> initial reaction when we were asked to implement
> IAccessibleEditableText::copyText, cutText, and pasteText in our
> StyledText control.... namely, "Why??".
> 
> Why do AT need a separate way of doing something that every editable
> text control already provides by a multitude of standard platform
> mechanisms?
> 
> I still don't have the answer. However, since we implemented
> IAccessibleEditableText in order to provide insert/replace/delete and
> allow setting of attributes, we then implemented cut/copy/paste anyway,
> but only because they were in the same interface.
> 
> Our second question was "What are the offsets for? Why not just use the
> selection?". The answer to that was that since IAccessibleText::copy
> needed offsets because there might not be a selection, someone decided
> that IAccessibleEditableText cut/copy/paste needed offsets, too, for
> consistency. This was an unfortunate decision, because it gives these
> methods a very strange side effect: the server is forced to change the
> selection to match the offsets before doing the clipboard operation. We
> grudgingly implemented these methods, and hoped that no AT would ever
> send us any offsets other than those for the already existing selection.
>  :)
> 
> (Epilogue: I cannot find a definition of IAccessibleText::copy anywhere,
> so the original reason that there are offsets for the clipboard methods
> has been completely lost in the mists of time).
> 
> Now, since multi-selection makes the strange semantics even stranger, I
> am happy to revert back to our initial impression that
> IAccessibleEditibleText[2] doesn't need clipboard operations at all.   :)
> 
> Carolyn
> 
> 
> From: 	Peter Korn <peter.korn at oracle.com>
> To: 	Carolyn MacLeod/Ottawa/IBM at IBMCA
> Cc: 	Alexander Surkov <surkov.alexander at gmail.com>,
> accessibility-ia2-bounces at lists.linuxfoundation.org, IA2 List
> <Accessibility-ia2 at lists.freestandards.org>
> Date: 	27/06/2011 02:22 PM
> Subject: 	Re: [Accessibility-ia2] next changes to IAccessible2
> 
> 
> ------------------------------------------------------------------------
> 
> 
> 
> Carolyn, gang,
> 
> Before we get too deep into (re-) designing this API... Do we have a
> clear set of use cases?  What AT do we expect to call this/these API
> call(s), supporting which use cases?
> 
> 
> Regards,
> 
> Peter
> 
> On 6/27/2011 11:13 AM, Carolyn MacLeod wrote:
> 
> Hi, all.
> 
> Regarding *IAccessibleEditableText2::pasteText(startOffset, endOffset,
> mimeType)*:
> 
> 1) If we are going to use "mime type" to specify the clipboard format,
> then I think we need an explicit mapping from mime type to
> platform-specific clipboard format so that AT and server are both
> speaking the same language. For example, something like:
> MIME type	Windows	GTK/ATK
> text/plain	CF_TEXT (predefined)	gdk_atom_intern("COMPOUND_TEXT")
> gdk_atom_intern("UTF8_STRING")
> gdk_atom_intern("STRING")
> text/rtf	CF_RTF = RegisterClipboardFormat("Rich Text Format")
> gdk_atom_intern("text/rtf")
> gdk_atom_intern("TEXT/RTF")
> gdk_atom_intern("application/rtf")
> text/html	CF_HTML = RegisterClipboardFormat("HTML Format")
> gdk_atom_intern("text/html")
> gdk_atom_intern("TEXT/HTML")
> 
> 
> 
> 
> 2) Do we allow only text/* MIME types? Or do we allow all mime types?
> For example, is *pasteText(10, 20, "image/jpeg")* allowed? Probably
> should be, given that most word processors can paste an image (among
> other things) at a specific location, but the name pasteText is wrong
> for that context (unless it means "paste the *into *the text"...).
> 
> For the curious, here is a typical list of text/* mime types, taken from
> Java AWT data transfer's "best text flavor" method:_
> __http://download.oracle.com/javase/6/docs/api/java/awt/datatransfer/DataFlavor.html#selectBestTextFlavor(java.awt.datatransfer.DataFlavor_
> <http://download.oracle.com/javase/6/docs/api/java/awt/datatransfer/DataFlavor.html#selectBestTextFlavor%28java.awt.datatransfer.DataFlavor>[])
> •"text/sgml"
> •"text/xml"
> •"text/html"
> •"text/rtf"
> •"text/enriched"
> •"text/richtext"
> •"text/uri-list"
> •"text/tab-separated-values"
> •"text/t140"
> •"text/rfc822-headers"
> •"text/parityfec"
> •"text/directory"
> •"text/css"
> •"text/calendar"
> •"application/x-java-serialized-object"
> •"text/plain"
> •"text/<other>"
> 
> 3) We need to more fully specify: "If mime type is missed then
> application choose more appropriate one depending on insertion context."
> i.e. If mime_type is empty string or NULL? Also, if mime_type is
> unsupported by the server, I assume we would return E_INVALIDARG and
> paste nothing?
> 
> 4) Paste gets a bit complicated in the context of multiple selections,
> so the following sentence may need revision:
> "If both start and end offsets are equal to IA2_TEXT_OFFSET_SELECTION
> (value -3) then they point to start and end offset of active selection
> (i.e. start or end of selection has a caret)."
> 
> If we truly mean that pasteText only pastes onto the selection that has
> the caret, then perhaps we can spec it as:*
> pasteText(IA2_TEXT_OFFSET_CARET, IA2_TEXT_OFFSET_SELECTION, mime_type)*
> rather than:*
> pasteText(IA2_TEXT_OFFSET_SELECTION, IA2_TEXT_OFFSET_SELECTION, mime_type)*
> This would make it clearer that it is the selection with the caret that
> will be operated on.
> 
> However, it probably makes more sense to spec that the paste should
> happen on all selections. Consider the following semantics of clipboard
> operations on multiple selections in MS Word (which is not the defining
> app for multiple selection clipboard operations, however their semantics
> mostly make sense... and when I tried FF, I got odd behavior for
> everything except Copy).
> 
> If we have this line of text:
> One fish two fish red fish blue fish
> Now select One two red blue by double-clicking while holding down Ctrl
> (I don't know how to do this with only the keyboard), then Copy, and the
> clipboard contains:
> One
> two
> red
> blue
> (I am not sure why line breaks are inserted between words... that is a
> bit odd. Multi-select copy in FF concatenates the words without line
> breaks).
> 
> Start again with the original text, select One two red blue as before,
> and Cut, and we are left with:
> fish fish fish fish
> The clipboard contains the same data as for Copy.
> 
> Start again with the original text, select One two red blue, and Paste
> "the ", and we get:
> the fish the fish the fish the fish
> 
> Finally, start again with the original text, select One two red blue,
> and type "green", and we have:
> One fish two fish red fish green fish
> 
> In other words, clipboard commands operate on all selections, but insert
> (typing) only operates on the selection that contains the caret.
> 
> What I am getting at by all of this is that maybe pasteText should not
> have any offset parameters at all, because startOffset and endOffset are
> inadequate to define multiple selections).
> Perhaps pasteText should simply be specified to always operate on the
> selection, whatever that may be.
> 
> And if that is the case, then it only makes sense to remove the
> parameters from copyText and cutText as well, for the same reason.
> 
> Carolyn
> 
> 
> From:	Alexander Surkov _<surkov.alexander at gmail.com>_
> <mailto:surkov.alexander at gmail.com>
> To:	Brian Cragun _<cragun at us.ibm.com>_ <mailto:cragun at us.ibm.com>
> Cc:	_accessibility-ia2-bounces at lists.linuxfoundation.org_
> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>, IA2 List
> _<Accessibility-ia2 at lists.freestandards.org>_
> <mailto:Accessibility-ia2 at lists.freestandards.org>
> Date:	08/06/2011 06:45 AM
> Subject:	Re: [Accessibility-ia2] next changes to IAccessible2
> Sent by:	_accessibility-ia2-bounces at lists.linuxfoundation.org_
> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>
> 
> 
> 
> ------------------------------------------------------------------------
> 
> 
> 
> Hi, Brian.
> 
> I added this suggestion to wiki -_
> __https://wiki.mozilla.org/Accessibility/IA2_1.3#IAccessibleEditableText2_interface_.
> 
> Thank you.
> Alex.
> 
> 
> On Tue, Jun 7, 2011 at 10:51 PM, Brian Cragun _<cragun at us.ibm.com>_
> <mailto:cragun at us.ibm.com>wrote:
>> I propose we add a way for IA2 Paste Text to provide both a Start and
> an End
>> Offset parameter.  Also to provide an Attributes parameter.
>> Copy-with-parameters should be implemented as an additional method in
>> IAccessibleText.
>>
>> See previous exchanges on this topic to the list:
>>
> _https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-September/001219.html_
>>
> _https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-October/001223.html_
>>
>> Previously this was not changed because it required a new interface.
>  Now we
>> are making new interfaces.  Good time to add.
>>
>> Regards,
>>
>> Brian
>>
>> Brian Cragun
>> IBM AbilityLab Consultant
>> Human Ability & Accessibility Center
>> _www.ibm.com/able_& w3.ibm.com/able
>> W:(720)-663-2801    H:(507)288-2437
>>
>>
>>
>>
>> From:        Pete Brunet _<pete at a11ysoft.com>_ <mailto:pete at a11ysoft.com>
>> To:        IA2 List _<Accessibility-ia2 at lists.freestandards.org>_
> <mailto:Accessibility-ia2 at lists.freestandards.org>
>> Date:        06/06/2011 11:12 PM
>> Subject:        Re: [Accessibility-ia2] next changes to IAccessible2
>> Sent by:        _accessibility-ia2-bounces at lists.linuxfoundation.org_
> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>
>> ________________________________
>>
>>
>> Hi all, Please take a look at this and provide your feedback:
>>
>> _https://wiki.mozilla.org/Accessibility/IA2_1.3_
>>
>> Thanks, Pete
>> --
>> Pete Brunet
>>
>> a11ysoft - Accessibility Architecture and Development
>> (512) 238-6967 (work), (512) 689-4155 (cell)
>> Skype: pete.brunet
>> IM: ptbrunet (AOL, Google), _ptbrunet at live.com_
> <mailto:ptbrunet at live.com>(MSN)
>> _http://www.a11ysoft.com/about/_
>> Ionosphere: WS4G
>>
>> On 3/11/2011 11:10 PM, Alexander Surkov wrote:
>> Hi, Jamie. I missed Mick suggestion on the list. It's sounds
> reasonable and
>> I agree we should try it before getting new API for this since the
> issue is
>> mostly about events.
>>
>> Thank you.
>> Alex.
>>
>>
>> On Sat, Mar 12, 2011 at 11:43 AM, James Teh _<jamie at nvaccess.org>_
> <mailto:jamie at nvaccess.org>wrote:
>> Hi.
>>
>> Nice work; good to get the discussion going. :)
>>
>> I still don't see a need for this registry API. Why not just use
>> IsWinEventHookInstalled(), as Mick suggested on the IA2 list?
>>
>> Thanks.
>>
>> Jamie
>>
>>
>> On 12/03/2011 3:48 AM, Alexander Surkov wrote:
>> Hi.
>>
>> I gathered ideas into one doc -
>> _https://wiki.mozilla.org/Accessibility/IA2_1.3_. Please give feedback
>> here and feel free to edit the wiki.
>>
>> Thank you.
>> Alex.
>>
>> --
>> James Teh
>> Vice President, Developer
>> NV Access Inc, ABN 61773362390
>> Email: _jamie at nvaccess.org_ <mailto:jamie at nvaccess.org>
>> Web site: _http://www.nvaccess.org/_
>>
>> _______________________________________________
>> Accessibility-ia2 mailing list
>> _Accessibility-ia2 at lists.linuxfoundation.org_
> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
>> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
>>
>>
>> _______________________________________________
>> Accessibility-ia2 mailing list
>> _Accessibility-ia2 at lists.linuxfoundation.org_
> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
>> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
>>
>>
> _______________________________________________
> Accessibility-ia2 mailing list_
> __Accessibility-ia2 at lists.linuxfoundation.org_
> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>_
> __https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
> 
> 
> 
> 
> _______________________________________________
> Accessibility-ia2 mailing list
> _Accessibility-ia2 at lists.linuxfoundation.org_
> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
> 
> 
> -- _
> _Oracle <http://www.oracle.com/>
> Peter Korn | Accessibility Principal
> Phone: _+1 650 5069522_ <tel:+1%20650%205069522>
> 500 Oracle Parkway | Redwood City, CA 94065_
> _Green Oracle <http://www.oracle.com/commitment>Oracle is committed to
> developing practices and products that help protect the environment
> 
> 
> 
> _______________________________________________
> Accessibility-ia2 mailing list
> Accessibility-ia2 at lists.linuxfoundation.org
> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2


More information about the Accessibility-ia2 mailing list