[Accessibility-ia2] next changes to IAccessible2

Thu Jun 9 05:41:37 PDT 2011

Oh, I forgot to answer to how it might be possible to deal with when a
device doesn't want to make the video wait: if would just tell the
browser whether it wants it to wait or not and the browser would only
increase the reference counter if AT actually asks it to wait.

Cheers,
Silvia.

On Thu, Jun 9, 2011 at 10:39 PM, Silvia Pfeiffer
<silviapfeiffer1 at gmail.com> wrote:
> Hi Jamie, Alex,
>
> This is a great discussion to have! I am not sure about the correct
> answers, but I hope to learn from the experts (you)! :-)
>
> One argument that speaks for having a specialized interface for video
> and audio elements is that you can then provide a means in the screen
> reader to stop all media elements on a page that have started playing
> (because of autoplay or because of script etc) without having to know
> where they are on the page and getting to them. I am not sure that is
> sufficient to create a completely new interface, but it's one use case
> that I can think of where you want more than just the normal controls.
>
> As for the text descriptions: to have the current algorithm work with
> multiple AT devices would just require adding a "reference counter" to
> the "pauseOnExit" flag. It would only change to "false" when all the
> AT devices that have decided to work in parallel to the cue have
> returned and asked for the next cueText. Something like this:
>
> 1.    When the browser has a new cue available for a text track, then
> the browser fires a "MEDIA_TEXT_CUE_CHANGE" event at AT
> 2.    AT picks up the cue, which causes the browser to set a
> "pauseOnExit" flag to true on this cue; for every AT that picks up the
> cue, a reference counter is increased; from here on, the AT and the
> video play asynchronously
>     2.1   AT reads out cue and calls "cueText" property when it is
> finished, which causes the browser to reduce the reference counter; if
> the reference counter reaches 0, the browser sets the "pauseOnExit"
> flag to false and unpauses the video if it is paused
>     2.2   in parallel the browser plays video until the end of the
> cue; if the "pauseOnExit" flag is true when it reaches the end time of
> the cue, the video pauses, otherwise it continues playing
> 3.    repeat from 1)
>
> Cheers,
> Silvia.
>
>
> On Thu, Jun 9, 2011 at 9:50 PM, Alexander Surkov
> <surkov.alexander at gmail.com> wrote:
>> Hi, Jamie.
>>
>> I can see your point but I'm not sure I share it entirely. In general
>> I agree that screen reader should expose that what's visible and
>> operable for sighted users. This idea makes all specialized interfaces
>> unneeded. However specialized interfaces allows to provide a common
>> way to deal with different objects and can be used to fix web page
>> accessibility errors. In the case of media interface screen reader can
>> provide unified way to play or pause the media for example. This can
>> be really be a good idea when web page authors didn't provided UI to
>> control the playback but added keyboard shortcuts and mouse gestures.
>> While this is operable by screen reader users still but this is a
>> discoverability problem.
>>
>> The proposed approach of cues handling missed the support of multiple
>> ATs. It's a huge gap from the point of API correctness (Firefox
>> doesn't work properly with multiple ATs running the same time but this
>> doens't excuse). I think it can be fixed by this or that way making it
>> more complicated but I think we should figure out if we need special
>> interface for media controls at all.
>>
>> I considered the idea of live regions usage, it's really good until
>> the media shouldn't be stopped until screen reader finishes the
>> reading. To handle this we should introduce new approach that's isn't
>> connected with aria live regions. And this makes me feel this is very
>> hacky way. On the another hand since extended text is supposed to be
>> visible by screen readers only then it implies that aria live regions
>> shouldn't be visible what might be a problem to process them correctly
>> on browser and AT levels.
>>
>> Thank you.
>> Alex.
>>
>>
>> On Thu, Jun 9, 2011 at 8:08 PM, James Teh <jamie at nvaccess.org> wrote:
>>> Hi.
>>>
>>> While I recognise the importance of media accessibility, I think special
>>> purpose interfaces are generally best avoided and so extreme caution
>>> needs to be exercised when considering them. I think we need to clearly
>>> determine the use cases for each interface and part thereof. My initial
>>> feeling is that most of this either does not belong in an accessibility
>>> API or can be done in a far more general way.
>>>
>>> I don't see the need for a special interface to control media. The media
>>> playback controls can already be exposed as normal controls to ATs (e.g.
>>> buttons, sliders, etc.), just as they are now, with the application
>>> providing keyboard shortcuts as appropriate. Selection and toggling of
>>> media tracks can (and should) similarly be handled by the application
>>> using a standard UI.
>>>
>>> Output of text cues can be generalised and handled using live regions.
>>>
>>> The only piece remaining is pausing of audio while the AT reports the
>>> cue to the user if that reporting would overlap with another cue. This
>>> really does venture outside the scope of current accessibility APIs, as
>>>  ATs don't generally change application behaviour, but rather allow
>>> exposure and interaction with UI. I don't really have a good solution
>>> for this yet. However, the proposed solution has several problems:
>>> 1. The event could be picked up by multiple ATs, but not all of them may
>>> want the pause on exit behaviour. An event shouldn't generally change
>>> application behaviour just because an app picked it up; it should be
>>> passive. Also, keep in mind that many a11y clients register for all
>>> events, even if they don't use them all.
>>> 2. What if a user is using multiple ATs; e.g. screen reader and
>>> magnifier? One might issue a request to fetch the next cue, but another
>>> might not be finished outputting it yet.
>>>
>>> Jamie
>>>
>>> On 8/06/2011 7:46 PM, Alexander Surkov wrote:
>>>> Hi.
>>>>
>>>> Based on my talk with Silvia couple days ago I think I can suggest API
>>>> for your consideration -
>>>> https://wiki.mozilla.org/Accessibility/IA2_1.3#IAccessibleMedia_interface.
>>>>
>>>> Thank you.
>>>> Alex.
>>>
>>> --
>>> James Teh
>>> Vice President, Developer
>>> NV Access Inc, ABN 61773362390
>>> Email: jamie at nvaccess.org
>>> Web site: http://www.nvaccess.org/
>>> _______________________________________________
>>> Accessibility-ia2 mailing list
>>> Accessibility-ia2 at lists.linuxfoundation.org
>>> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2
>>>
>> _______________________________________________
>> Accessibility-ia2 mailing list
>> Accessibility-ia2 at lists.linuxfoundation.org
>> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2
>>
>