[Accessibility-atspi] atktest-0.1.1
Stew Benedict
stewb at aysenterprises.com
Fri Mar 30 07:33:54 PDT 2007
OK,
Spent some time on this. I'll re-iterate I'm fairly ignorant on
accessibility issues, so feel free to educate me as needed :)
I've included a patch which addresses:
1) More feedback in the journal as to why the test fails
2) Tweak the Makefile to:
a) make it more portable
b) just some housekeeping (allow lsbcc some day, make clean)
c) allow us to do testing as part of the build process
2.c is part of the "lessons learned" in the 3.1.1 update work. We
repeatedly got bit late in the game by tests which changed behavior either
due to upstream changes or changes on our build hosts(s). Moving forward,
we're hoping to be able to catch many of these things earlier, using
autobuilds/autotests. Being able to do "make test" in the package build
and having the autobuild fail would give us an early heads up.
How this particular test will fit into the packaging is still up in the
air, but for the interim having "make test" helps me in working with it.
Issues:
--------------------------------------------------------------------
Test Results:
--------------------------------------------------------------------
I get 6 of 6 failures on SLES10 (using lsb-xvfb as the X environment).
I know there was some question about using the lsb-tet3-lite vs upstream,
but using either one didn't seem to make any difference here. I did:
TET_ROOT=/opt/lsb-tet3-lite make clean
TET_ROOT=/opt/lsb-tet3-lite make test
and
TET_ROOT=../tet37a make clean
TET_ROOT=../tet37a make test
with ../tet37 being a locally built copy per the README
No difference in the test results using either tet source.
Is there something that needs to be done to setup the environment for
testing?
Journal with the test results also attached.
----------------------------------------------------------------------
Build fails with lsbcc:
----------------------------------------------------------------------
CC=lsbcc TET_ROOT=/opt/lsb-tet3-lite make
-- snip some output --
-- end snip --
lsbcc -o ATKComponent \
ATKCommon.o ATKComponent.o \
-Wl,--export-dynamic -L/usr/X11R6/lib -L/opt/gnome/lib -pthread
-lcspi -lspi -lgnomeui-2 -lpthread -lgnome-2 -latk-1.0 -lpango-1.0
-lgobject-2.0 -lbonobo-2 -lbonobo-activation -lORBit-2 -lgmodule-2.0
-lgthread-2.0 -lglib-2.0 \
/opt/lsb-tet3-lite/lib/tet3/tcm.o \
/opt/lsb-tet3-lite/lib/tet3/libapi.a
/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/bin/ld:
cannot find -lcspi
collect2: ld returned 1 exit status
make: *** [ATKComponent] Error 1
It looks like we're using things not part of LSB yet.
Part of our open bugs is to get the tests so they can build with lsbcc.
LIBS = -Wl,--export-dynamic -pthread -lcspi -lspi -lgnomeui-2 -lpthread
-lgnome-2 -latk-1.0 -lpango-1.0 -lgobject-2.0 -lbonobo-2 -lbonobo-activation
-lORBit-2 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0
mysql> select * from Library where Lname like '%bonobo%' or Lname like
'%spi%' or Lname like '%ORB%' or LName like '%gnome%';
Empty set (0.00 sec)
Do we intend to add any of this to the spec, or would we need to look at
compiling these libs in statically? (cspi/spi seems like it might be key
to what we're trying to test).
The library dependency of this single app is pretty long:
SLES10-presario30:~/rpm/BUILD/atktest-0.1.1-lsb/src$ ldd ATKComponent | wc
-l
67
Not that big of deal I suppose, but I imagine we may hit issues on distros
lacking some of the libs and having the test fail, with no journal output,
if we leave it linked as is.
------------------------------------------------------------------
So, I guess what I really need at this point is some guidance on how to
get the test to PASS, and what if any, inclusions into the LSB spec might
be associated with this test.
Regards,
--
Stew Benedict
-------------- next part --------------
0|3.7-lite 10:03:58 20070330|User: stew (500) TCC Start, Command line: tcc -e -p .
5|Linux presario30.e-artisan.org 2.6.17-11mdv #1 SMP Thu Feb 22 13:56:30 CET 2007 i686|System Information
20|/home/stew/rpm/BUILD/atktest-0.1.1-lsb/src/./tetexec.cfg 1|Config Start
30||TET_OUTPUT_CAPTURE=false
30||TET_API_COMPLIANT=True
30||TET_PASS_TC_NAME=False
30||TET_VERSION=3.7-lite
40||Config End
70||"Starting ATK test suite"
70||"total tests in ATKComponent 6"
10|0 /atk/ATKComponent 10:03:58|TC Start, scenario ref 3-0
15|0 3.7-lite 6|TCM Start
520|0 0 00008627 1 1|Begin Component Interface test startup
520|0 0 00008627 1 2|End Component Interface test startup
400|0 1 1 10:03:58|IC Start
200|0 1 10:03:58|TP Start
520|0 1 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getPosition
520|0 1 00008627 1 2|ATK Set Position return FALSE
520|0 1 00008627 1 3|Expected GetXPosition 500, got 0
520|0 1 00008627 1 4|Expected GetYPosition 150, got 0
520|0 1 00008627 1 5|End test of Component Interface function AccessibleComponent_getPosition
220|0 1 1 10:03:58|FAIL
410|0 1 1 10:03:58|IC End
400|0 2 1 10:03:58|IC Start
200|0 2 10:03:58|TP Start
520|0 2 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getExtents
520|0 2 00008627 1 2|ATK Set Extents return FALSE
520|0 2 00008627 1 3|Expected GetXPosition = 100, got 0
520|0 2 00008627 1 4|Expected GetYPosition = 250, got 0
520|0 2 00008627 1 5|Expected GetWidth = 210, got 0
520|0 2 00008627 1 6|Expected GetHeight = 229, got 0
520|0 2 00008627 1 7|End test of Component Interface function AccessibleComponent_getExtents
220|0 2 1 10:03:58|FAIL
410|0 2 1 10:03:58|IC End
400|0 4 1 10:03:58|IC Start
200|0 3 10:03:58|TP Start
520|0 3 00008627 1 1|Begin test of Component Interface function AccessibleComponent_contains
520|0 3 00008627 1 2|Expected ContainsResult = TRUE, got FALSE
520|0 3 00008627 1 3|End test of Component Interface function AccessibleComponent_contains
220|0 3 1 10:03:58|FAIL
410|0 4 1 10:03:58|IC End
400|0 5 1 10:03:58|IC Start
200|0 4 10:03:58|TP Start
520|0 4 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getLayer
520|0 4 00008627 1 2|Expected GetLayer = 3, got 0
520|0 4 00008627 1 3|End test of Component Interface function AccessibleComponent_getLayer
220|0 4 1 10:03:58|FAIL
410|0 5 1 10:03:58|IC End
400|0 6 1 10:03:58|IC Start
200|0 5 10:03:58|TP Start
520|0 5 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getMDIZOrder
520|0 5 00008627 1 2|Expected GetMDIZOrder = -2147483648, got 0
520|0 5 00008627 1 3|End test of Component Interface function AccessibleComponent_getMDIZOrder
220|0 5 1 10:03:58|FAIL
410|0 6 1 10:03:58|IC End
400|0 8 1 10:03:58|IC Start
200|0 6 10:03:58|TP Start
520|0 6 00008627 1 1|Begin test of Component Interface function AccessibleComponent_grabFocus
520|0 6 00008627 1 2|Expected GrabFocus = TRUE, got FALSE
520|0 6 00008627 1 3|End test of Component Interface function AccessibleComponent_grabFocus
220|0 6 1 10:03:58|FAIL
410|0 8 1 10:03:58|IC End
520|0 0 00008627 1 1|Begin ATK Component Interface test cleanup
520|0 0 00008627 1 2|End ATK Component Interface test cleanup
80|0 0 10:03:59|TC End, scenario ref 3-0
900|10:03:59|TCC End
-------------- next part --------------
--- ./tet_scen.lsb 2007-03-29 16:04:29.000000000 -0400
+++ ./tet_scen 2007-03-29 16:24:41.000000000 -0400
@@ -0,0 +1,5 @@
+all
+ "Starting ATK test suite"
+ "total tests in ATKComponent 6"
+ /atk/ATKComponent
+
--- ./run_tests.lsb 2007-03-29 16:16:41.000000000 -0400
+++ ./run_tests 2007-03-30 08:45:49.000000000 -0400
@@ -0,0 +1,15 @@
+#!/bin/sh
+touch .test_start
+/opt/lsb/bin/Xvfb :1 &
+echo "sleep so we can find the journal"
+sleep 5
+TET_ROOT=`pwd` DISPLAY=:1 tcc -e -p .
+killall Xvfb
+rm -f /tmp/.X1-lock
+JOURNAL=`find results -cnewer .test_start -name journal`
+if [ -n "$JOURNAL" ];then
+ tjreport -v $JOURNAL
+fi
+rm -f .test_start
+
+
--- ./ATKComponent.c.lsb 2007-03-30 08:39:15.000000000 -0400
+++ ./ATKComponent.c 2007-03-30 09:24:43.000000000 -0400
@@ -92,7 +92,7 @@ void Test_getPosition()
FunctionReturn = atk_component_set_position (ATKAccessibleComponent, SetXPosition, SetYPosition, CoordType);
if (FunctionReturn == FALSE)
- printf ("ATK Set Position return FALSE\n");
+ tet_printf ("ATK Set Position return FALSE");
}
/* get values from the GTK test widget using SPI calls */
@@ -112,8 +112,14 @@ void Test_getPosition()
/* check values and return appropriate TET code */
{
- if (GetXPosition != SetXPosition) tet_result (TET_FAIL);
- if (GetYPosition != SetYPosition) tet_result (TET_FAIL);
+ if (GetXPosition != SetXPosition) {
+ tet_printf("Expected GetXPosition %d, got %d", SetXPosition, GetXPosition);
+ tet_result (TET_FAIL);
+ }
+ if (GetYPosition != SetYPosition) {
+ tet_printf ("Expected GetYPosition %d, got %d", SetYPosition, GetYPosition);
+ tet_result (TET_FAIL);
+ }
tet_result (TET_PASS);
}
@@ -165,7 +171,7 @@ void Test_getExtents ()
FunctionReturn = atk_component_set_extents (ATKAccessibleComponent, SetXPosition, SetYPosition, SetWidth, SetHeight, CoordType);
if (FunctionReturn == FALSE)
- printf ("ATK Set Extents return FALSE\n");
+ tet_printf ("ATK Set Extents return FALSE");
/* width and height are allocated by the window manager */
SetWidth = 210;
@@ -191,11 +197,22 @@ void Test_getExtents ()
/* check values and return appropriate TET code */
{
- if (GetXPosition != SetXPosition) tet_result (TET_FAIL);
- if (GetYPosition != SetYPosition) tet_result (TET_FAIL);
- if (GetWidth != SetWidth) tet_result (TET_FAIL);
- if (GetHeight != SetHeight) tet_result (TET_FAIL);
-
+ if (GetXPosition != SetXPosition) {
+ tet_printf ("Expected GetXPosition = %d, got %d", SetXPosition, GetXPosition);
+ tet_result (TET_FAIL);
+ }
+ if (GetYPosition != SetYPosition) {
+ tet_printf ("Expected GetYPosition = %d, got %d", SetYPosition, GetYPosition);
+ tet_result (TET_FAIL);
+ }
+ if (GetWidth != SetWidth) {
+ tet_printf ("Expected GetWidth = %d, got %d", SetWidth, GetWidth);
+ tet_result (TET_FAIL);
+ }
+ if (GetHeight != SetHeight) {
+ tet_printf ("Expected GetHeight = %d, got %d", SetHeight, GetHeight);
+ tet_result (TET_FAIL);
+ }
tet_result (TET_PASS);
}
@@ -232,7 +249,7 @@ void Test_getSize ()
FunctionReturn = atk_component_set_size ((AtkComponent *) ATKAccessibleComponent, SetWidth, SetHeight);
if (FunctionReturn == FALSE)
- printf ("ATK Set Extents return FALSE\n");
+ tet_printf ("ATK Set Extents return FALSE");
if (debug == 1)
printf ("ATK Set width = %d, height = %d\n", SetWidth, SetHeight);
@@ -267,9 +284,14 @@ void Test_getSize ()
/* check values and return appropriate TET code */
{
- if (GetWidth != SetWidth) tet_result (TET_FAIL);
- if (GetHeight != SetHeight) tet_result (TET_FAIL);
-
+ if (GetWidth != SetWidth) {
+ tet_printf("Expected GetWidth = %d, got %d", SetWidth, GetWidth);
+ tet_result (TET_FAIL);
+ }
+ if (GetHeight != SetHeight) {
+ tet_printf("Expected GetHeight = %d, got %d", SetHeight, GetHeight);
+ tet_result (TET_FAIL);
+ }
tet_result (TET_PASS);
}
@@ -322,8 +344,10 @@ void Test_contains ()
/* check values and return appropriate TET code */
{
- if (ContainsResult == FALSE) tet_result (TET_FAIL);
-
+ if (ContainsResult == FALSE) {
+ tet_printf ("Expected ContainsResult = TRUE, got FALSE");
+ tet_result (TET_FAIL);
+ }
tet_result (TET_PASS);
}
@@ -365,8 +389,10 @@ void Test_getLayer ()
/* check values and return appropriate TET code */
{
- if (GetLayer != SetLayer) tet_result (TET_FAIL);
-
+ if (GetLayer != SetLayer) {
+ tet_printf ("Expected GetLayer = %d, got %d", SetLayer, GetLayer);
+ tet_result (TET_FAIL);
+ }
tet_result (TET_PASS);
}
@@ -409,8 +435,10 @@ void Test_getMDIZOrder ()
/* check values and return appropriate TET code */
{
- if (GetMDIZOrder != SetMDIZOrder) tet_result (TET_FAIL);
-
+ if (GetMDIZOrder != SetMDIZOrder) {
+ tet_printf ("Expected GetMDIZOrder = %d, got %d", SetMDIZOrder, GetMDIZOrder);
+ tet_result (TET_FAIL);
+ }
tet_result (TET_PASS);
}
@@ -511,8 +539,10 @@ void Test_grabFocus ()
/* check values and return appropriate TET code */
{
- if (GrabFocus == FALSE) tet_result (TET_FAIL);
-
+ if (GrabFocus == FALSE) {
+ tet_printf ("Expected GrabFocus = TRUE, got FALSE");
+ tet_result (TET_FAIL);
+ }
tet_result (TET_PASS);
}
--- ./tetexec.cfg.lsb 2007-03-29 16:18:31.000000000 -0400
+++ ./tetexec.cfg 2007-03-29 16:06:18.000000000 -0400
@@ -0,0 +1 @@
+TET_OUTPUT_CAPTURE=false
--- ./Makefile.lsb 2007-03-29 15:40:25.000000000 -0400
+++ ./Makefile 2007-03-30 08:35:02.000000000 -0400
@@ -2,12 +2,14 @@
TET_INCS = ${TET_ROOT}/inc/tet3
TET_LIBS = ${TET_ROOT}/lib/tet3
-CC = gcc
+CC ?= gcc
CCFLAGS = -g -O2
-INCLUDES = -DHAVE_CONFIG_H -DXTHREADS -D_REENTRANT -DXUSE_MTSAFE_API -DORBIT2=1 -pthread -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/at-spi-1.0 -I/usr/include/libbonobo-2.0 -I/usr/include/orbit-2.0 -I/usr/include/bonobo-activation-2.0 -I/usr/include/libgnome-2.0 -I/usr/include/libbonoboui-2.0 -I/usr/include/cairo -I${TET_INCS}
+INCLUDES = -DHAVE_CONFIG_H -DXTHREADS -D_REENTRANT -DXUSE_MTSAFE_API -DORBIT2=1 -pthread $(shell pkg-config --cflags gtk+-2.0 libgnomeui-2.0 atk cspi-1.0) -I${TET_INCS}
-LIBS = -Wl,--export-dynamic -pthread -lcspi -lspi -lgnomeui-2 -lpthread -lgnome-2 -latk-1.0 -lpango-1.0 -lgobject-2.0 -lbonobo-2 -lbonobo-activation -lORBit-2 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0
+LIBDIRS = $(shell pkg-config --libs-only-L gtk+-2.0 libgnomeui-2.0 atk cspi-1.0)
+
+LIBS = -Wl,--export-dynamic ${LIBDIRS} -pthread -lcspi -lspi -lgnomeui-2 -lpthread -lgnome-2 -latk-1.0 -lpango-1.0 -lgobject-2.0 -lbonobo-2 -lbonobo-activation -lORBit-2 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0
all: ATKComponent
@@ -26,3 +28,17 @@ ATKCommon.o: ATKCommon.c ATKCommon.h
ATKComponent.o: ATKComponent.c ATKCommon.h
${CC} $(INCLUDES) ${CCFLAGS} -c ATKComponent.c
+clean:
+ rm -f *.o
+ rm -f ATKComponent
+ rm -f tet_xres
+ rm -fr atk
+ rm -fr tet_tmp_dir
+ rm -fr results
+
+test: ATKComponent
+ rm -fr atk
+ mkdir atk
+ cp ATKComponent atk
+ ./run_tests
+
More information about the Accessibility-atspi
mailing list