[Accessibility-atspi] atktest-0.1.1

Stew Benedict stewb at aysenterprises.com
Fri Mar 30 07:33:54 PDT 2007


OK,

Spent some time on this. I'll re-iterate I'm fairly ignorant on 
accessibility issues, so feel free to educate me as needed :)

I've included a patch which addresses:

1) More feedback in the journal as to why the test fails
2) Tweak the Makefile to:
	a) make it more portable
	b) just some housekeeping (allow lsbcc some day, make clean)
	c) allow us to do testing as part of the build process

2.c is part of the "lessons learned" in the 3.1.1 update work. We 
repeatedly got bit late in the game by tests which changed behavior either 
due to upstream changes or changes on our build hosts(s). Moving forward, 
we're hoping to be able to catch many of these things earlier, using 
autobuilds/autotests. Being able to do "make test" in the package build 
and having the autobuild fail would give us an early heads up.

How this particular test will fit into the packaging is still up in the 
air, but for the interim having "make test" helps me in working with it.

Issues:
--------------------------------------------------------------------
Test Results:
--------------------------------------------------------------------

I get 6 of 6 failures on SLES10 (using lsb-xvfb as the X environment). 
I know there was some question about using the lsb-tet3-lite vs upstream, 
but using either one didn't seem to make any difference here. I did:

TET_ROOT=/opt/lsb-tet3-lite make clean
TET_ROOT=/opt/lsb-tet3-lite make test

and

TET_ROOT=../tet37a make clean
TET_ROOT=../tet37a make test

with ../tet37 being a locally built copy per the README

No difference in the test results using either tet source.
Is there something that needs to be done to setup the environment for 
testing?

Journal with the test results also attached.

----------------------------------------------------------------------
Build fails with lsbcc:
----------------------------------------------------------------------
CC=lsbcc TET_ROOT=/opt/lsb-tet3-lite make

-- snip some output --
-- end snip --

lsbcc -o ATKComponent \
        ATKCommon.o ATKComponent.o \
        -Wl,--export-dynamic -L/usr/X11R6/lib -L/opt/gnome/lib   -pthread 
-lcspi -lspi -lgnomeui-2 -lpthread -lgnome-2 -latk-1.0 -lpango-1.0 
-lgobject-2.0 -lbonobo-2 -lbonobo-activation -lORBit-2 -lgmodule-2.0 
-lgthread-2.0 -lglib-2.0   \
        /opt/lsb-tet3-lite/lib/tet3/tcm.o \
        /opt/lsb-tet3-lite/lib/tet3/libapi.a
/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/bin/ld: 
cannot find -lcspi
collect2: ld returned 1 exit status
make: *** [ATKComponent] Error 1

It looks like we're using things not part of LSB yet.
Part of our open bugs is to get the tests so they can build with lsbcc.

LIBS = -Wl,--export-dynamic -pthread -lcspi -lspi -lgnomeui-2 -lpthread 
-lgnome-2 -latk-1.0 -lpango-1.0 -lgobject-2.0 -lbonobo-2 -lbonobo-activation 
-lORBit-2 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0

mysql> select * from Library where Lname like '%bonobo%' or Lname like 
'%spi%' or Lname like '%ORB%' or LName like '%gnome%';
Empty set (0.00 sec)

Do we intend to add any of this to the spec, or would we need to look at 
compiling these libs in statically? (cspi/spi seems like it might be key 
to what we're trying to test).

The library dependency of this single app is pretty long:

SLES10-presario30:~/rpm/BUILD/atktest-0.1.1-lsb/src$ ldd ATKComponent | wc 
-l
67

Not that big of deal I suppose, but I imagine we may hit issues on distros 
lacking some of the libs and having the test fail, with no journal output, 
if we leave it linked as is.

------------------------------------------------------------------

So, I guess what I really need at this point is some guidance on how to 
get the test to PASS, and what if any, inclusions into the LSB spec might 
be associated with this test.

Regards,
-- 
Stew Benedict
-------------- next part --------------
0|3.7-lite 10:03:58 20070330|User: stew (500) TCC Start, Command line: tcc -e -p .
5|Linux presario30.e-artisan.org 2.6.17-11mdv #1 SMP Thu Feb 22 13:56:30 CET 2007 i686|System Information
20|/home/stew/rpm/BUILD/atktest-0.1.1-lsb/src/./tetexec.cfg 1|Config Start
30||TET_OUTPUT_CAPTURE=false
30||TET_API_COMPLIANT=True
30||TET_PASS_TC_NAME=False
30||TET_VERSION=3.7-lite
40||Config End
70||"Starting ATK test suite"
70||"total tests in ATKComponent 6"
10|0 /atk/ATKComponent 10:03:58|TC Start, scenario ref 3-0
15|0 3.7-lite 6|TCM Start
520|0 0 00008627 1 1|Begin Component Interface test startup
520|0 0 00008627 1 2|End Component Interface test startup
400|0 1 1 10:03:58|IC Start
200|0 1 10:03:58|TP Start
520|0 1 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getPosition
520|0 1 00008627 1 2|ATK Set Position return FALSE
520|0 1 00008627 1 3|Expected GetXPosition 500, got 0
520|0 1 00008627 1 4|Expected GetYPosition 150, got 0
520|0 1 00008627 1 5|End test of Component Interface function AccessibleComponent_getPosition
220|0 1 1 10:03:58|FAIL
410|0 1 1 10:03:58|IC End
400|0 2 1 10:03:58|IC Start
200|0 2 10:03:58|TP Start
520|0 2 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getExtents
520|0 2 00008627 1 2|ATK Set Extents return FALSE
520|0 2 00008627 1 3|Expected GetXPosition = 100, got 0
520|0 2 00008627 1 4|Expected GetYPosition = 250, got 0
520|0 2 00008627 1 5|Expected GetWidth = 210, got 0
520|0 2 00008627 1 6|Expected GetHeight = 229, got 0
520|0 2 00008627 1 7|End test of Component Interface function AccessibleComponent_getExtents
220|0 2 1 10:03:58|FAIL
410|0 2 1 10:03:58|IC End
400|0 4 1 10:03:58|IC Start
200|0 3 10:03:58|TP Start
520|0 3 00008627 1 1|Begin test of Component Interface function AccessibleComponent_contains
520|0 3 00008627 1 2|Expected ContainsResult = TRUE, got FALSE
520|0 3 00008627 1 3|End test of Component Interface function AccessibleComponent_contains
220|0 3 1 10:03:58|FAIL
410|0 4 1 10:03:58|IC End
400|0 5 1 10:03:58|IC Start
200|0 4 10:03:58|TP Start
520|0 4 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getLayer
520|0 4 00008627 1 2|Expected GetLayer = 3, got 0
520|0 4 00008627 1 3|End test of Component Interface function AccessibleComponent_getLayer
220|0 4 1 10:03:58|FAIL
410|0 5 1 10:03:58|IC End
400|0 6 1 10:03:58|IC Start
200|0 5 10:03:58|TP Start
520|0 5 00008627 1 1|Begin test of Component Interface function AccessibleComponent_getMDIZOrder
520|0 5 00008627 1 2|Expected GetMDIZOrder = -2147483648, got 0
520|0 5 00008627 1 3|End test of Component Interface function AccessibleComponent_getMDIZOrder
220|0 5 1 10:03:58|FAIL
410|0 6 1 10:03:58|IC End
400|0 8 1 10:03:58|IC Start
200|0 6 10:03:58|TP Start
520|0 6 00008627 1 1|Begin test of Component Interface function AccessibleComponent_grabFocus
520|0 6 00008627 1 2|Expected GrabFocus = TRUE, got FALSE
520|0 6 00008627 1 3|End test of Component Interface function AccessibleComponent_grabFocus
220|0 6 1 10:03:58|FAIL
410|0 8 1 10:03:58|IC End
520|0 0 00008627 1 1|Begin ATK Component Interface test cleanup
520|0 0 00008627 1 2|End ATK Component Interface test cleanup
80|0 0 10:03:59|TC End, scenario ref 3-0
900|10:03:59|TCC End
-------------- next part --------------
--- ./tet_scen.lsb	2007-03-29 16:04:29.000000000 -0400
+++ ./tet_scen	2007-03-29 16:24:41.000000000 -0400
@@ -0,0 +1,5 @@
+all
+	"Starting ATK test suite"
+	"total tests in ATKComponent 6"
+	/atk/ATKComponent
+
--- ./run_tests.lsb	2007-03-29 16:16:41.000000000 -0400
+++ ./run_tests	2007-03-30 08:45:49.000000000 -0400
@@ -0,0 +1,15 @@
+#!/bin/sh
+touch .test_start
+/opt/lsb/bin/Xvfb :1 &
+echo "sleep so we can find the journal"
+sleep 5
+TET_ROOT=`pwd` DISPLAY=:1 tcc -e -p .
+killall Xvfb
+rm -f /tmp/.X1-lock
+JOURNAL=`find results -cnewer .test_start -name journal`
+if [ -n "$JOURNAL" ];then
+  tjreport -v $JOURNAL
+fi
+rm -f .test_start
+
+
--- ./ATKComponent.c.lsb	2007-03-30 08:39:15.000000000 -0400
+++ ./ATKComponent.c	2007-03-30 09:24:43.000000000 -0400
@@ -92,7 +92,7 @@ void Test_getPosition()
 		FunctionReturn = atk_component_set_position (ATKAccessibleComponent, SetXPosition, SetYPosition, CoordType);
 
 		if (FunctionReturn == FALSE)
-			printf ("ATK Set Position return FALSE\n");
+			tet_printf ("ATK Set Position return FALSE");
 	}
 
 	/* get values from the GTK test widget using SPI calls */
@@ -112,8 +112,14 @@ void Test_getPosition()
 
 	/* check values and return appropriate TET code */
 	{
-		if (GetXPosition != SetXPosition) tet_result (TET_FAIL);
-		if (GetYPosition != SetYPosition) tet_result (TET_FAIL);
+		if (GetXPosition != SetXPosition) {
+			tet_printf("Expected GetXPosition %d, got %d", SetXPosition, GetXPosition);
+			tet_result (TET_FAIL);
+		}
+		if (GetYPosition != SetYPosition) {
+			tet_printf ("Expected GetYPosition %d, got %d", SetYPosition, GetYPosition);
+			tet_result (TET_FAIL);
+		}
 
 		tet_result (TET_PASS);
 	}
@@ -165,7 +171,7 @@ void Test_getExtents () 
 		FunctionReturn = atk_component_set_extents (ATKAccessibleComponent, SetXPosition, SetYPosition, SetWidth, SetHeight, CoordType);
 
 		if (FunctionReturn == FALSE)
-			printf ("ATK Set Extents return FALSE\n");
+			tet_printf ("ATK Set Extents return FALSE");
 
 		/* width and height are allocated by the window manager */
 		SetWidth = 210;
@@ -191,11 +197,22 @@ void Test_getExtents () 
 
 	/* check values and return appropriate TET code */
 	{
-		if (GetXPosition != SetXPosition) tet_result (TET_FAIL);
-		if (GetYPosition != SetYPosition) tet_result (TET_FAIL);
-		if (GetWidth != SetWidth) tet_result (TET_FAIL);
-		if (GetHeight != SetHeight) tet_result (TET_FAIL);
-
+		if (GetXPosition != SetXPosition) {
+			tet_printf ("Expected GetXPosition = %d, got %d", SetXPosition, GetXPosition);
+			tet_result (TET_FAIL);
+		}
+		if (GetYPosition != SetYPosition) {
+			tet_printf ("Expected GetYPosition = %d, got %d", SetYPosition, GetYPosition);
+			tet_result (TET_FAIL);
+		}
+		if (GetWidth != SetWidth) {
+			tet_printf ("Expected GetWidth = %d, got %d", SetWidth, GetWidth);
+			tet_result (TET_FAIL);
+		}
+		if (GetHeight != SetHeight) {
+			tet_printf ("Expected GetHeight = %d, got %d", SetHeight, GetHeight);
+			tet_result (TET_FAIL);
+		}
 		tet_result (TET_PASS);
 	}
 
@@ -232,7 +249,7 @@ void Test_getSize () 
 		FunctionReturn = atk_component_set_size ((AtkComponent *) ATKAccessibleComponent, SetWidth, SetHeight);
 
 		if (FunctionReturn == FALSE)
-			printf ("ATK Set Extents return FALSE\n");
+			tet_printf ("ATK Set Extents return FALSE");
 
 		if (debug == 1)
 			printf ("ATK Set width = %d, height = %d\n", SetWidth, SetHeight);
@@ -267,9 +284,14 @@ void Test_getSize () 
 
 	/* check values and return appropriate TET code */
 	{
-		if (GetWidth != SetWidth) tet_result (TET_FAIL);
-		if (GetHeight != SetHeight) tet_result (TET_FAIL);
-
+		if (GetWidth != SetWidth) {
+			tet_printf("Expected GetWidth = %d, got %d", SetWidth, GetWidth);
+			tet_result (TET_FAIL);
+		}
+		if (GetHeight != SetHeight) {
+			tet_printf("Expected GetHeight = %d, got %d", SetHeight, GetHeight);
+			tet_result (TET_FAIL);
+		}
 		tet_result (TET_PASS);
 	}
 
@@ -322,8 +344,10 @@ void Test_contains () 
 
 	/* check values and return appropriate TET code */
 	{
-		if (ContainsResult == FALSE) tet_result (TET_FAIL);
-
+		if (ContainsResult == FALSE) {
+			tet_printf ("Expected ContainsResult = TRUE, got FALSE");
+			tet_result (TET_FAIL);
+		}
 		tet_result (TET_PASS);
 	}
 
@@ -365,8 +389,10 @@ void Test_getLayer () 
 
 	/* check values and return appropriate TET code */
 	{
-		if (GetLayer != SetLayer) tet_result (TET_FAIL);
-
+		if (GetLayer != SetLayer) {
+			tet_printf ("Expected GetLayer = %d, got %d", SetLayer, GetLayer);
+			tet_result (TET_FAIL);
+		}
 		tet_result (TET_PASS);
 	}
 
@@ -409,8 +435,10 @@ void Test_getMDIZOrder () 
 
 	/* check values and return appropriate TET code */
 	{
-		 if (GetMDIZOrder != SetMDIZOrder) tet_result (TET_FAIL);
-
+		if (GetMDIZOrder != SetMDIZOrder) {
+			tet_printf ("Expected GetMDIZOrder = %d, got %d", SetMDIZOrder, GetMDIZOrder);
+			tet_result (TET_FAIL);
+		}
 		tet_result (TET_PASS);
 	}
 
@@ -511,8 +539,10 @@ void Test_grabFocus ()
 
 	/* check values and return appropriate TET code */
 	{
-		if (GrabFocus == FALSE) tet_result (TET_FAIL);
-
+		if (GrabFocus == FALSE) {
+			tet_printf ("Expected GrabFocus = TRUE, got FALSE");
+			tet_result (TET_FAIL);
+		}
 		tet_result (TET_PASS);
 	}
 
--- ./tetexec.cfg.lsb	2007-03-29 16:18:31.000000000 -0400
+++ ./tetexec.cfg	2007-03-29 16:06:18.000000000 -0400
@@ -0,0 +1 @@
+TET_OUTPUT_CAPTURE=false
--- ./Makefile.lsb	2007-03-29 15:40:25.000000000 -0400
+++ ./Makefile	2007-03-30 08:35:02.000000000 -0400
@@ -2,12 +2,14 @@
 TET_INCS = ${TET_ROOT}/inc/tet3
 TET_LIBS = ${TET_ROOT}/lib/tet3
 
-CC = gcc
+CC ?= gcc
 CCFLAGS = -g -O2
 
-INCLUDES = -DHAVE_CONFIG_H -DXTHREADS -D_REENTRANT -DXUSE_MTSAFE_API -DORBIT2=1 -pthread -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/at-spi-1.0 -I/usr/include/libbonobo-2.0 -I/usr/include/orbit-2.0 -I/usr/include/bonobo-activation-2.0 -I/usr/include/libgnome-2.0 -I/usr/include/libbonoboui-2.0 -I/usr/include/cairo -I${TET_INCS}
+INCLUDES = -DHAVE_CONFIG_H -DXTHREADS -D_REENTRANT -DXUSE_MTSAFE_API -DORBIT2=1 -pthread $(shell pkg-config --cflags gtk+-2.0 libgnomeui-2.0 atk cspi-1.0) -I${TET_INCS}
 
-LIBS = -Wl,--export-dynamic -pthread -lcspi -lspi -lgnomeui-2 -lpthread -lgnome-2 -latk-1.0 -lpango-1.0 -lgobject-2.0 -lbonobo-2 -lbonobo-activation -lORBit-2 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0  
+LIBDIRS = $(shell pkg-config --libs-only-L gtk+-2.0 libgnomeui-2.0 atk cspi-1.0)
+
+LIBS = -Wl,--export-dynamic ${LIBDIRS} -pthread -lcspi -lspi -lgnomeui-2 -lpthread -lgnome-2 -latk-1.0 -lpango-1.0 -lgobject-2.0 -lbonobo-2 -lbonobo-activation -lORBit-2 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0  
 
 all: ATKComponent
 
@@ -26,3 +28,17 @@ ATKCommon.o: ATKCommon.c ATKCommon.h
 ATKComponent.o: ATKComponent.c ATKCommon.h 
 	${CC} $(INCLUDES) ${CCFLAGS} -c ATKComponent.c
 
+clean:
+	rm -f *.o
+	rm -f ATKComponent
+	rm -f tet_xres
+	rm -fr atk
+	rm -fr tet_tmp_dir
+	rm -fr results
+
+test:	ATKComponent
+	rm -fr atk
+	mkdir atk
+	cp ATKComponent atk
+	./run_tests
+


More information about the Accessibility-atspi mailing list