[lsb-discuss] Certification for 3.1 update
Wichmann, Mats D
mats.d.wichmann at intel.com
Tue Mar 6 08:43:25 PST 2007
> With the new test framework coming out soon, it is probably best to
> see how we can make the existing process work for 3.1 update.
install "lsb-dtk-manager" and go from there....
> First, I downloaded distro testkit. While there were yum and apt
> repositories, I used a tarball on a Fedora installation and everything
> installed and ran fairly easily. Comments on running the distro
> - Downloaded the testkit bundle. I suspect downloading
> and installing individual tests would work as well, but I
> couldn't see a reason to do it. Individual tests might
> be good for debugging an environment, but for a quick go-
> no-go test, the bundle was quick.
> - Ran liblibchk. No problems.
> - Ran lsbcmdchk. No problems.
> - Set up and ran the runtime tests. No problems. Took
> several hours. Was mildly annoying to have to keep entering
> the root password. Using a virtual terminal may not be
> intuitive to the user. Produced a journal of over 45K lines.
yes, it's a lot of fun. the issue, of course, is that you can't
shell-script responses to password prompts, because of the way
those are implemented, and all of this stuff happens pre-test, in
the run_tests shell script which will eventually kick off the
actual tests. the solution most people pick is to use expect or
a related tool. this was not done for runtime-test because it
would be an additional requirement for running the tests that
is beyond what LSB itself requires. I'm not sure whether or not
we should give up and just add the use of expect, or if there's
something else we could use to script this. since several
of the other tests (c++ and desktop, to name two) already require
lsb-python we could possibly use pyexpect, the python-native version
it's certainly worth discussing!
> - Set up and ran the lsb-c++ test suite. The topic of locales
> could certainly cause confusion. I had all the locales installed
> on the system, but kept getting errors like:
> It seems that the de_DE locale is not set up correctly in your system.
> Please check and rerun run_test again.
> In some discussions with Mats, it is not deterministic whether
> this warning is fatal or non-fatal to the test. We should talk
> about this at the call tomorrow.
I believe the story is that the released tests have this as a
fatail error. I don't think the fix to make it non-fatal went
in until after the 3.1 release, which means it's only in the beta
copies, and will be released next week.
> - Ran the lsb-vsw4 test suite. Had to load xvfb. The documentation
> does not reflect the output of the test (seems to have changed).
Which documentation are you referring to, and we'll go check.
More information about the lsb-discuss