[cgl_discussion] Minutes taken from the CGL Valildation team meeting 9-4-02
craiger at osdl.org
Wed Sep 4 16:14:13 PDT 2002
Craig Thomas phone: 503-626-2455 ext. 33
Open Source Development Labs email: craiger at osdl.org
15275 SW Koll Pkwy, Suite H
Beaverton, OR 97006
-------------- next part --------------
The discussion started with Julie's enhancements to the validation
framework directives to support a --abat switch in the wrapper.sh
script for each validation suite. Rusty then descirbed the ABAT
execution process at a high level: ABAT is a test harness loaded onto
a system after a new kernel is loaded. Once rebooted, abat is started
and it looks to a specific directory to run tests. Each test is run,
results are reported to STDOUT and mailed to a "receiving" system.
Each validation suite would include code in its wrapper.sh script so
that the test maintainer could identify a set of "smoke" tests for that
suite that ABAT would be able to run as a quick sanity check for the
We came up with idea about reworking ABAT so that it has the same
consistent look and feel as the rest of the suite in terms of the
results output. From there other ideas came to mind regarding how to
add enhancemetns to ABAT. The plan is to create a 'ToDo' list for
feature improvements for the Validation suite and ABAT. Initial
+ modify abat to produce results the same way as the validation suite
+ enhance abat to utilize a separate monitor/driver that can tell when
a system reboots after the load of a new kernel
We discussed whether or not ABAT should be moved under the cglvalidation
CVS tree or kept on sourceforge as an open project. The advantages of
bringing ABAT under the CGL validation tree is that the two could be
more tightly coupled and ABAT could take advantage of the defined
directory layout of the validation suite. The disadvantage would be
extra code to maintain. It was later thought that it would be best to
keep ABAT separate from the CGL validation suite to allow a more general
use of ABAT.
In discussing the status of integrating the CGL validation suite into
STP, Craig stated that no work has gone into adding the suite to STP, but
initial investigation has occurred. Resources at OSDL are on other
projects at this time, but it is expected that by next week, work can begin
to create a selectable test called cgl_validation (or something similar)
that would trigger a validation run with a seleted number of CGL tests.
The plan is to create the entire scaffolding to make it easy to add the
CGL test suites later. Craig then went into a descirption of the STP
framework and process: STP is web based, and is generally used in two
scenario 1: developer builds new kernel based on new patches. Uploads
kernel onto osdl.org and then goes to the STP web page and logs on (need
to be an OSDL associate -- which is a free signup). Once logged in, the
tester selects 'new test' link and then chooses the kernel to run, the
type of system, and the test to run (which in this case would be a cgl
test) then submits the run. STP automatically queues the test and runs
it. STP wipes the disks clean on the system under test, loads the new
kernel, loads the selected test, then executes. Results are captured
and the user is notified via email when the test is complete. The tester
can then check the results though the OSDL web site.
scenario 2: a tester wants to select any kernel build and run all of
the cgl tests. STP can be set up to automatically run the entire suite
of tests when a new kernel is build. We would want to configure STP to
do this (or at least run the set of ABAT tests for a smoke test).
Abat and STP can not handle multiple nodes at this time. Work needs
to be done here. Would STP push the XML output? Would STP or CGL
need to change to support the validation suite? These questions need
to be addressed in order to allow the tools we use to work within the
specifiations of the validation suite.
Need to create a 'todo' list for ABAT features and validattion suite
features rather than logging defects.
Discussion of the developer.org validation page: The projects that are
currently defined need to have the other projects and requirements added
to the web site. There was a thought about sending an email about cross
referencing the current projects with those identified in the CGL
project management milestones and with the requirements document. Julie
remembered an email already sent out that did preliminary work for
identifying missing projects on the validation web page. Rusty noted that
work needs to be done to update the validation web site. Therefore, a
cross check will be postponed until the validation web site could be
- integrate the new requriements approved in 8/23 minutes.
Craig will write a strategy plan for using STP for the validation suites.
We never had a chance to cover the status of the current maintainers
for the various validation suites. Our conference call ended.
More information about the cgl_discussion