[cgl_discussion] Validation Sub-team Meeting Minutes for 04/18/2002

[Lynch, Rusty]

Validation Sub-team Meeting Minutes
Thursday April 18, 2002

Attendees:
·	Craig Thomas (OSDL)
·	Stephanie Glass (IBM)
·	Julie Fleischer (Intel)
·	Rusty Lynch (Intel)

Action Items:
·	Rusty to talk with PoC chair (Khalid) about his strategy for 
      publishing the PoC website, and take this into consideration 
      on deciding the proper way to roll out the real validation 
      sub-team website.
·	Craig to write a better-phrased description for requirement 
      #3.1 (LSB).
·	Rusty to review requirements #15 through #35.x
·	Craig to review requirements #50 through #56
·	Julie to review requirements #70 through #70.5
·	Change web site database so that a feature is associated with 
      a specific requirement

Agenda Items:
·	Proposal for web site design: 
	o	Previewed collaboration web site design and discussed the 
            implications on how the validation sub-team would do it's work
	o	A request was made to link each feature with a specific
requirement.  
            With type of data, we create a web page the provided a matrix of

            which requirements are covered by which validation suite
features.
	o	Several new fields were brainstormed for tracking/reviewing 
            requirements.  Each will need to be included in the web site
database, 
            and the requirements tracking tool (i.e. the web page GUI to the

            requirements tracking data.)
·	Permanent location for validation sub-group web site:
	o	Discussed the different options available to the validation
sub-group.  
            These include hosting the site on an OSDL server with an
associated 
            sourceforge project.
	o	It looks like the only other complex web site (i.e. needing
more then 
            just some flashing text and a list of links) spawned by the
CGLWG appears 
            to be the site to be created by the PoC group.  It makes a lot
of sense 
            for both CGLWG spawned sites to follow the same strategy.
	o	Whatever the end outcome, we need to resolve this very
quickly.
·	Process for reviewing requirements: A complete list of data to
capture on each 
      requirement is contained at the end of the minutes.
	o	Discussed our strategy for accomplishing coverage of the
requirements.  
            The group agreed that a breadth verse depth decision needed to
be made 
            case-by-case for each requirement.  As a result, the new field
will be 
            added to the requirements tracking table.
	o	The group will individually split up the requirements for
review, and then 
            provide the feedback to Rusty (or if the website comes up then
directly 
            input the data to the tracking tool). Feedback will then be feed
back to 
            the requirements group as official feedback from the validation
subgroup. 
	o	Decided to add a field indicating owner ship, with an
associated URL for 
            requirements that are covered by external validation suites
(that this 
            group with not be touching).
	o	Decided to capture if the requirement requires a special
configuration.
            In the case of special configuration requirements, the OSDL lab
infrastructure
            would be useful.
	o	Decided to capture if the requirement can be automated.

List of items to capture for each requirement: (Each of these will be
contained in 
                                                the requirements tracking
tool as soon 
                                                as it is updated.)
1. Available Suite (YES/NO):  Is there an existing open source validation
suite 
                              that can be used to validate this requirement?
2. Testable (YES/NO): Is the requirement testable.
3. Traceable (YES/NO): 
4. Complete (YES/NO): Is the requirement complete, or is there missing
information.  
                         For example,  "networking must work" is not a
complete requirement.
5. Effort (SMALL, MEDIUM, LARGE): Gut feel for how big an effort is required
to 
                                  validate this requirement.  A requirement
mandating 
                                  all IPv6 RFC compliance would be 'LARGE',
and a 
                                  requirement mandating the "/dev/watchdog"
interface to 
                                  work as documented in a reference
specification containing 
                                  just a few features would be 'SMALL'.  As
we learn more about 
                                  the specific requirement we might need to
adjust this value.
6. Coverage Strategy (DEPTH/BREADTH): Gut feel for how we should approach
this validation work. 
                                      As we learn more about the specific
requirement we might 
                                      need to adjust this value.
7. Ownership (OSDL, External): By ownership, I mean that this working group
will add a 
                               validation suite to our framework, not that
we will need 
                               to create a validation suite from scratch.
For example, 
                               requirement #3.1 requires LSB compliance.
Our approach for 
                               validating this requirement is to just have a
"check box" 
                               asking if the distribution has followed
whatever procedures 
                               LSB has set-up for proving compliance, so
running the automated 
                               validation framework will not validate the
LSB requirement.
8. Special Configuration Required (YES/NO): Are there special configurations
that need to be 
                                            made before running an automated
validation suite? 
                                            Examples are all the
requirements having to do with 
                                            the various aspects of hot-swap
support.
9. Can be Automated (YES/NO): Can an automated validation suite test this
requirement?
10. Comments (free text): Any other important notations that can not be
contained in the 
                          above fields.

NOTE: Once we have this data in the database, we can query for types of
requirements that 
      need feedback to the requirements group.