[Cgl_discussion] Draft outline of the validation framework specification

Robert Hubbard rhubbard at cisco.com
Fri Mar 29 15:17:17 PST 2002


H Craig ,
Craig Thomas wrote:

>Below is a draft outline of content that would be included in the
>CGLinux Validation Framework Specification.  Although it is a bit
>early to finalize the content, it may be sufficient to distribute 
>to the groups and solicit feedback.
>

This is an excellent start . I am trying to understand the interaction 
if any with this charter here and the Service availability Forum and how 
the two combined areas could work in conjunction if at all . There are 
many members on the S.A forum that potentially would be interested in 
the same area here .

> http://www.saforum.org



Regards


C U

Bob.


>
>
>
>Draft Outline of document "Implementing the CGL Validation
>Framework and Validation Suites"
>
>Possible Subjects to cover:
>
> 1.  Introduction
>       a) Carrier Grade Linux project description, problem
>          to be solved, etc.
>       b) Validation Framework group and charter
>       c) Scope of this document
>             + defines the framework and tools
>             + defines the test categories
>             + defines the roadmap and milestones for test development
>       d) note that this doc is evolving; changes are made as
>          requirements change, or as milestones are completed
>
> 2.  Requirements of Carrier Grade Linux
>       a) describe test approach and methodology
>             + provide robustness to carrier grade systems
>               running linux
>             + provides performance benchmarks
>             + provides certification?
>       b) define test categories - provide a brief prose description of
>          the kinds of tests that are need to support the requirements
>          categories defined in the requirements document
>             + performance
>             + standards
>             + availability
>             + interoperability
>             + install/reboot
>             + etc.
>
> 3.  Framework Architecture
>       a) environment needed to run tests
>       b) tools used to execute tests
>       c) the test framework (TET, PAN, STAF, etc)
>       d) File system structure of test environment (results,
>          wrappers, logs, test cases, etc)
>
> 4.  Test Suite descriptions
>       a) detailed each test category and list the actual tests
>          contained within that category.
>            + describe each test in some detail
>            + cross reference each test with the requirement that
>              it satisfies
>       b) list tests that still need to be developed (if not created)
>            + describe what the test needs to accomplish to satisfy
>              a requirement
>            + provide a milestone date when it would need to be
>              complete and delivered to the test framework
>
> 5.  Execution of the Test Framework
>       a)  downloading the Framework
>       b)  execution of the tests
>       c)  analyzing the results
>       d)  running the framework at OSDL under STP (as a test option)
>       e)  posting results to the Carrier Grade Linux community
>
> 6.  Contributing Tests to the Framework
>       a) how to contribute tests
>       b) how to validate tests
>            + code reviews?
>            + successful execution on UP, 2 proc at minimum?
>       c) how to notify the Carrier Grade Linux community that new
>          tests exist
>       d) procedure for code check-ins
>       e) procedure for updating this framework document (test
>          case description and the requirements it satisfies,
>          roadmap table)
>
> 7.  Controling release versions of the framework
>       a) policies for releasing
>            + semi-anual, quarterly, etc.
>       b) policies of what is bundled and released
>       c) notification of new framework version release
>
> 8.  Roadmap of tests under development
>       a) table of all tests and their current status
>            + complete
>            + in progress
>            + not started
>       b) general dates for estimated completion of deliverables
>
> 9.  References and links to other projects
>       a) references to standards to be complied with
>       b) references of test projects
>            + LSB
>            + LTP
>            + OSDL STP
>
>10.  Revision History (could go at the top)
>
>11.  Appendicies
>
>
>





More information about the cgl_discussion mailing list