[agl-discussions] [RFC] CIAT Continuous Integration and Automated Test
Paul Sherwood
paul.sherwood at codethink.co.uk
Fri Jul 24 15:28:01 UTC 2015
Hi folks,
GENIVI Tools team has undertaken to work towards establishing an open
infrastructure to provide build and some automated testing of GENIVI
outputs (including ongoing versions of individual GENIVI projects,
GENIVI Baselines and GENIVI Demo Platforms). We hope that some or
possibly all of this work may be adopted by AGL too.
Below is a first draft "Statement of Work' for this - we would
appreciate any feedback and suggestions to improve it.
br
Paul
SoW: Continuous Integration and Automated Test
==============================================
Background
----------
Various open source 'umbrella' projects including GENIVI and AGL have a
need to establish and maintain public-facing infrastructure for
Continuous Integration and Automated Testing of integrated sets of open
source software.
This aims to document generalised requirements for CIAT in a 'Statement
of Work' form, so that actual implementations can be planned and
delivered. It is hoped that a workable solution can be found which
satisfies requirements for all of these organisations, but it is
expected that each may have further specific requirements, and may for
its own reasons choose alternative approaches for some or all of the
parts of this infrastructure to satisfy organisation-specific goals.
The content here is to be offered to all of the mentioned organisations
(and more widely) for consideration. It is recognised that maintaining a
CIAT infrastructure can be of significant engineering value, but
involves ongoing work and cost. Therefore collaboration on a generally
useful approach can offer particular benefits both in terms reduced cost
per project, and increased quality through (for example)
cross-validation and re-use of tests.
Scope
-----
CIAT includes
* collection of source code from upstreams including new versions as
required on an ongoing basis
* automated instructions for building the upstream components including
their dependencies
* automated instructions for installing the built upstream components
into an integrated 'distro'
* Continuous Integration pipeline which executes the automated
instructions automatically, either
* on user demand, or
* triggered by changes to the instructions or components themselves
* ability to include binary input artifacts (eg drivers or proprietary
executables) into the CI pipeline
* publication of the built/integrated component and distro artefacts
generated by the CI pipeline
* Automated Test pipeline which executes sets of tests on deployed
instances of the integrated software
* publication of the results of the automated tests, including at least
* pass/fail
* complete log files
* built artifacts
* demonstration of license compliance, including
* ability to add licence validation to the pipeline
* ensuring availability of source code for all published artefacts
* machanism for formal code review prior to merging of changes, either
* before build and test or
* after build and test as candidate (preferred)
Requirements
------------
* all of the CIAT software tooling must be FOSS
* any custom work to make the solution fit specific use-cases for
GENIVI and/or AGL must be FOSS
* where possible the solution should align with the upstream developers
for its components
* instructions for creation and setup of a working instance of CIAT
infrastructure must be provided
* minimal manual maintenance - the system should be mostly automated:
* minimal effort for license compliance (eg scripted license
checking, publish applicable source for all artefacts)
* developer/integrator engineers can trigger build/test using git
push.
* CIAT must support
* multiple automated test frameworks (eg FitNesse, Phoronix, LTSI
Test Project)
* testing scenarios on multiple architectures (eg ARMv7, x86_64,
MIPS) for both virtual machines (eg QEMU) and actual hardware
* multiple build systems (eg autotools, cmake, qmake, bitbake)
Deliverables
------------
* A working implementation of the CIAT infrastructure
* source code for any and all custom software required to achieve the
CIAT solution, published in Git at an established Git hosting service
* Instructions for creation of further instances of the CIAT
More information about the automotive-discussions
mailing list