[Fuego] [PATCH] LTP_one_test: create a Fuego test for a single LTP test
Liu, Wenlong
liuwl.fnst at cn.fujitsu.com
Mon Apr 16 05:37:45 UTC 2018
> -----Original Message-----
> From: Tim.Bird at sony.com [mailto:Tim.Bird at sony.com]
> Sent: Saturday, April 07, 2018 3:11 AM
> To: Liu, Wenlong/刘 文龙 <liuwl.fnst at cn.fujitsu.com>;
> daniel.sangorrin at toshiba.co.jp; fuego at lists.linuxfoundation.org
> Subject: [PATCH] LTP_one_test: create a Fuego test for a single LTP test
>
> Hey Daniel and Liu,
>
> Here is a test that runs an individual LTP test. It's now in my master
> branch. The reason for pointing this out is to get your feedback on the
> approach of writing a test that uses a sub-set of LTP.
> Note the use of 'ftc run-test' with the '-p b' parameter, to build LTP
> if needed. This avoids having to share build script details between this
> test and LTP.
>
> I'm thinking this approach might be useful to split out the OpenPosix and
> realtime subsets of LTP. Also, it might work for handling setup and cleanup
> for subtest groups or individual tests, like the network-related tests or
> smack.
>
> For example, instead of doing smack-specific machine setup in the LTP test,
> I think we should make something like Functional.LTP-smack, use the
> techniques demonstrated here for building it. But then have pre_test,
> deply, run, and processing phases be customized for that test's needs. in
> Functional.LTP-smack's fuego_test.sh.
I didn't have some time to do some other tests like this way on my side yet.
But, I think it'll be a good instance which can help us to split those special cases/testset.
> Let me know what you think.
> -- Tim
>
> Patch follows:
> ----------------------------
> LTP is a ginormous collection of tests. When something fails, it is very
> handy to be able to run a single test in isolation.
> This test allows you to do that. To run an individual test, just add a
> new spec to the spec file (or, when ftc supports setting a test variable
> on the command line, use that to specify the name of the individual test
> program to run).
>
> This test also demonstrates how one test can use ftc and phases to execute
> just the build of another test. And how one test can re-use the build
> directory of another test.
>
> Signed-off-by: Tim Bird <tim.bird at sony.com>
> ---
> engine/tests/Functional.LTP_one_test/fuego_test.sh | 40
> +++++++++++++++++
> engine/tests/Functional.LTP_one_test/parser.py | 51
> ++++++++++++++++++++++
> engine/tests/Functional.LTP_one_test/spec.json | 12 +++++
> engine/tests/Functional.LTP_one_test/test.yaml | 31 +++++++++++++
> 4 files changed, 134 insertions(+)
> create mode 100755 engine/tests/Functional.LTP_one_test/fuego_test.sh
> create mode 100755 engine/tests/Functional.LTP_one_test/parser.py
> create mode 100644 engine/tests/Functional.LTP_one_test/spec.json
> create mode 100644 engine/tests/Functional.LTP_one_test/test.yaml
>
> diff --git a/engine/tests/Functional.LTP_one_test/fuego_test.sh
> b/engine/tests/Functional.LTP_one_test/fuego_test.sh
> new file mode 100755
> index 0000000..2ebcfc5
> --- /dev/null
> +++ b/engine/tests/Functional.LTP_one_test/fuego_test.sh
> @@ -0,0 +1,40 @@
> +one_test=$FUNCTIONAL_LTP_ONE_TEST_TEST
> +
> +# Don't allow jobs to share build directories # the
> +"test_successfully_built" flag for one spec # is not accurate for
> +another spec, which has a different # individual program to deploy.
> +FUNCTIONAL_LTP_ONE_TEST_PER_JOB_BUILD="true"
> +
> +function test_pre_check {
> + assert_define FUNCTIONAL_LTP_ONE_TEST_TEST }
> +
> +function test_build {
> + # check for LTP build directory
> + LTP_BUILD_DIR=$(echo $JOB_BUILD_DIR | sed s/LTP_one_test/LTP/ | sed
> +s/$TESTSPEC/default/)
> +
> + # if not already built, build LTP
> + if [ ! -e
> ${WORKSPACE}/${LTP_BUILD_DIR}/fuego_test_succesfully_built ] ; then
> + echo "Building parent LTP test..."
> + ftc run-test -b $NODE_NAME -t Functional.LTP -p b
> + # NOTE: vars used in ftc run-test should not leak into this
> environment
> + # that is, none of our test vars should have changed.
> + fi
> +
> + cp
> ${WORKSPACE}/${LTP_BUILD_DIR}/target_bin/testcases/bin/$one_test .
> +}
> +
> +function test_deploy {
> + put $one_test $BOARD_TESTDIR/fuego.$TESTDIR/ }
> +
> +function test_run {
> + # FIXTHIS - deal with arguments better
> + report "cd $BOARD_TESTDIR/fuego.$TESTDIR; \
> + ./$one_test $FUNCTIONAL_LTP_ONE_TEST_ARGS"
> +}
> +
> +function test_processing {
> + return
> +}
> diff --git a/engine/tests/Functional.LTP_one_test/parser.py
> b/engine/tests/Functional.LTP_one_test/parser.py
> new file mode 100755
> index 0000000..fe2336f
> --- /dev/null
> +++ b/engine/tests/Functional.LTP_one_test/parser.py
> @@ -0,0 +1,51 @@
> +#!/usr/bin/python
> +# See common.py for description of command-line arguments
> +
> +import os
> +import sys
> +import collections
> +import re
> +
> +sys.path.insert(0, os.environ['FUEGO_CORE'] + '/engine/scripts/parser')
> +import common as plib
> +
> +results = collections.OrderedDict()
> +
> +# LTP results come in largely 2 formats:
> +# sched_getattr01 1 TPASS : attributes were read back correctly
> +# request_key01.c:49: PASS: request_key() succeed
> +
> +log_lines = open(plib.TEST_LOG, "r").readlines() re_pattern =
> +r"(\S+)\s+(\S)+\s+(TPASS|TCONF)\s+:(.*)"
> +
> +testcase_count = 1
> +for line in log_lines:
> + #print "line=", line
> + if ".c:" in line:
> + parts = line.split(":")
> + filename = parts[0]
> + line_no = parts[1]
> + testcase_num = str(testcase_count)
> + testcase_count += 1
> + result = parts[2].strip()
> + message = parts[3].strip()
> + else:
> + m = re.match(re_pattern, line)
> + #print "m=", m
> + if m:
> + test_name = m.group(1)
> + testcase_num = m.group(2)
> + result = m.group(3)[1:] # strip off the leading 'T'
> + message = m.group(4).strip()
> +
> + # translate to Fuego results
> + if result == 'PASS':
> + results[testcase_num] = 'PASS'
> + elif result == 'FAIL':
> + results[testcase_num] = 'FAIL'
> + elif result == 'CONF':
> + results[testcase_num] = 'SKIP'
> + else:
> + results[testcase_num] = 'ERR'
> +
> +sys.exit(plib.process(results))
> diff --git a/engine/tests/Functional.LTP_one_test/spec.json
> b/engine/tests/Functional.LTP_one_test/spec.json
> new file mode 100644
> index 0000000..9f0b325
> --- /dev/null
> +++ b/engine/tests/Functional.LTP_one_test/spec.json
> @@ -0,0 +1,12 @@
> +{
> + "testName": "Functional.LTP_one_test",
> + "specs": {
> + "default": {
> + "TEST":"brk01"
> + },
> + "fanotify07": {
> + "TEST":"fanotify07"
> + }
> + }
> +}
> +
> diff --git a/engine/tests/Functional.LTP_one_test/test.yaml
> b/engine/tests/Functional.LTP_one_test/test.yaml
> new file mode 100644
> index 0000000..765c98d
> --- /dev/null
> +++ b/engine/tests/Functional.LTP_one_test/test.yaml
> @@ -0,0 +1,31 @@
> +fuego_package_version: 1
> +name: Functional.LTP_one_test
> +description: |
> + This is a convenience test. LTP is a large collection of tests, which
> + takes a lot of infrastructure to set up and execute. Sometimes it
> is
> + handy to run a single LTP test program by itself. This Fuego test
> + exists to allow you to easily do that.
> +
> + To run an individual test, add a new spec for it in the spec.json file.
> + Specify the individual test name to run in the TEST variable. By
> + convention the name of the spec should match the name of the test
> + to run.
> +license: multiple (same as LTP)
> +author: Tim Bird
> +maintainer: Tim Bird <tim.bird at sony.com>
> +version: 1.0
> +fuego_release: 1
> +type: Functional
> +tags: ['general']
> +params:
> + - TEST:
> + description: Name of the individual LTP test program to run.
> + example: fanotify07
> + optional: no
> + - ARGS:
> + description: Command line arguments for the LTP test program.
> + example: -s foo
> + optional: yes
> +data_files:
> + - spec.json
> + - parser.py
> --
> 1.9.1
>
>
More information about the Fuego
mailing list