[Fuego] [PATCH 4/9] rt-tests: use PARAMS instead of a single variable

Bird, Timothy Tim.Bird at sony.com
Thu Jan 11 21:12:09 UTC 2018



> -----Original Message-----
> From: Daniel Sangorrin on Thursday, January 11, 2018 12:28 AM
> This is much more flexible and easier to manage.

Hmmm.  At first I balked at this, because I think having separate variables
makes it easier for users to make different specs with their own combinations.
Having the different variables for different command line options makes it apparent
what "knobs" there are for users to fiddle with, and shows what command line
options are used for tweaking those knobs, which is missing from the changed spec
file. 

However, after thinking about it, the knobs are still available.  But they are not
documented or referenced in Fuego.  It is not clear to me that the inversions,
groups and duration are indeed the knobs that a test user would actually be most
likely to tweak.  (That is, I don't have enough experience with this priority inversion
test to know if these are a reasonable subset of variables to expose or not.)

In the final analysis, since there are no other specs which use these variables, it's
kind of silly to have them.  And once we have other specs, that tweak these
parameters, it will be obvious to end users how to adjust those in their own spec
variations.

So, to make a short story long - I'm applying this.  But it took me a lot longer to 
decide about it than I expected. :-)
 -- Tim


> Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> ---
>  engine/tests/Benchmark.hackbench/fuego_test.sh | 10 ++++++----
>  engine/tests/Benchmark.hackbench/spec.json     |  3 +--
>  engine/tests/Functional.pi_tests/fuego_test.sh |  6 ++----
>  engine/tests/Functional.pi_tests/spec.json     |  4 +---
>  4 files changed, 10 insertions(+), 13 deletions(-)
> 
> diff --git a/engine/tests/Benchmark.hackbench/fuego_test.sh
> b/engine/tests/Benchmark.hackbench/fuego_test.sh
> index a9823bf..cc05c74 100755
> --- a/engine/tests/Benchmark.hackbench/fuego_test.sh
> +++ b/engine/tests/Benchmark.hackbench/fuego_test.sh
> @@ -2,17 +2,19 @@ tarball=../rt-tests/rt-tests-v1.1.1.tar.gz
> 
>  NEED_ROOT=1
> 
> +function test_pre_check {
> +    assert_define BENCHMARK_HACKBENCH_PARAMS
> +}
> +
>  function test_build {
>      patch -p1 -N -s < $TEST_HOME/../rt-tests/0001-Add-scheduling-policies-
> for-old-kernels.patch
>      make NUMA=0 hackbench
>  }
> 
>  function test_deploy {
> -	put hackbench  $BOARD_TESTDIR/fuego.$TESTDIR/
> +    put hackbench  $BOARD_TESTDIR/fuego.$TESTDIR/
>  }
> 
>  function test_run {
> -	report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./hackbench $groups"
> +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./hackbench
> $BENCHMARK_HACKBENCH_PARAMS"
>  }
> -
> -
> diff --git a/engine/tests/Benchmark.hackbench/spec.json
> b/engine/tests/Benchmark.hackbench/spec.json
> index 4fd1a07..47d25ba 100644
> --- a/engine/tests/Benchmark.hackbench/spec.json
> +++ b/engine/tests/Benchmark.hackbench/spec.json
> @@ -2,8 +2,7 @@
>      "testName": "Benchmark.hackbench",
>      "specs": {
>          "default": {
> -            "GPOUPS":"20"
> +            "PARAMS": " "
>          }
>      }
>  }
> -
> diff --git a/engine/tests/Functional.pi_tests/fuego_test.sh
> b/engine/tests/Functional.pi_tests/fuego_test.sh
> index 747bc3e..a95218a 100755
> --- a/engine/tests/Functional.pi_tests/fuego_test.sh
> +++ b/engine/tests/Functional.pi_tests/fuego_test.sh
> @@ -3,9 +3,7 @@ tarball=../rt-tests/rt-tests-v1.1.1.tar.gz
>  NEED_ROOT=1
> 
>  function test_pre_check {
> -    assert_define FUNCTIONAL_PI_TESTS_INVERSIONS
> -    assert_define FUNCTIONAL_PI_TESTS_GROUPS
> -    assert_define FUNCTIONAL_PI_TESTS_DURATION
> +    assert_define FUNCTIONAL_PI_TESTS_PARAMS
>  }
> 
>  function test_build {
> @@ -18,7 +16,7 @@ function test_deploy {
>  }
> 
>  function test_run {
> -    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./pi_stress --
> duration=$FUNCTIONAL_PI_TESTS_DURATION --
> groups=$FUNCTIONAL_PI_TESTS_GROUPS --
> inversions=$FUNCTIONAL_PI_TESTS_INVERSIONS"
> +    report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./pi_stress
> $FUNCTIONAL_PI_TESTS_PARAMS"
>  }
> 
>  function test_processing {
> diff --git a/engine/tests/Functional.pi_tests/spec.json
> b/engine/tests/Functional.pi_tests/spec.json
> index 7908634..a550e17 100644
> --- a/engine/tests/Functional.pi_tests/spec.json
> +++ b/engine/tests/Functional.pi_tests/spec.json
> @@ -2,9 +2,7 @@
>      "testName": "Functional.pi_tests",
>      "specs": {
>          "default": {
> -            "inversions":"100",
> -            "groups":"1",
> -            "duration":"10"
> +            "PARAMS": "--duration=60"
>          }
>      }
>  }
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego


More information about the Fuego mailing list