[Fuego] Integration of Fuego and Linaro test-definitons
Chase Qi
chase.qi at linaro.org
Thu Feb 14 08:51:16 UTC 2019
On Thu, Feb 14, 2019 at 4:10 PM <daniel.sangorrin at toshiba.co.jp> wrote:
>
> Hi again Chase,
>
> > -----Original Message-----
> > From: fuego-bounces at lists.linuxfoundation.org <fuego-bounces at lists.linuxfoundation.org> On Behalf Of
> [...]
> > This is my current work-in-progress approach:
> > https://github.com/sangorrin/test-definitions/tree/master/automated/linux/fuego
> >
> > - Manual usage (run locally)
> > $ git clone https://github.com/sangorrin/test-definitions
> > $ cd test-definitions
> > $ . ./automated/bin/setenv.sh
> > $ cd automated/linux/fuego/
> > $ ./fuego.sh -d Functional.hello_world
> > $ tree output/
> > output/
> > ├── build <- equivalent to fuego buildzone
> > │ ├── hello
> > │ ├── hello.c
> > │ ├── Makefile
> > │ └── README.md
> > ├── fuego.Functional.hello_world <- equivalent to board test folder
> > │ └── hello
> > └── logs <- equivalent to logdir
> > └── testlog.txt
> > - test-runner usage (run on remote board)
> > $ cd test-definitions
> > $ test-runner -g root at 192.168.1.45 -d ./automated/linux/fuego/fuego.yaml -s -o ../output
> > $ ls ../output
> > result.csv
> > result.json
> >
> > I have yet to add the LAVA messages and prepare result.txt but it will be working soon.
>
> I have modified the code to prepare a result.txt and generate LAVA messages but I have no LAVA setup right now. If you test it please let me know if LAVA recognized the test result.
Result looks good in LAVA. I tested with lxc device,
https://validation.linaro.org/scheduler/job/1906258 test failed as
make is missing. Once build-essential installed, test passed
https://validation.linaro.org/results/1906260/1_fuego-hello-world
Thanks,
Chase
>
> Thanks,
> Daniel
>
> >
> > By the way, I couldn't reuse some parts of Fuego that usually run on the host because they depend on bash.
> > Currently Functional.hello_world is working on sh but I will find similar issues as I add more test definitions.
> > Is sh a hard requirement for you guys? or would you be fine with tests requiring bash.
> >
> > Thanks,
> > Daniel
> >
> > > We probably should start a new thread for this topic to share progress?
> > >
> > > Thanks,
> > > Chase
> > >
> > > [1] https://github.com/Linaro/test-definitions/blob/master/automated/lib/sh-test-lib#L250
> > >
> > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > > > -----Original Message-----
> > > > > From: Tim.Bird at sony.com <Tim.Bird at sony.com>
> > > > > Sent: Thursday, February 14, 2019 6:51 AM
> > > > > To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin at toshiba.co.jp>;
> > > > > fuego at lists.linuxfoundation.org
> > > > > Subject: RE: [Fuego] [PATCH] tests: add support for Linaro test-definitons
> > > > >
> > > > > Comments inline below.
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Daniel Sangorrin
> > > > > >
> > > > > > This adds initial support for reusing Linaro test-definitions.
> > > > > > It is still a proof of concept and only tested with
> > > > > > smoke tests. I have written a few FIXTHIS to indicate what
> > > > > > is left.
> > > > > >
> > > > > > To try it follow these steps:
> > > > > >
> > > > > > - prepare SSH_KEY for your board
> > > > > > Eg: Inside fuego's docker container do
> > > > > > > su jenkins
> > > > > > > cp path/to/bbb_id_rsa ~/.ssh/
> > > > > > > vi ~/.ssh/config
> > > > > > > Host 192.167.1.99 <- replace with your boards ip address ($IPADDR)
> > > > > > > IdentityFile ~/.ssh/bbb_id_rsa
> > > > > > - ftc add-job -b bbb -t Functional.linaro
> > > > > > - execute the job from jenkins
> > > > > > - expected results
> > > > > > - table with each test case and the results (PASS/FAIL/SKIP)
> > > > > > - run.json
> > > > > > - csv
> > > > > >
> > > > > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> > > > > > ---
> > > > > > tests/Functional.linaro/chart_config.json | 3 ++
> > > > > > tests/Functional.linaro/fuego_test.sh | 59
> > > > > > +++++++++++++++++++++++++++++++
> > > > > > tests/Functional.linaro/parser.py | 25 +++++++++++++
> > > > > > tests/Functional.linaro/spec.json | 16 +++++++++
> > > > > > tests/Functional.linaro/test.yaml | 27 ++++++++++++++
> > > > > > 5 files changed, 130 insertions(+)
> > > > > > create mode 100644 tests/Functional.linaro/chart_config.json
> > > > > > create mode 100755 tests/Functional.linaro/fuego_test.sh
> > > > > > create mode 100755 tests/Functional.linaro/parser.py
> > > > > > create mode 100644 tests/Functional.linaro/spec.json
> > > > > > create mode 100644 tests/Functional.linaro/test.yaml
> > > > > >
> > > > > > diff --git a/tests/Functional.linaro/chart_config.json
> > > > > > b/tests/Functional.linaro/chart_config.json
> > > > > > new file mode 100644
> > > > > > index 0000000..b8c8fb6
> > > > > > --- /dev/null
> > > > > > +++ b/tests/Functional.linaro/chart_config.json
> > > > > > @@ -0,0 +1,3 @@
> > > > > > +{
> > > > > > + "chart_type": "testcase_table"
> > > > > > +}
> > > > > > diff --git a/tests/Functional.linaro/fuego_test.sh
> > > > > > b/tests/Functional.linaro/fuego_test.sh
> > > > > > new file mode 100755
> > > > > > index 0000000..17b56a9
> > > > > > --- /dev/null
> > > > > > +++ b/tests/Functional.linaro/fuego_test.sh
> > > > > > @@ -0,0 +1,59 @@
> > > > > > +gitrepo="https://github.com/Linaro/test-definitions.git"
> > > > > > +
> > > > > > +# Root permissions required for
> > > > > > +# - installing dependencies on the target (debian/centos) when -s is not
> > > > > > specified
> > > > > > +# - executing some of the tests
> > > > > > +# FIXTHIS: don't force root permissions for tests that do not require them
> > > > > > +NEED_ROOT=1
> > > > > > +
> > > > > > +function test_pre_check {
> > > > > > + # linaro parser dependencies
> > > > > > + # FIXTHIS: use dependencies specified in the test definition yaml
> > > > > > + assert_has_program sed
> > > > > > + assert_has_program awk
> > > > > > + assert_has_program grep
> > > > > > + assert_has_program egrep
> > > > > > + assert_has_program tee
> > > > > I missed this earlier, but Fuego requires 'grep' and 'tee', so there's
> > > > > no need to check for them here.
> > > > > I already made a patch to remove those lines.
> > > > >
> > > > > > +
> > > > > > + # test-runner requires a password-less connection
> > > > > > + # Eg: Inside fuego's docker container do
> > > > > > + # su jenkins
> > > > > > + # cp path/to/bbb_id_rsa ~/.ssh/
> > > > > > + # vi ~/.ssh/config
> > > > > > + # Host 192.167.1.99 <- replace with your boards ip address ($IPADDR)
> > > > > > + # IdentityFile ~/.ssh/bbb_id_rsa
> > > > > > + assert_define SSH_KEY "Please setup SSH_KEY on your board file (fuego-
> > > > > > ro/boards/$NODE_NAME.board)"
> > > > > > +}
> > > > > > +
> > > > > > +function test_build {
> > > > > > + source ./automated/bin/setenv.sh
> > > > > > + pip install -r $REPO_PATH/automated/utils/requirements.txt --user
> > > > >
> > > > > OK. I gave this a spin, and here's an error I got:
> > > > >
> > > > > ===== doing fuego phase: build =====
> > > > > Clone repository https://github.com/Linaro/test-definitions.git.
> > > > > Cloning into 'fuego_git_repo'...
> > > > > Checkout branch/tag/commit id master.
> > > > > Already on 'master'
> > > > > Your branch is up-to-date with 'origin/master'.
> > > > > REPO_PATH: /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf
> > > > > BIN_PATH:
> > > > >
> > > /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf/automated/bin:/usr/local/bin:/usr/local/bin:
> > > > > /usr/bin:/bin:/usr/local/games:/usr/games
> > > > > Downloading/unpacking pexpect (from -r
> > > > > /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf/automated/utils/requirements.txt (line 1))
> > > > > Requirement already satisfied (use --upgrade to upgrade): pyyaml in /usr/lib/python2.7/dist-packages (from
> > -r
> > > > > /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf/automated/utils/requirements.txt (line 2))
> > > > > Requirement already satisfied (use --upgrade to upgrade): requests in /usr/lib/python2.7/dist-packages
> > (from
> > > -r
> > > > > /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf/automated/utils/requirements.txt (line 3))
> > > > > Downloading/unpacking ptyprocess>=0.5 (from pexpect->-r
> > > > > /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf/automated/utils/requirements.txt (line 1))
> > > > > Downloading ptyprocess-0.6.0-py2.py3-none-any.whl
> > > > > Installing collected packages: pexpect, ptyprocess
> > > > > Compiling /tmp/pip-build-65KFAp/pexpect/pexpect/_async.py ...
> > > > > File "/tmp/pip-build-65KFAp/pexpect/pexpect/_async.py", line 19
> > > > > transport, pw = yield from asyncio.get_event_loop()\
> > > > > ^
> > > > > SyntaxError: invalid syntax
> > > > >
> > > > > Successfully installed pexpect ptyprocess
> > > > > Cleaning up...
> > > > > Fuego test_build duration=1.56257462502 seconds
> > > > >
> > > > > Weirdly, I only see this on the first build. I think the pip install alters the docker
> > > > > container, so that even if I specify 'rebuild' for this job, I don't see the "compilation"
> > > > > error after the first build of the job.
> > > > >
> > > > > > +}
> > > > > > +
> > > > > > +function test_run {
> > > > > > + source $WORKSPACE/$JOB_BUILD_DIR/automated/bin/setenv.sh
> > > > > > +
> > > > > > + yaml_file=${FUNCTIONAL_LINARO_YAML:-
> > > > > > "automated/linux/smoke/smoke.yaml"}
> > > > > > + if [ ! -e "${REPO_PATH}/$yaml_file" ]; then
> > > > > > + abort_job "$yaml_file not found"
> > > > > > + fi
> > > > > > +
> > > > > > + if startswith "$yaml_file" "plans"; then
> > > > > > + echo "using test plan: $yaml_file"
> > > > > > + test_or_plan_flag="-p"
> > > > > > + else
> > > > > > + echo "using test definition: $yaml_file"
> > > > > > + test_or_plan_flag="-d"
> > > > > > + fi
> > > > > > +
> > > > > > + if [ -n "$FUNCTIONAL_LINARO_PARAMS" ]; then
> > > > > > + PARAMS="-r $FUNCTIONAL_LINARO_PARAMS"
> > > > > > + else
> > > > > > + PARAMS=""
> > > > > > + fi
> > > > > > +
> > > > > > + # FIXTHIS: don't use -s for targets with debian/centos
> > > > > > + test-runner -o ${LOGDIR} $test_or_plan_flag ${REPO_PATH}/$yaml_file
> > > > > > $PARAMS -g $LOGIN@$IPADDR -s -e
> > > > > > +}
> > > > > > +
> > > > > > +# FIXTHIS: the log directory is populated with a copy of the whole
> > > > > > repository, clean unnecessary files
> > > > > > diff --git a/tests/Functional.linaro/parser.py
> > > > > > b/tests/Functional.linaro/parser.py
> > > > > > new file mode 100755
> > > > > > index 0000000..48b502b
> > > > > > --- /dev/null
> > > > > > +++ b/tests/Functional.linaro/parser.py
> > > > > > @@ -0,0 +1,25 @@
> > > > > > +#!/usr/bin/python
> > > > > > +
> > > > > > +import os, sys, collections
> > > > > > +import common as plib
> > > > > > +import json
> > > > > > +
> > > > > > +# allocate variable to store the results
> > > > > > +measurements = {}
> > > > > > +measurements = collections.OrderedDict()
> > > > > > +
> > > > > > +# read results from linaro result.json format
> > > > > > +with open(plib.LOGDIR + "/result.json") as f:
> > > > > > + data = json.load(f)[0]
> > > > > > +
> > > > > > +for test_case in data['metrics']:
> > > > > > + test_case_id = test_case['test_case_id']
> > > > > > + result = test_case['result']
> > > > > > + # FIXTHIS: add measurements when available
> > > > > > + # measurement = test_case['measurement']
> > > > > > + # units = test_case['units']
> > > > > > + measurements['default.' + test_case_id] = result.upper()
> > > > > > +
> > > > > > +# FIXTHIS: think about how to get each test's log from stdout.log
> > > > > > +
> > > > > > +sys.exit(plib.process(measurements))
> > > > > > diff --git a/tests/Functional.linaro/spec.json
> > > > > > b/tests/Functional.linaro/spec.json
> > > > > > new file mode 100644
> > > > > > index 0000000..561e2ab
> > > > > > --- /dev/null
> > > > > > +++ b/tests/Functional.linaro/spec.json
> > > > > > @@ -0,0 +1,16 @@
> > > > > > +{
> > > > > > + "testName": "Functional.linaro",
> > > > > > + "specs": {
> > > > > > + "default": {
> > > > > > + "yaml": "automated/linux/smoke/smoke.yaml",
> > > > > > + "extra_success_links": {"csv": "result.csv"},
> > > > > > + "extra_fail_links": {"csv": "results.csv"}
> > > > > > + },
> > > > > > + "smoke": {
> > > > > > + "yaml": "automated/linux/smoke/smoke.yaml",
> > > > > > + "params": "TESTS='pwd'",
> > > > > > + "extra_success_links": {"csv": "result.csv"},
> > > > > > + "extra_fail_links": {"csv": "results.csv"}
> > > > > > + }
> > > > > > + }
> > > > > > +}
> > > > > > diff --git a/tests/Functional.linaro/test.yaml
> > > > > > b/tests/Functional.linaro/test.yaml
> > > > > > new file mode 100644
> > > > > > index 0000000..a2efee8
> > > > > > --- /dev/null
> > > > > > +++ b/tests/Functional.linaro/test.yaml
> > > > > > @@ -0,0 +1,27 @@
> > > > > > +fuego_package_version: 1
> > > > > > +name: Functional.linaro
> > > > > > +description: |
> > > > > > + Linaro test-definitions
> > > > > > +license: GPL-2.0
> > > > > > +author: Milosz Wasilewski, Chase Qi
> > > > > > +maintainer: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> > > > > > +version: latest git commits
> > > > > > +fuego_release: 1
> > > > > > +type: Functional
> > > > > > +tags: ['kernel', 'linaro']
> > > > > > +git_src: https://github.com/Linaro/test-definitions
> > > > > > +params:
> > > > > > + - YAML:
> > > > > > + description: test definiton or plan.
> > > > > > + example: "automated/linux/smoke/smoke.yaml"
> > > > > > + optional: no
> > > > > > + - PARAMS:
> > > > > > + description: List of params for the test PARAM1=VALUE1
> > > > > > [PARAM2=VALUE2]
> > > > > > + example: "TESTS='pwd'"
> > > > > > + optional: yes
> > > > > > +data_files:
> > > > > > + - chart_config.json
> > > > > > + - fuego_test.sh
> > > > > > + - parser.py
> > > > > > + - spec.json
> > > > > > + - test.yaml
> > > > > > --
> > > > > > 2.7.4
> > > > >
> > > > > And here's output from one of my initial runs. I haven't debugged it yet.
> > > > > The issue may be something weird in my board file or configuration.
> > > > >
> > > > > ===== doing fuego phase: run =====
> > > > > -------------------------------------------------
> > > > > REPO_PATH: /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf
> > > > > BIN_PATH:
> > > > >
> > > /fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf/automated/bin:/usr/local/bin:/usr/local/bin:
> > > > > /usr/bin:/bin:/usr/local/games:/usr/games
> > > > > using test definition: automated/linux/smoke/smoke.yaml
> > > > > Warning: Permanently added '10.0.1.74' (ECDSA) to the list of known hosts.
> > > > > 2019-02-13 21:46:14,364 - RUNNER: INFO: Tests to run:
> > > > > Warning: Permanently added '10.0.1.74' (ECDSA) to the list of known hosts.
> > > > > 2019-02-13 21:46:14,814 - RUNNER.TestSetup: INFO: Test repo copied to:
> > > > > /fuego-rw/logs/Functional.linaro/bbb.default.6.6/smoke_2efe23f3-2a7b-4655-b785-d25c11b84ea8
> > > > > 2019-02-13 21:46:14,826 - RUNNER.TestRun: INFO: Archiving test files
> > > > > 2019-02-13 21:46:14,845 - RUNNER.TestRun: INFO: Creating test path
> > > > > Warning: Permanently added '10.0.1.74' (ECDSA) to the list of known hosts.
> > > > > 2019-02-13 21:46:15,133 - RUNNER.TestRun: INFO: Copying test archive to target host
> > > > > Warning: Permanently added '10.0.1.74' (ECDSA) to the list of known hosts.
> > > > > 2019-02-13 21:46:16,260 - RUNNER.TestRun: INFO: Unarchiving test files on target
> > > > > Warning: Permanently added '10.0.1.74' (ECDSA) to the list of known hosts.
> > > > > 2019-02-13 21:46:16,674 - RUNNER.TestRun: INFO: Removing test file archive from target
> > > > > Warning: Permanently added '10.0.1.74' (ECDSA) to the list of known hosts.
> > > > > 2019-02-13 21:46:16,978 - RUNNER.TestRun: INFO: Executing
> > > > > /root/output/smoke_2efe23f3-2a7b-4655-b785-d25c11b84ea8/run.sh remotely on root at 10.0.1.74
> > > > > {'path':
> > > > > '/fuego-rw/buildzone/bbb.default.Functional.linaro-debian-armhf/automated/linux/smoke/smoke.yaml',
> > > > > 'uuid': '2efe23f3-2a7b-4655-b785-d25c11b84ea8', 'timeout': None, 'skip_install': True}
> > > > > Warning: Permanently added '10.0.1.74' (ECDSA) to the list of known hosts.
> > > > > + export TESTRUN_ID=smoke-tests-basic
> > > > > + cd /root/output/smoke_2efe23f3-2a7b-4655-b785-d25c11b84ea8
> > > > > + cat uuid
> > > > > + UUID=2efe23f3-2a7b-4655-b785-d25c11b84ea8
> > > > > + echo <STARTRUN smoke-tests-basic 2efe23f3-2a7b-4655-b785-d25c11b84ea8>
> > > > > <STARTRUN smoke-tests-basic 2efe23f3-2a7b-4655-b785-d25c11b84ea8>
> > > > > + export PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
> > > > > + cd ./automated/linux/smoke/
> > > > > + ./smoke.sh -s True -t pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat, lsblk
> > > > > INFO: install_deps skipped
> > > > >
> > > > > INFO: Running pwd test...
> > > > > /root/output/smoke_2efe23f3-2a7b-4655-b785-d25c11b84ea8/automated/linux/smoke
> > > > > pwd pass
> > > > >
> > > > > INFO: Running lsb_release test...
> > > > > ./smoke.sh: 1: eval: lsb_release: not found
> > > > > lsb_release fail
> > > > >
> > > > > INFO: Running uname test...
> > > > > Linux beaglebone 4.4.88-ti-r125 #1 SMP Thu Sep 21 19:23:24 UTC 2017 armv7l GNU/Linux
> > > > > uname pass
> > > > >
> > > > > INFO: Running ip test...
> > > > > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
> > > > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > > > > inet 127.0.0.1/8 scope host lo
> > > > > valid_lft forever preferred_lft forever
> > > > > inet6 ::1/128 scope host
> > > > > valid_lft forever preferred_lft forever
> > > > > 2: eth0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group
> > default
> > > > > qlen 1000
> > > > > link/ether 90:59:af:54:cd:e6 brd ff:ff:ff:ff:ff:ff
> > > > > inet 10.0.1.74/24 brd 10.0.1.255 scope global eth0
> > > > > valid_lft forever preferred_lft forever
> > > > > inet6 fe80::9259:afff:fe54:cde6/64 scope link
> > > > > valid_lft forever preferred_lft forever
> > > > > 3: can0: <NOARP,ECHO> mtu 16 qdisc noop state DOWN group default qlen 10
> > > > > link/can
> > > > > 4: can1: <NOARP,ECHO> mtu 16 qdisc noop state DOWN group default qlen 10
> > > > > link/can
> > > > > 5: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen
> > 1000
> > > > > link/ether 90:59:af:54:cd:e8 brd ff:ff:ff:ff:ff:ff
> > > > > inet 192.168.7.2/30 brd 192.168.7.3 scope global usb0
> > > > > valid_lft forever preferred_lft forever
> > > > > inet6 fe80::9259:afff:fe54:cde8/64 scope link
> > > > > valid_lft forever preferred_lft forever
> > > > > 6: usb1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen
> > 1000
> > > > > link/ether 90:59:af:54:cd:eb brd ff:ff:ff:ff:ff:ff
> > > > > inet 192.168.6.2/30 brd 192.168.6.3 scope global usb1
> > > > > valid_lft forever preferred_lft forever
> > > > > inet6 fe80::9259:afff:fe54:cdeb/64 scope link
> > > > > valid_lft forever preferred_lft forever
> > > > > ip pass
> > > > >
> > > > > INFO: Running lscpu test...
> > > > > Architecture: armv7l
> > > > > Byte Order: Little Endian
> > > > > CPU(s): 1
> > > > > On-line CPU(s) list: 0
> > > > > Thread(s) per core: 1
> > > > > Core(s) per socket: 1
> > > > > Socket(s): 1
> > > > > Model: 2
> > > > > Model name: ARMv7 Processor rev 2 (v7l)
> > > > > CPU max MHz: 1000.0000
> > > > > CPU min MHz: 300.0000
> > > > > BogoMIPS: 995.32
> > > > > Flags: half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32
> > > > > lscpu pass
> > > > >
> > > > > INFO: Running vmstat test...
> > > > > procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
> > > > > r b swpd free buff cache si so bi bo in cs us sy id wa st
> > > > > 0 0 0 381532 2780 58072 0 0 1 3 38 2 0 0 99 0 0
> > > > > vmstat pass
> > > > >
> > > > > INFO: Running lsblk test...
> > > > > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> > > > > mmcblk0 179:0 0 14.9G 0 disk
> > > > > |-mmcblk0p1 179:1 0 6G 0 part /
> > > > > `-mmcblk0p2 179:2 0 8.9G 0 part /data
> > > > > mmcblk1 179:8 0 1.8G 0 disk
> > > > > `-mmcblk1p1 179:9 0 1.8G 0 part /mnt/on-board-mmc
> > > > > mmcblk1boot0 179:16 0 1M 1 disk
> > > > > mmcblk1boot1 179:24 0 1M 1 disk
> > > > > lsblk pass
> > > > > + ../../utils/send-to-lava.sh ./output/result.txt
> > > > > <TEST_CASE_ID=pwd RESULT=pass>
> > > > > <TEST_CASE_ID=lsb_release RESULT=fail>
> > > > > <TEST_CASE_ID=uname RESULT=pass>
> > > > > <TEST_CASE_ID=ip RESULT=pass>
> > > > > <TEST_CASE_ID=lscpu RESULT=pass>
> > > > > <TEST_CASE_ID2019-02-13 21:46:18,376 - RUNNER.TestRun: INFO:
> > > > > smoke_2efe23f3-2a7b-4655-b785-d25c11b84ea8 test finished.
> > > > >
> > > > > 2019-02-13 21:46:18,397 - RUNNER.ResultParser: INFO: Result files saved to:
> > > > > /fuego-rw/logs/Functional.linaro/bbb.default.6.6/smoke_2efe23f3-2a7b-4655-b785-d25c11b84ea8
> > > > > =vmstat RESULT=pass>
> > > > > <TEST_CASE_ID=lsblk RESULT=pass>
> > > > > + echo <ENDRUN smoke-tests-basic 2efe23f3-2a7b-4655-b785-d25c11b84ea8>
> > > > > <ENDRUN smoke-tests-basic 2efe23f3-2a7b-4655-b785-d25c11b84ea8>
> > > > > --- Printing result.csv ---
> > > > > name,test_case_id,result,measurement,units,test_params
> > > > > smoke-tests-basic,pwd,pass,,,"TESTS=pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat,
> > > > > lsblk;SKIP_INSTALL=False"
> > > > > smoke-tests-basic,lsb_release,fail,,,"TESTS=pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat,
> > > > > lsblk;SKIP_INSTALL=False"
> > > > > smoke-tests-basic,uname,pass,,,"TESTS=pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat,
> > > > > lsblk;SKIP_INSTALL=False"
> > > > > smoke-tests-basic,ip,pass,,,"TESTS=pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat,
> > > > > lsblk;SKIP_INSTALL=False"
> > > > > smoke-tests-basic,lscpu,pass,,,"TESTS=pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat,
> > > > > lsblk;SKIP_INSTALL=False"
> > > > > smoke-tests-basic,vmstat,pass,,,"TESTS=pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat,
> > > > > lsblk;SKIP_INSTALL=False"
> > > > > smoke-tests-basic,lsblk,pass,,,"TESTS=pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat,
> > > > > lsblk;SKIP_INSTALL=False"
> > > > >
> > > > > -------------------------------------------------
> > > > > ===== doing fuego phase: post_test =====
> > > > > scp: /home/fuego/fuego.Functional.linaro/Functional.linaro.log: No such file or directory
> > > > > INFO: the test did not produce a test log on the target
> > > > > ===== doing fuego phase: processing =====
> > > > > ### WARNING: Program returned exit code ''
> > > > > ### WARNING: Log evaluation may be invalid
> > > > > ### Unrecognized results format
> > > > > ### Unrecognized results format
> > > > > ### Unrecognized results format
> > > > > ### Unrecognized results format
> > > > > ### Unrecognized results format
> > > > > ### Unrecognized results format
> > > > > ### Unrecognized results format
> > > > > ERROR: results did not satisfy the threshold
> > > > > Fuego: requested test phases complete!
> > > > > Build step 'Execute shell' marked build as failure
> > > > >
> > > > > ----------
> > > > >
> > > > > It looks like I'm close. I'll keep playing with it, but if you see something
> > > > > that I should fix, please let me know.
> > > > >
> > > > > Note that I *do* get a results table in Jenkins. lsb_release fails, but the
> > > > > other tests (ip, lsblk, lscpu, pwd, uname, and vmstat) all pass. But
> > > > > testlog.txt has 'INFO: the test did not produce a test log on the target'.
> > > > >
> > > > > Just FYI. Thanks for the code.
> > > > > -- Tim
> > > >
> > > > _______________________________________________
> > > > Fuego mailing list
> > > > Fuego at lists.linuxfoundation.org
> > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > _______________________________________________
> > Fuego mailing list
> > Fuego at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
More information about the Fuego
mailing list