[Fuego] [PATCH] autopkgtest: add support for debian package tests

[Daniel Sangorrin]

autopkgtest allows running tests included in Debian
source packages. This commits adds support for running
autopkgtest on the target (it needs to be installed)
and parse the results.

One important thing to notice is that autopkgtest installs
packaged dependencies (testbed) and therefore leaves the
board's Debian in a different state. For that reason,
it is recommended that an OS deploy phase (e.g. using LAVA)
is executed before the test.

The spec currently contains very few tests but there
are many more available: https://ci.debian.net/data/packages/

The patch and test specs were tested on BeagleBone Black
using bone-debian-9.2-iot-armhf-2017-10-10-4gb.img.xz

Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
---
 engine/tests/Functional.autopkgtest/fuego_test.sh | 19 +++++++++++++++++
 engine/tests/Functional.autopkgtest/parser.py     | 25 +++++++++++++++++++++++
 engine/tests/Functional.autopkgtest/spec.json     | 11 ++++++++++
 3 files changed, 55 insertions(+)
 create mode 100755 engine/tests/Functional.autopkgtest/fuego_test.sh
 create mode 100755 engine/tests/Functional.autopkgtest/parser.py
 create mode 100644 engine/tests/Functional.autopkgtest/spec.json

diff --git a/engine/tests/Functional.autopkgtest/fuego_test.sh b/engine/tests/Functional.autopkgtest/fuego_test.sh
new file mode 100755
index 0000000..5aac549
--- /dev/null
+++ b/engine/tests/Functional.autopkgtest/fuego_test.sh
@@ -0,0 +1,19 @@
+function test_pre_check {
+    assert_define FUNCTIONAL_AUTOPKGTEST_PACKAGES
+    is_on_target autopkgtest AUTOPKGTEST_PROGRAM /usr/bin
+    assert_define AUTOPKGTEST_PROGRAM
+}
+
+function test_run {
+    report "echo 'Running tests for these packages: $FUNCTIONAL_AUTOPKGTEST_PACKAGES'"
+    for PACKAGE in $FUNCTIONAL_AUTOPKGTEST_PACKAGES; do
+        report_append "cd $BOARD_TESTDIR/fuego.$TESTDIR; mkdir $PACKAGE; cd $PACKAGE; \
+            apt-get source $PACKAGE; cd $PACKAGE-*; autopkgtest --output-dir ./output-dir-$PACKAGE -B  . -- null; \
+            echo 'Fuego test_set: $PACKAGE' >> $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt; \
+            cat ./output-dir-$PACKAGE/summary >> $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt"
+    done
+}
+
+function test_fetch_results {
+    get $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt $LOGDIR/summary.txt
+}
diff --git a/engine/tests/Functional.autopkgtest/parser.py b/engine/tests/Functional.autopkgtest/parser.py
new file mode 100755
index 0000000..98a2024
--- /dev/null
+++ b/engine/tests/Functional.autopkgtest/parser.py
@@ -0,0 +1,25 @@
+#!/bin/python
+
+import os, re, sys
+
+sys.path.insert(0, os.environ['FUEGO_CORE'] + '/engine/scripts/parser')
+import common as plib
+
+AUTOPKGTST_SUMMARY = plib.LOGDIR + "/summary.txt"
+
+print "Parsing: " + AUTOPKGTST_SUMMARY
+
+results = {}
+with open(AUTOPKGTST_SUMMARY) as f:
+    for line in f:
+        print line.strip()
+        if line.startswith("Fuego test_set:"):
+            test_set = line.split()[2]
+            continue
+        if line.strip():
+            print line.strip()
+            fields = line.split()
+            test_case = fields[0]
+            results[test_set+'.'+test_case] = "PASS" if fields[1] == 'PASS' else "FAIL"
+
+sys.exit(plib.process(results))
diff --git a/engine/tests/Functional.autopkgtest/spec.json b/engine/tests/Functional.autopkgtest/spec.json
new file mode 100644
index 0000000..afa8653
--- /dev/null
+++ b/engine/tests/Functional.autopkgtest/spec.json
@@ -0,0 +1,11 @@
+{
+    "testName": "Functional.autopkgtest",
+    "specs": {
+        "smoketest": {
+            "packages": "fakesleep"
+        },
+        "default": {
+            "packages": "libzerg libproxy fakesleep"
+        }
+    }
+}
-- 
2.7.4