[Fuego] [PATCH] Fuego Release Test Repository

Guilherme Camargo guicc at profusion.mobi
Tue Feb 20 02:25:34 UTC 2018


Hello, Tim

On Mon, Feb 19, 2018 at 9:56 PM, <Tim.Bird at sony.com> wrote:

>
>
> > -----Original Message-----
> > From: Guilherme Campos Camargo
> > Hello, everyone
> >
> > This repository
> What repository?  https://bitbucket.org/profusionmobi/fuego?
>
​Not this one.
​


>
> I see a repository on bitbucket, under profusionmobi, called
> 'fuego-release-test'?  Is that the one?  I can find no mention
> of the repository below...
>

​Yes. That's the one. I'm sorry for not being clear on my first e-mail.
https://bitbucket.org/profusionmobi/fuego-release-test​

>
> I presume I clone something, then run build_and_run.sh?
> Do I need to have any fuego materials pre-installed?
> (the 'driving' fuego?)
>

​You shouldn't need anything ​besides what's in fuego-release-test.
And you're right. After cloning the repo, you just need to run
build_and_run.sh to start a new fuego instance with a pre-installed
fuego-test board and with a Functional.fuegotest job added to it.

The basic command is:
./build_and_run.sh -b YOUR_PREFERRED_PORT up

But I just noticed (as you mentioned in your next e-mail) that there's
a bug that's requiring you to 'execute from scratch' at least once with.

./build_and_run.sh -c -b YOUR_PREFERRED_PORT up

Sorry. I'll fix that.

Then you can just open fuego-jenkins' interface through
localhost:YOUR_PREF_PORT/fuego
and will you see that there's a fuego-test board defined (with a functional
test added to it).

​
>
> Thanks,
>  -- Tim
>
>
> > has been created to store the resources needed for the
> > generation of a Docker image, based on fuego-base, that has all the
> > tools needed for the automation of the tests that are required for
> > validating a new Fuego release with Fuego itself.
> >
> > It consists basically of:
> >
> > * A Dockerfile containing the installation of additional dependencies.
> > * A new fuego-test board (fuego-ro/boards/fuego-test)
> > * A functional test (tests/Functional.fuegotest) that will download a
> > specific branch of Fuego from a repository provided by the user, start
> > an inner container running that specific fuego release and run basic
> > tests on it.
> >
> > The functional test currently just adds a board and a test plan to the
> > Fuego-Under-Test, reporting success if everything goes well and failure
> > if not.
> >
> > Please check the documentation (README.md) for instructions in how to
> > build/run the image with the helper script `build_and_run.sh`.
> >
> > Also, it's important to note that this patch represents a sort of
> > paradigm shift in the way that tests are added to Fuego, given that:
> > - The software requirements are being installed through a Dockerfile
> >   that extends fuego-base.
> > - The test routines/plans and the new boards are moved to Fuego during
> >   build time, acting almost as an "overlay".
> > For that reason, it's imperative that current users/contributors provide
> > their feedback regarding this new approach before we move on. Also,
> > since I just started contributing with and using fuego, it's likely that
> > I missed something, specially related to Fuego usability. So please,
> > kindly add your comments and correct me if so.
> >
> > Please also see a few more comments in the code below.
> >
> > ---
> >  Dockerfile                               | 30 ++++++++++++++
> >  README.md                                | 38 ++++++++++++++++++
> >  build_and_run.sh                         | 67
> ++++++++++++++++++++++++++++++++
> >  fuego-ro/boards/README                   |  4 ++
> >  fuego-ro/boards/fuego-test.board         | 13 +++++++
> >  fuego-ro/conf/README                     |  1 +
> >  fuego-ro/conf/fuego.conf                 |  3 ++
> >  fuego-rt-entrypoint.sh                   | 26 +++++++++++++
> >  tests/Functional.fuegotest/fuego_test.sh | 36 +++++++++++++++++
> >  tests/Functional.fuegotest/spec.json     |  9 +++++
> >  tests/Functional.fuegotest/test_run.py   | 19 +++++++++
> >  11 files changed, 246 insertions(+)
> >  create mode 100644 Dockerfile
> >  create mode 100644 README.md
> >  create mode 100755 build_and_run.sh
> >  create mode 100644 fuego-ro/boards/README
> >  create mode 100644 fuego-ro/boards/fuego-test.board
> >  create mode 100644 fuego-ro/conf/README
> >  create mode 100644 fuego-ro/conf/fuego.conf
> >  create mode 100755 fuego-rt-entrypoint.sh
> >  create mode 100755 tests/Functional.fuegotest/fuego_test.sh
> >  create mode 100644 tests/Functional.fuegotest/spec.json
> >  create mode 100755 tests/Functional.fuegotest/test_run.py
> >
> > diff --git a/Dockerfile b/Dockerfile
> > new file mode 100644
> > index 0000000..5dcb755
> > --- /dev/null
> > +++ b/Dockerfile
> > @@ -0,0 +1,30 @@
> > +from fuegotest/fuego-base:latest
> > +
> > +ARG DEBIAN_FRONTEND=noninteractive
> > +RUN apt-get update && \
> > +    apt-get -yV install \
> > +        apt-transport-https \
> > +        ca-certificates \
> > +        curl \
> > +        gnupg2 \
> > +        python3 \
> > +        python3-pip \
> > +        software-properties-common && \
> > +    curl -fsSL \
> > +        https://download.docker.com/linux/$(\
> > +            source /etc/os-release; echo "$ID")/gpg \
> > +        | sudo apt-key add - && \
> > +    add-apt-repository \
> > +        "deb [arch=amd64] https://download.docker.com/linux/$(\
> > +            source /etc/os-release; echo "$ID") \
> > +            $(lsb_release -cs) stable" && \
> > +    apt-get update && \
> > +    apt-get -yV install \
> > +        docker-ce && \
> > +    pip3 install pexpect
> > +
> > +RUN echo "jenkins ALL = (root) NOPASSWD: /usr/bin/docker, /bin/sh" >>
> > /etc/sudoers
> > +
> > +COPY fuego-ro /fuego-ro
> > +COPY tests /fuego-core/engine/tests
> > +COPY fuego-rt-entrypoint.sh /
> > diff --git a/README.md b/README.md
> > new file mode 100644
> > index 0000000..faa4d83
> > --- /dev/null
> > +++ b/README.md
> > @@ -0,0 +1,38 @@
> > +# Fuego Release Test
> > +
> > +This Fuego Image has all the tools needed for the automation of the
> tests
> > that
> > +are required for validating a new Fuego release with Fuego itself.
> > +
> > +It consists basically of:
> > +* A Dockerfile containing the installation of additional dependencies.
> > +* A new fuego-test board (fuego-ro/boards/fuego-test)
> > +* A functional test (tests/Functional.fuegotest) that will download a
> specific
> > +  branch of Fuego from a repository provided by the user, start an inner
> > +  container running that specific fuego release and run basic tests on
> it.
> > +
> > +## Build and Run Instructions
> > +
> > +In order to build and run the tests with minimal effort, just use the
> given
> > +`build_and_run.sh` script with the following parameters:
> > +
> > +```
> > +$ build_and_run.sh -r ${fuego-repo:-FUEGO_OFFICIAL_REPO} -b ${branch:-
> > MASTER} -p ${fuego_port} up
> > +```
> > +
> > +In order to stop and save your session simply execute.
> > +
> > +```
> > +$ build_and_run.sh down
> > +```
> > +
> > +Please see `$ build_and_run.sh -h` for more options.
> > +
> > +
> > +## Additional Instructions
> > +
> > +By running fuego-release-test, as explained above, you should be able to
> > access
> > +fuego through `http://localhost:${fuego_port}/fuego/`
> > +
> > +There you will see that the fuego-test board is already available and
> that
> > the
> > +default test has been added to it. Simply use Fuego's Jenkins interface
> as
> > +usual to test Fuego itself.
> > diff --git a/build_and_run.sh b/build_and_run.sh
> > new file mode 100755
> > index 0000000..e448ddf
> > --- /dev/null
> > +++ b/build_and_run.sh
> > @@ -0,0 +1,67 @@
> > +#!/bin/bash
> > +
> > +source ./fuego-ro/conf/fuego.conf
> > +
> > +readonly internal_fuego_port=${jenkins_port:-8080}
> > +readonly fuego_rt_image=fuego-rt
> > +readonly fuego_rt_container=fuego-rt-container
> > +detached_mode_opts="--interactive --attach"
> > +port_opts="-p 8080:${internal_fuego_port}"
> > +
> > +usage() {
> > +    echo "usage: $0 [OPTIONS] up|down"
> > +    grep " .)\\ #" "${0}"
> > +    exit 0
> > +}
> > +
> > +[ $# -eq 0 ] && usage
> > +while getopts "hdcr:b:p:" arg; do
> > +  case $arg in
> > +    d) # Run in detached mode
> > +      detached_mode_opts=""
> > +      ;;
> > +    c) # Start from scratch, rebuilding image
> > +      clean_start="true"
> > +      ;;
> > +    r) # Fuego Repository to be tested (default: Fuego Official Repo)
> > +      fuego_repo_opts="-e TEST_REPO=${OPTARG}"
> > +      ;;
> > +    b) # Fuego Branch to be tested (default: Master)
> > +      fuego_branch_opts="-e TEST_BRANCH=${OPTARG}"
> > +      ;;
> > +    p) # Fuego (the stable, not the one under test) port mapped on Host
> > (default: 8090)
> > +      port_opts="-p ${OPTARG}:${internal_fuego_port}"
> > +      ;;
> > +    h | *) # Display help.
> > +      usage
> > +      exit 0
> > +      ;;
> > +  esac
> > +done
> > +shift "$((OPTIND - 1))"
> > +
> > +if [ "${1}" != "up" ] && [ "${1}" != "down" ]; then
> > +    usage
> > +fi
> > +
> > +if [ "$1" = "down" ]; then
> > +    docker stop ${fuego_rt_container}
> > +fi
> > +
> > +if [ "$1" = "up" ]; then
> > +    if [ -v clean_start ]; then
> > +        docker build -t "${fuego_rt_image}" .
> > +        docker rm -f "${fuego_rt_container}"
> > +        docker run -dit --name ${fuego_rt_container} \
> > +            -v /var/run/docker.sock:/var/run/docker.sock \
> > +            --net=bridge \
> > +            ${port_opts} ${fuego_rt_image}
> > +       docker exec \
> > +           ${fuego_repo_opts} \
> > +           ${fuego_branch_opts} \
> > +           -e FUEGO_PORT="${internal_fuego_port}" \
> > +           ${fuego_rt_container} /fuego-rt-entrypoint.sh
> > +       docker stop ${fuego_rt_container}
> > +    fi
> > +    docker start ${detached_mode_opts} ${fuego_rt_container}
> > +fi
> > diff --git a/fuego-ro/boards/README b/fuego-ro/boards/README
> > new file mode 100644
> > index 0000000..bf20dc8
> > --- /dev/null
> > +++ b/fuego-ro/boards/README
> > @@ -0,0 +1,4 @@
> > +This directory contains board files.
> > +
> > +See docs for reference on what variables should they define.
> > +
> > diff --git a/fuego-ro/boards/fuego-test.board b/fuego-ro/boards/fuego-
> > test.board
> > new file mode 100644
> > index 0000000..185e0b4
> > --- /dev/null
> > +++ b/fuego-ro/boards/fuego-test.board
> > @@ -0,0 +1,13 @@
> > +inherit "base-board"
> > +include "base-params"
> > +
> > +IPADDR=127.0.0.1
> > +SRV_IP=127.0.0.1
> > +#LOGIN="root"
> > +#PASSWORD="root"
> > +#SSH_KEY="path/to/id_rsa"
> > +BOARD_TESTDIR="$FUEGO_RW/tests"
> > +PLATFORM="x86_64"
> > +TRANSPORT="local"
> > +ARCHITECTURE="x86_64"
> > +DISTRIB="nosyslogd.dist"
> > diff --git a/fuego-ro/conf/README b/fuego-ro/conf/README
> > new file mode 100644
> > index 0000000..83942d4
> > --- /dev/null
> > +++ b/fuego-ro/conf/README
> > @@ -0,0 +1 @@
> > +This directory is for configuration files for Fuego and associated
> tools.
> > diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
> > new file mode 100644
> > index 0000000..0c95eae
> > --- /dev/null
> > +++ b/fuego-ro/conf/fuego.conf
> > @@ -0,0 +1,3 @@
> > +docker_jenkins_uid=1000
> > +docker_jenkins_gid=987
> > +jenkins_port=8090
> > diff --git a/fuego-rt-entrypoint.sh b/fuego-rt-entrypoint.sh
> > new file mode 100755
> > index 0000000..3c680dd
> > --- /dev/null
> > +++ b/fuego-rt-entrypoint.sh
> > @@ -0,0 +1,26 @@
> > +#!/bin/bash
> > +
> > +set -e
> > +
> > +function wait_for_fuego {
> > +    readonly fuego_port=${FUEGO_PORT:-8080}
> > +    readonly jenkins_url="http://localhost:${fuego_port}/fuego/"
> > +
> > +    echo "Waiting for Jenkins on ${jenkins_url}..."
> > +    while ! curl --output /dev/null --silent --head --fail
> "${jenkins_url}";
> > +    do
> > +        sleep 1
> > +    done
> > +}
> > +
> > +test_repo=${TEST_REPO:-https://bitbucket.org/profusionmobi/fuego}
> > +test_repo_escaped=$(sed 's/[&/\]/\\&/g' <<< "${test_repo}")
> > +sed -i "s/FUEGOTEST_REPO/${test_repo_escaped}/" /fuego-
> > core/engine/tests/Functional.fuegotest/spec.json
> >
> > As you can see, I'm using placeholders (FUEGOTEST_REPO and
> > FUEGOTEST_BRANCH) in order to parametrize the test (those are passed by
> > the user when running build_and_run.sh).
> >
> > I feel that there might be a better way of doing this. Does anyone have
> > any suggestion?
> >
> > Also, note that we're using our own repo/branch for the
> > fuego-under-test. This is necessary for now because there are some
> > required changes that have not been merged yet either on master or next
> > of the official repo (I'm sending that patch for review in a separate
> > e-mail). The idea is to use the official repo and next as default.
> >
> > +
> > +test_branch=${TEST_BRANCH:-"fuego-base"}
> > +test_branch_escaped=$(sed 's/[&/\]/\\&/g' <<< "${test_branch}")
> > +sed -i "s/FUEGOTEST_BRANCH/${test_branch_escaped}/" /fuego-
> > core/engine/tests/Functional.fuegotest/spec.json
> > +
> > +wait_for_fuego
> > +ftc add-nodes fuego-test
> > +ftc add-jobs -b fuego-test -t Functional.fuegotest
> > diff --git a/tests/Functional.fuegotest/fuego_test.sh
> > b/tests/Functional.fuegotest/fuego_test.sh
> > new file mode 100755
> > index 0000000..6843b00
> > --- /dev/null
> > +++ b/tests/Functional.fuegotest/fuego_test.sh
> > @@ -0,0 +1,36 @@
> > +#!/bin/bash
> > +
> > +set -e
> > +
> > +readonly fuego_release_dir=fuego-release
> > +readonly fuego_release_img_name=fuego-release
> > +readonly fuego_release_container_name=fuego-release-container
> > +
> > +function test_build {
> > +    if [ -d ${fuego_release_dir} ]; then
> > +        rm -r ${fuego_release_dir}
> > +    fi
> > +    git clone --quiet --depth 1 --single-branch \
> > +        --branch "${FUNCTIONAL_FUEGOTEST_BRANCH}" \
> > +        "${FUNCTIONAL_FUEGOTEST_REPO}" \
> > +        "${fuego_release_dir}"
> > +    cd "${fuego_release_dir}"
> > +    git submodule update --init --recursive
> > +    cd -
> > +}
> > +
> > +function test_run {
> > +    cd "${fuego_release_dir}"
> > +    ./install.sh "${fuego_release_img_name}"
> > +    sudo -n docker rm -f "${fuego_release_container_name}" || true
> > +    sudo -n docker create -it --name "${fuego_release_container_name}"
> \
> > +        -p 8081:8080 \
> > +        --net="bridge" "${fuego_release_img_name}"
> > +
> > +    python3 ${TEST_HOME}/test_run.py
> > +    report "echo ok 1 minimal test on target"
> > +}
> > +
> > +function test_processing {
> > +    log_compare "$TESTDIR" "1" "^ok" "p"
> > +}
> > diff --git a/tests/Functional.fuegotest/spec.json
> > b/tests/Functional.fuegotest/spec.json
> > new file mode 100644
> > index 0000000..703cb0d
> > --- /dev/null
> > +++ b/tests/Functional.fuegotest/spec.json
> > @@ -0,0 +1,9 @@
> > +{
> > +    "testName": "Functional.fuegotest",
> > +    "specs": {
> > +        "default": {
> > +            "REPO":"FUEGOTEST_REPO",
> > +            "BRANCH":"FUEGOTEST_BRANCH"
> > +        }
> > +    }
> > +}
> > diff --git a/tests/Functional.fuegotest/test_run.py
> > b/tests/Functional.fuegotest/test_run.py
> > new file mode 100755
> > index 0000000..139ad9e
> > --- /dev/null
> > +++ b/tests/Functional.fuegotest/test_run.py
> > @@ -0,0 +1,19 @@
> > +#!/usr/bin/env python
> > +
> > +import pexpect
> > +
> > +# TODO: Check for errors
> >
> > As you see, this test is just a very simple bootstrap. The plan is to
> > include many more steps in it in the future.
> >
> > +
> > +c = pexpect.spawnu('sudo docker start --interactive --attach
> fuego-release-
> > container')
> > +c.sendline('\r\n')
> > +c.expect('# $')
> > +
> > +print('Adding Node')
> > +c.sendline('ftc add-nodes docker')
> > +c.expect('# $', timeout=120)
> > +
> > +print('Adding Jobs')
> > +c.sendline('ftc add-jobs -b docker -p testplan_docker')
> > +c.expect('# $', timeout=120)
> > +
> > +print('Node and Jobs have been added')
> > --
> > 2.16.1
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/fuego/attachments/20180219/60ec461a/attachment-0001.html>


More information about the Fuego mailing list