[Fuego] [PATCH 7/7] kernel_build: add multiple deploy options

Tim.Bird at sony.com Tim.Bird at sony.com
Wed Sep 5 04:51:45 UTC 2018


Comments inline below.

> -----Original Message-----
> From: Daniel Sangorrin
> 
> Provide a few simple deploy options for now. It needs refining.
> FIXTHIS: add support for dtbs
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> ---
>  engine/tests/Functional.kernel_build/fuego_test.sh | 33
> ++++++++++++++++++----
>  engine/tests/Functional.kernel_build/spec.json     |  2 +-
>  engine/tests/Functional.kernel_build/test.yaml     |  7 ++---
>  3 files changed, 32 insertions(+), 10 deletions(-)
> 
> diff --git a/engine/tests/Functional.kernel_build/fuego_test.sh
> b/engine/tests/Functional.kernel_build/fuego_test.sh
> index 678099a..6e4ac45 100755
> --- a/engine/tests/Functional.kernel_build/fuego_test.sh
> +++ b/engine/tests/Functional.kernel_build/fuego_test.sh
> @@ -69,13 +69,36 @@ function test_build {
>  }
> 
>  function test_deploy {
> -    if [ -z "$FUNCTIONAL_KERNEL_DEPLOY" ]; then
> -        FUNCTIONAL_KERNEL_DEPLOY="$LOGDIR"
> +    # FIXTHIS: add support for device trees
> +    if [ -z "$FUNCTIONAL_KERNEL_DEPLOY_METHOD" ]; then
> +        FUNCTIONAL_KERNEL_DEPLOY="log"
My preference would be to call this "logdir".  Let me know if you object.

>      fi
> 
> -    # FIXTHIS: output the modules and dtbs as well
> -    echo "Deploying kernel to $FUNCTIONAL_KERNEL_DEPLOY"
> -    cp $WORKSPACE/$JOB_BUILD_DIR/arch/$ARCHITECTURE/boot/*Image
> $FUNCTIONAL_KERNEL_DEPLOY
> +    if [ "$FUNCTIONAL_KERNEL_DEPLOY" == "log" ]; then
> +        echo "Deploying kernel to $LOGDIR"
> +        cp
> $WORKSPACE/$JOB_BUILD_DIR/arch/$ARCHITECTURE/boot/$FUNCTIONAL_
> KERNEL_BUILD_TARGET $LOGDIR
> +        if [ -f $WORKSPACE/$JOB_BUILD_DIR/modules.tar.gz ]; then
> +            cp $WORKSPACE/$JOB_BUILD_DIR/modules.tar.gz $LOGDIR
> +        fi
> +    fi
> +
> +    if [ "$FUNCTIONAL_KERNEL_DEPLOY" == "tftp" ]; then
> +        echo "Deploying kernel to /var/lib/tftp/"
> +        cp
> $WORKSPACE/$JOB_BUILD_DIR/arch/$ARCHITECTURE/boot/$FUNCTIONAL_
> KERNEL_BUILD_TARGET /var/lib/tftp/
> +        if [ -f $WORKSPACE/$JOB_BUILD_DIR/modules.tar.gz ]; then
> +            cp $WORKSPACE/$JOB_BUILD_DIR/modules.tar.gz /var/lib/tftp/
> +        fi
> +    fi
> +
> +    if [ "$FUNCTIONAL_KERNEL_DEPLOY" == "scp" ]; then
Since 'put' is the name of the Fuego remote copy operation,
I'd rather this deploy method was named 'put'.

> +        echo "Deploying kernel to board $NODE_NAME"
> +        put
> $WORKSPACE/$JOB_BUILD_DIR/arch/$ARCHITECTURE/boot/$FUNCTIONAL_
> KERNEL_BUILD_TARGET /boot
> +        if [ -f $WORKSPACE/$JOB_BUILD_DIR/modules.tar.gz ]; then
> +            put $WORKSPACE/$JOB_BUILD_DIR/modules.tar.gz /
> +            cmd "cd /; tar xvf modules.tar.gz"
> +        fi
> +        cmd "update-grub" || true
> +    fi
>  }
> 
>  function test_processing {
> diff --git a/engine/tests/Functional.kernel_build/spec.json
> b/engine/tests/Functional.kernel_build/spec.json
> index 06df53d..35412ed 100644
> --- a/engine/tests/Functional.kernel_build/spec.json
> +++ b/engine/tests/Functional.kernel_build/spec.json
> @@ -26,7 +26,7 @@
>              "config": "defconfig",
>              "target": "Image",
>              "params": "-j8",
> -            "deploy": "/var/lib/tftp/",
> +            "deploy_method": "tftp",
Hmmm.  this is OK, but possibly not as generic as I'd like.

I'm just brainstorming, but how about something like:
deploy_method: "local_dir",
deploy_path: "/var/lib/tftp"

The reason for this is that I used to have an elaborate system
with copies under /target/<board>/boot.  I don't think that
everyone uses /var/lib/tftp.
Currently, I use the path "/var/lib/tftpboot"

Other locations in the file system might be used as a rendezvous
point for external deploy systems.  I'm doing this for my
minnowboard, which uses an external program to boot into
a "safe" kernel, and write from a kernel build directory
directly to target:/boot

One idea, besides logdir, would be to put it into /fuego-rw/board/<board-name>/,
and always have a variable in Fuego for that location:
FUEGO_RW_BOARD_DIR
or something like that.  Maybe we could support a "board_dir" deploy_method.

I'd have to think about whether that would be racy (it probably would be - 
so locking might be needed).

>              "regex_p": "^  OBJCOPY arch/arm64/boot/Image"
>          }
>      }
> diff --git a/engine/tests/Functional.kernel_build/test.yaml
> b/engine/tests/Functional.kernel_build/test.yaml
> index a8e53d5..903f5ad 100644
> --- a/engine/tests/Functional.kernel_build/test.yaml
> +++ b/engine/tests/Functional.kernel_build/test.yaml
> @@ -30,11 +30,10 @@ params:
>          example: "-j8"
>          optional: yes
>          default: "-j$(nproc)"
> -    - deploy:
> -        description: folder to deploy the kernel after build
> -        example: "/var/lib/tftp/"
> +    - deploy_method:
> +        description: deploy method (log, tftp, scp)
>          optional: yes
> -        default: $LOGDIR
> +        default: log
>      - regex_p:
>          description: regular expression to confirm a build was succesful
>          example: "^  OBJCOPY arch/arm64/boot/Image"
> --
> 2.7.4

Note.  I reviewed the other patches in this series (the kernel_build 1-6) and
they all look fine on visual inspection.  I haven't had time to apply the
patches and try to run them.  This would take a while, and I trust your judgement.
I'm out of time tonight, but I'll likely apply these tomorrow, and give them a
smoketest, then push them and let you know when they're applied.

Let me know if you want to  re-spin 7 (this one) based on my feedback.  I don't
mind applying it as is and working on it more after it's applied (e.g. if you're OK with
changes.)

BTW - I have been working on external methods of deployment, that I was considering
putting into a separate Fuego test, then stringing the kernel build together with something
like "Functional.kernel_deploy" in a batch job, to perform the replacement of the software
under test on the board.  However, I wanted to fix some things about how we did batch
jobs first.  I'm inclined to leave this deployment stuff here in Functional.kernel_build for now.
We can always move it around later if I ever find the time to work on integrating my
deployement stuff with it.

One thing that kind of blows people's minds about Fuego, is that our 'deploy' operation
is a deploy of the test software, NOT the software under test (which we have historically
left as an exercise for the user).  It would be really nice to also support deploying (/installing)
the software under test (the kernel, modules and distro/rootfs) as well.  Doing this generically
is a big task, but this is a nice start.

Thanks very much.
  -- Tim


More information about the Fuego mailing list