[Fuego] [QUESTION] bc : problem occurs when adding test case to bc

Tim.Bird at sony.com Tim.Bird at sony.com
Wed Oct 10 23:54:23 UTC 2018



> -----Original Message-----
> From: Daniel Sangorrin 
> 
> > >Daniel just recently added a feature to record the md5sum for tarballs.
> > >It would be good to add some support to print a warning, or to
> automatically
> > remove the old build
> > >directory, and re-unpack the tarball, if the TESTSUITE_VERSION of the
> tarball is
> > different from the
> > >one in the build directory.
> > I'd like to learn the relevant code and try to modify it.
> > In the process, if there are any problems, I'll ask you again ^_^
> 
> Just in case, my patch "if_source_changed" was not accepted and that patch
> did remove the old build and re-unpack the tarball. It also updated the test's
> source git repository.
> 
> I believe that it was not accepted because kernel_build currently uses the
> test_build phase and we want to use the test_run phase.

No.  That was a separate issue.  I've gone back and forth on that aspect of the
Functional.kernel_build test.  Right now, I think I'm in the camp of wanting to
do the build in the test_run phase.

But the issue with if_source_changed was the change in enumerated types
for the --rebuild flag.  I have a nagging feeling that having the default value
be something strange, like "if_source_changed", would be confusing to users.

Pardon the long post, but I'm going to think through this as I write this...

Our current meanings are:
 "--rebuild true" - always rebuild from the test software, no matter what
 "--rebuild false" - build the software if it has not already been built.

Here’s the state machine for this:
rebuild flag   |  build state    |    action
---------------------------------------------
false                 |  not built     | build the software
false                |  built             | do not build the software
true                 | not built      | build the software
true                 | built             | build the software

I think this is highly intuitive (partly because 'rebuild' has the 're' prefix
which indicates that the flag has something to do with building *again* as
opposed to the first time.

Introducing a new feature, the ability to automatically update
the software, introduces some new state into the state machine.  the old
state just consisted of "built" or "not built".  The new states consists of:
"built", "built but out of date", and "not built"

The proposal was (I believe):
rebuild flag   |  build state    |    action
--------------------------------------------------------
false                 |  not built       | build the software
false                |  built                | do not build the software
false                | built, out of date | do not build the software (I think?)
if_source_changed| not built | build the software
if_source_changed| built       | do not build the software
if_source_changed | built, out of date | update and rebuild the software
true                 | not built        | build the software
true                 | built               | build the software
true                 | built, out of date | update and rebuild the software

Another issue is that I'm not sure that the command line is the best
place to handle the new state.  It seems to me that deciding whether
to update the test software might be a global decision, that is better
placed into a configuration value, than in individual test jobs.  In other words
it might make more sense as a site-wide policy.

Another issue is that this might not really a job-specific setting, but rather
a test-specific setting.  Some software you might want to get the 
latest version for, and others you don't care.  So maybe there should be
a flag in the test.yaml file or a new variable in fuego_test.sh.

For my lab, I would prefer a global policy of no automatic test program
upgrades.  But I might make exceptions for some tests, which I would
change in the jobs for those tests, or it the test meta-data for those tests.

I think I'm starting to lean towards the following solution:
2 flags: one for forced rebuild and one for automatic update,
with defaults for both in the global fuego.conf file.

rebuild flag   | auto-update flag |  build state   | action
--------------------------------------------------------------------------
false               | false                          | not built        | build the software
false               | false                          | built                | don't build the software
false               | false                         | built, out of date | don’t re-build the software
false               | true                           | not built        | build the software
false               | true                           | built                | don't build the software
false               | true                         | built, out of date | ** update and re-build the software **
true               | false                          | not built        | build the software
true               | false                          | built                | build the software
true               | false                         | built, out of date | ** rebuild the old software **
true               | true                          | not built        | build the software
true               | true                          | built                | rebuild the software
true               | true                          | built, out of date | ** update and re-build the software **

I don't like flag proliferation, because I want most users to be able to use
Fuego without having a huge learning curve.  So my preference would be to
add the flag to ftc, but not put it in the command-lines in newly created jobs.

Feedback?
 -- Tim

P.S. Completely independent of this is the actual code to perform
the out-of-date detection and the bring-up-to-date operation.
We should have these no matter what flag solution we come up with.
(So - keep the if_source_changed patch around.  It has all the
actual implementation needed for the real work, once the decisions
are made.)




More information about the Fuego mailing list