[Fuego] issue report: “PER_JOB_BUILD” related

[Bird, Timothy]

> -----Original Message-----
> From: Daniel Sangorrin [mailto:daniel.sangorrin at toshiba.co.jp]
> > -----Original Message-----
> > From: Bird, Timothy on Wednesday, November 08, 2017 9:36 AM
> > > -----Original Message-----
> > > From: Liu, Wenlong on Monday, November 06, 2017 7:54 AM
> > > Just report some issue with “PER_JOB_BUILD”.
> > >
> > > Issue 1：
> > >
> > > Jobs “boardx.bc-add.Functional.bc’ and “boardx.bc-
> by2.Functional.bc’cannot
> > > all PASS in the current Fuego because “Functional.bc” has 2 different
> tarball.
> > >
> > > So, variable “PER_JOB_BUILD” might be added to spec file.
> 
> I didn't know about this functionality.

Yeah.  Sorry about that.  This was added after LTP builds started driving
me nuts.  It is on my list of things to document for the 1.2(.1) release.

> 
> >
> > That would solve this problem.  But looking at the Functional.bc test, having
> > two different tar files, each of which contains a ~150-byte script, seems
> really
> > silly.  But this test is used for demo/tutorial purposes.
> >
> > Daniel - was this just done as a demonstration of a putting the tarball build
> > variable in a spec file?
> 
> Yes, in this case it was just a demonstration.
> But in the kernel build test for example each kernel is built from a different
> git
> repository.

Yes, I see that.

> 
> > If so, I'll a per_job_build var to this, to get the desired behavior.  Another
> > option would be to use a different script name (rather than have both
> > tarballs provide bc-device.sh).
> 
> I think we should use a more unique name for the TEST_BUILD_DIR and
> remove
> PER_JOB_BUILD. Even with a normal test, the current TEST_BUILD_DIR has
> issues.
> 
> Current:
>   TEST_BUILD_DIR="${TESTDIR}-${PLATFORM}"  (Ex: Benchmark.cyclictest-
> x86_64)
> 
> This has several issues:
>   - When running the same test on multiple boards that use the same
> toolchain we
>      have a race condition.
I changed the locking to address this.  The locking is per $JOB_BUILD_DIR.
When the build directories are shared, this works because then the lock file
is actually in the correct $TEST_BUILD_DIR, and it locks all jobs out of 
the shared directory.

>   - As Liu pointed, if the spec points to a different tarball or git repository
> there will
>      be conflicts
 
That's exactly what PER_JOB_BUILD addresses.  I should have added it to
Functional.kernel_build.

> Proposal: we should use JOB_BUILD_DIR instead:
>    <board>.<spec>.<testdir>-<platform>

In terms of visibility to the test, this is actually what we do.
We always build in $JOB_BUILD_DIR.  However, most of the time
(ie, when specs use the same source and binaries as other specs,
and do not have 'per_job_build' defined), this is a symlink
to $TEST_BUILD_DIR, which is shared among all specs (for
builds for the same $PLATFORM)

Sharing the source is fairly important for extremely large tests
like LTP.  I run LTP with multiple specs quite frequently, and I was
ending up 10s of gigabytes of duplicated source code in my
build directory.  We are likely to have many specs for Functional.kselftest,
which will have the same problem, and for which sharing the source
should not be a problem.  Note that sharing the build directory
also saves us a lot of build time (besides file system space). In
the case of LTP, the build takes about 10 minutes, and it was really
rankling me to have to build for myboard.smoketest.Functional.LTP
and myboard.quickhit.Functional.LTP and myboard.ptsonly.Functional.LTP,
etc., when they all had identical source.

I think it might be useful, in the case of Functional.kernel_build, to 
put the per_job_build variable into fuego_test.sh, instead of into
the spec.json file.
We should be able to define FUNCTIONAL_KERNEL_BUILD_PER_JOB_BUILD="true"
in fuego_test.sh, and not have to put 'per_job_build' into every single
spec entry for that test.  I could also modify the test in functions.sh:pre_build,
to check for just PER_JOB_BUILD, to make use of this variable simpler
from fuego_test.sh.

Let me know what you think.
 -- Tim