[Fuego] [PATCH 05/14] Allow working_dir and install_dirs to be different

[Tim.Bird at sony.com]

> -----Original Message-----
> From: Guilherme Camargo 
> 
> On Wed, Apr 25, 2018 at 8:20 PM, Tim Bird wrote:
> 	> -----Original Message-----
> 	> From: Guilherme Campos Camargo
> 	>
> 	> The `working_dir` is where all the test assets reside and also where
> all
> 	> the results (for instance the resulting images for the
> CheckScreenshot
> 	> test) will be stored after the tests.
> 	>
> 	> The `install_dir` is where the install script can be found.
> 	>
> 	> `install_dir` is a required argument, while `working_dir` is optional
> 	> and defaults to `install_dir`.
> 	>
> 	> On this patch we're also modifying fuego_test.sh to copy the
> required
> 	> assets from the test directory to the buildzone.
> 
> 	I'm not sure why this is needed.  I hate duplicating the files
> unnecessarily.
> 	We could have lots of them eventually.
> 
> 	Why can't the assets just be accessed from the TEST_HOME
> directory?
> 	 -- Tim
> 
> 
> 
> 
> ​We currently have two working dirs: one in which fuego - and the
> install script - resides  (install-dir) , and another (-w) in which the
> test will look for assets and store things (as screenshot diff results,
> etc.).
> 
> I don't like the idea of having things stored in the TEST_HOME,
> given that it's the TEST_HOME is inside fuego-core and I also don't
> think we should store these files in the install-dir, one option
> for cleaning this up would be:
> 
> - Add a new argument to test_run.sh to separate the assets dir
> (TEST_HOME/screenshots)
> from where the captured screenshots and diffs will be stored), having
> something like:
> 
>    $ test_run.py" "${fuego_release_dir}/fuego" -a ${assets_path} -w
> ${output_path}
> 
> What do you think?

OK - I understand the design better.  I didn't realize that we have artifacts we
are saving from the run.  What you have proposed looks good to me.
There are pros and cons to putting the reference assets in the same directory
as the artifacts generated from the run.  It's easier for users to look at them and
see what happened if they're all in the same directory.  But it's also nice to
not replicate the reference assets for every run of the test.

I would put the generated artifacts in the run directory (the log directory), not
the build directory.  (I didn't check to see where you're putting them now).

In any event, specifying the reference asset path seems like a good idea.

 -- Tim