[Fuego] [PATCH 0/8] fuego-release-test fixes

[Tim.Bird at sony.com]

This looks nice.  This test will be now be a candidate for our
per-testcase parsing.   See the parser.py for curl as an example.

Thanks.

All patches in this series were applied and pushed.
 -- Tim


> -----Original Message-----
> From: fuego-bounces at lists.linuxfoundation.org [mailto:fuego-
> bounces at lists.linuxfoundation.org] On Behalf Of Guilherme Campos
> Camargo
> Sent: Wednesday, May 02, 2018 7:21 AM
> To: fuego at lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 0/8] fuego-release-test fixes
> 
> Hello, Tim
> 
> These patches are implementing the changes you've requested in your
> review and a few other fixes, in summary:
> 
>   - Replace working_dir with two separate dirs: resources_dir (for test
>     assets) and output_dir (for test resulting files)
> 
>   - Print the results of the tests individually, prefixing them with an
>     id and with a summary - and parse them individually with log_compare
>     ("ok" and "not ok")
> 
>     Output Example (Log level set as INFO)
> 
>     ```
>     ...
>     test_run:REPORT: run 15 check text build queue fuego board check
>     test_run:INFO: Executing Selenium Command 'CheckText'
>     test_run:INFO:   Matching text found
>     test_run:REPORT: ok 15
>     test_run:REPORT: run 16 click img build hello world
>     test_run:INFO: Executing Selenium Command 'Click'
>     test_run:INFO:   Element clicked
>     test_run:REPORT: ok 16
>     test_run:REPORT: run 17 check text executors hello world
>     test_run:INFO: Executing Selenium Command 'CheckText'
>     test_run:INFO:   Matching text found
>     test_run:REPORT: ok 17
>     test_run:REPORT: run 18 check screenshot side-panel-tasks
>     test_run:INFO: Executing Selenium Command 'CheckScreenshot'
>     test_run:INFO:   Resulting difference (0.00123834) below threshold (0.1)
>     test_run:INFO:   Element's screenshot matches the reference
>     test_run:REPORT: ok 18
>     test_run:REPORT: run 19 check screenshot footer
>     test_run:INFO: Executing Selenium Command 'CheckScreenshot'
>     test_run:INFO:   Resulting difference (0.0) below threshold (0.1)
>     test_run:INFO:   Element's screenshot matches the reference
>     test_run:REPORT: ok 19
>     test_run:INFO: All tests finished with SUCCESS
>     ```
> 
>     Output Example (Log level set as REPORT)
> 
>     ```
>     ...
>     test_run:REPORT: run 15 check text build queue fuego board check
>     test_run:REPORT: ok 15
>     test_run:REPORT: run 16 click img build hello world
>     test_run:REPORT: ok 16
>     test_run:REPORT: run 17 check text executors hello world
>     test_run:REPORT: ok 17
>     test_run:REPORT: run 18 check screenshot side-panel-tasks
>     test_run:REPORT: ok 18
>     test_run:REPORT: run 19 check screenshot footer
>     test_run:REPORT: ok 19
> 
>   - Do not require running the test with sudo - See the commit message
>     `Remove requirement of running the test script as sudo`
> 
>   - Implement a few other minor fixes as updating the readme and catch
>     an PermissionError on image.save().
> 
> Please let me know what you think.
> 
> Thanks
> 
> --
> Guilherme
> 
> Guilherme Campos Camargo (8):
>   Update ip example of README.md
>   Replace working_dir arg with resources_dir and output_dir
>   Remove requirement of running the test script as sudo
>   Catch PermissionError when saving screenshots
>   Add new log level REPORT
>   Add `description` optional argument to test cases
>   Add description to tests in COMMANDS_TO_TEST
>   Log test results with the REPORT log level and print descriptions
> 
>  .../Functional.fuego_release_test/README.md   |  45 ++---
>  .../fuego_test.sh                             |  13 +-
>  .../Functional.fuego_release_test/test_run.py | 159 +++++++++++++-----
>  3 files changed, 144 insertions(+), 73 deletions(-)
> 
> --
> 2.17.0
> 
> _______________________________________________
> Fuego mailing list
> Fuego at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego