[Fuego] [PATCH 10/14] Add a README.md
Guilherme Campos Camargo
guicc at profusion.mobi
Tue Apr 24 17:24:47 UTC 2018
Add a README.md that explains the usage of the fuego-release-test as
well as the usage of the helper scripts.
Signed-off-by: Guilherme Campos Camargo <guicc at profusion.mobi>
---
.../Functional.fuego_release_test/README.md | 182 ++++++++++++++++++
1 file changed, 182 insertions(+)
create mode 100644 engine/tests/Functional.fuego_release_test/README.md
diff --git a/engine/tests/Functional.fuego_release_test/README.md b/engine/tests/Functional.fuego_release_test/README.md
new file mode 100644
index 0000000..1ea6ab3
--- /dev/null
+++ b/engine/tests/Functional.fuego_release_test/README.md
@@ -0,0 +1,182 @@
+# Fuego Release Test
+
+This Fuego Functional test may be used for automating tests of new Fuego
+Releases through Fuego itself, or may be used in standalone mode (outside
+Fuego) to test a Fuego version that's present in your filesystem.
+
+## Dependencies
+
+ - Docker
+ - Python3
+ - Python-Pillow
+ - Pexpect
+ - Chromium
+ - SeleniumHQ
+
+## Usage
+
+### Within Fuego
+
+For using it within Fuego, simply add a `fuego-test` board and associate this
+test to it (`Functional.fuego_release_test`) with the following commands on
+Fuego shell.
+
+```
+$ ftc add-nodes fuego-test
+$ ftc add-jobs -b fuegotest -t Functional.fuego_release_test
+```
+
+After doing that, use Fuego as usual to trigger the `fuego_release_test` and
+check the results.
+
+By default `fuego_release_test` clones Fuego and Fuego-Core from the official
+master branches (default specs on `spec.json`), but `test` and a `next` specs
+are also already available on the `specs.json`. You can always edit the specs
+to add your preferred repos/branches.
+
+### Standalone
+
+Sometimes it's useful to execute `fuego_release_test` outside Fuego, with a
+Fuego version that's present locally in your filesystem.
+
+For doing that, simply execute the `test_run.py` script pointing to your Fuego
+installation directory.
+
+See examples below:
+
+Test a `Fuego` version that's located at `~/fuego` and use the current
+directory as working-dir (location where test assets will be saved and looked
+for (screenshots, for example).
+```
+$ ./test_run.py -w . ~/fuego
+```
+
+Same as above, but do not remove (neither stops) the `fuego-container` after
+the execution of the tests. You'll be able to access the inner Jenkins (Fuego
+that runs inside Fuego) through `${fuego_container_ip}:8080/fuego`. The
+`fuego_container ip` is dynamically set by docker, and is displayed in the
+first lines of the output of the script.
+```
+$ ./test_run.py --no-rm-container -p 8080 -w . ~/fuego
+```
+
+Output example with the IP of the container:
+```
+...
+test_run:DEBUG: Container 'fuego-release-container' created
+test_run:DEBUG: Starting container 'fuego-release-container'
+test_run:DEBUG: Running fetch_ip()
+test_run:DEBUG: Try number 1...
+test_run:DEBUG: Success
+test_run:DEBUG: Trying to reach jenkins at container 'fuego-release-container' via the container's IP '172.17.0.2' at port '8080'
+test_run:DEBUG: Running ping_jenkins()
+...
+```
+
+Docker commands will be executed from within the script. For that reason, you
+may be required to execute it with `sudo` or as a user with root permissions.
+
+## Test Classes
+
+Fuego Release Test implements a few wrapper classes on top of SeleniumHQ and
+Pexpect. Those classes are used for the instantiation of Test Case objects that
+are executed sequentially through `execute_tests()`.
+
+A few examples are given below:
+
+### Pexpect Commands
+
+#### ShExpect
+Execute a shell command expecting its output to match a given regexp.
+
+```
+ShExpect('ftc list-nodes -q', r'.*docker.*')
+```
+
+### Selenium Commands
+
+#### Visit
+Ask for SeleniumHQ to visit a given URL.
+
+```
+Visit(url='http://localhost:8080/fuego')
+```
+
+#### Click
+Find the first element containing a partial link text matching 'docker' and
+click it.
+
+```
+Click(By.PARTIAL_LINK_TEXT, 'docker'),
+```
+
+#### CheckText
+Find the element whose id is 'executors' and check if its text matches the
+given string ('docker.defaut...')
+```
+CheckText(By.ID, 'executors',
+ text='docker.default.Functional.hello_world'),
+```
+
+#### CheckScreenshot
+
+Find an element whose class name is 'page_generated', take a screenshot of it
+and compare with a reference screenshot, given a comparison mask (B/W in which
+Black regions will not be compared) and a threshold (percentage of non-matching
+pixels that are tolerated)
+```
+CheckScreenshot(By.CLASS_NAME, 'page_generated',
+ rm_images_on_success=False,
+ mask_img='screenshots/page-generated-mask.png',
+ ref_img='screenshots/page-generated.png',
+ threshold=0.1),
+```
+
+Take a full screenshot and compare with the image in
+`screenshots/full_screenshot.png` using the ignore-mask
+`screenshots/full_screenshot_mask.png`. The viewport resolution can be set on
+the SeleniumContext constructor.
+```
+CheckScreenshot(ref_img='screenshots/full_screenshot.png',
+ rm_images_on_success=False,
+ mask_img='screenshots/full_screenshot_mask.png',
+ threshold=0.01),
+```
+
+Note that the test by default does not store the currently captured screenshot
+and the diff image in case of success. If you'd like to see those images, set
+`rm_images_on_success` to False, and those images will be available in the
+working directory (script's -w argument).
+
+## Helpers
+
+A `take_screenshot.py` helper script has been implemented to aid the generation
+of reference screenshots (used by CheckScreenshot tests).
+
+The usage of the script is documented on `take_screnshot.py --help` and a few
+examples are given below:
+
+#### Full Screen
+
+Take a full viewport screenshot (default resolution is 1920x1080) from
+`http://localhost:8080/fuego` and save it in the current directory as
+`full_viewport.png`.
+
+```
+./take_screenshot.py http://localhost:8080/fuego
+```
+
+Take a full viewport screenshot as above, but sets the viewport to 800x600
+
+```
+./take_screenshot.py http://localhost:8080/fuego -r 800x600
+```
+
+#### Element Screenshot
+Finds a element by CLASS_NAME (-l locator) that matches the string
+'page_generated' (-p pattern) saving it in the current directory as
+`CLASS_NAME_page_generated.png`.
+
+```
+./take_screenshot.py http://172.17.0.2:8080/fuego/ -l CLASS_NAME -p page_generated
+```
--
2.17.0
More information about the Fuego
mailing list