[Fuego] [PATCH] docs: Convert pages categorized as "Feature reference"
Pooja
pooja.sm at pathpartnertech.com
Thu Nov 5 05:23:16 UTC 2020
From: Pooja More <pooja.sm at pathpartnertech.com>
List ofpages converted:
Fuego_charting
Log_files
Jenkins
Jenkins_User_Interface
Signed-off-by: Pooja More <pooja.sm at pathpartnertech.com>
---
docs/rst_src/Fuego_charting.rst | 19 +++
docs/rst_src/Jenkins.rst | 47 ++++++
docs/rst_src/Jenkins_User_Interface.rst | 249 +++++++++++++++++++++++++++++++-
docs/rst_src/Log_files.rst | 185 ++++++++++++++++++++++++
4 files changed, 498 insertions(+), 2 deletions(-)
create mode 100644 docs/rst_src/Fuego_charting.rst
create mode 100644 docs/rst_src/Jenkins.rst
create mode 100644 docs/rst_src/Log_files.rst
diff --git a/docs/rst_src/Fuego_charting.rst b/docs/rst_src/Fuego_charting.rst
new file mode 100644
index 0000000..3ff9380
--- /dev/null
+++ b/docs/rst_src/Fuego_charting.rst
@@ -0,0 +1,19 @@
+################
+Fuego charting
+################
+
+Fuego has the capability of creating charts (either plots or tables)
+from results data. These charts are used either in the Jenkins
+interface, or in reports generation.
+
+This page has a few notes on Fuego's charting features, for users.
+
+Charts are produced during the "processing" phase of test execution.
+This is done after the test results have been parsed from the test log
+and the results placed in the unified output format (run.json) file.
+
+See :ref:`Jenkins Visualization` for information about charts that are
+produced for the Jenkins interface. See :ref:`process <parser func
+process>` and :ref:`chart_config.json <chart config.json>` for information about the variables
+and files used for chart creation, and
+how they can be customized.
diff --git a/docs/rst_src/Jenkins.rst b/docs/rst_src/Jenkins.rst
new file mode 100644
index 0000000..342ec7d
--- /dev/null
+++ b/docs/rst_src/Jenkins.rst
@@ -0,0 +1,47 @@
+############
+Jenkins
+############
+
+Jenkins is a continuous integration system with a lot of features and
+a rich plugin ecosystem. It is used as the default front end (user
+interface) for the Fuego test framework.
+
+The Jenkins website is at: `<https://jenkins.io/>`_
+
+=================================
+How Jenkins is used by Fuego
+=================================
+
+Jenkins is used to organize the tests provided by Fuego, and present
+the user interface for test selection, test execution, and information
+about test history, logs, etc.
+
+Jenkins is also used to see the tests for a particular set of boards.
+
+Jenkins also provides the triggers for the test system. That is,
+Jenkins initiates jobs, as configured by the user, for starting tests
+and collecting test results.
+
+
+======================
+Miscelaneous notes
+======================
+
+ * Fuego logs (test log, devlog, syslog and parsed log - see :ref:`Log
+ files` are not available through the Jenkins interface (as of
+ Aug-2016). This seems like a big oversight. See :ref:`Issue 0010`
+
+ * builds are marked by Jenkins as: successful, failed, stable, unstable.
+
+ * a freestyle build is successful if the return code from executing
+ the code snippet is 0
+ * a freestyle build is considered 'failed' if
+ the return code from executing the code snippet is non-zero
+ * a build is stable unless explicitly marked otherwise by some
+ Jenkins action
+
+ * they are usually marked so in a post-test operation, by
+ something like TextFinder, groovy, or some other thing
+
+ * see `<http://stackoverflow.com/questions/8148122/
+ how-to-mark-a-build-unstable-in-jenkins-when-running-shell-scripts>`_
diff --git a/docs/rst_src/Jenkins_User_Interface.rst b/docs/rst_src/Jenkins_User_Interface.rst
index 022dee7..b379796 100644
--- a/docs/rst_src/Jenkins_User_Interface.rst
+++ b/docs/rst_src/Jenkins_User_Interface.rst
@@ -1,5 +1,250 @@
-.. _jUsrinterface:
-
#######################
Jenkins User interface
#######################
+
+By default, Fuego uses the Jenkins continuous integration system to
+manage boards, tests, logs, and test results.
+
+The Jenkins user interface is web-based. This page shows several
+screenshots of different pages in the Jenkins interface.
+
+Through this interface, you can see the status of tests that have run,
+review the logs for tests, and schedule new tests to run on target
+boards. You also use this interface to add new boards and new tests
+to the system.
+
+Note that Jenkins objects are:
+
+ * nodes
+ * jobs
+ * builds
+ * views
+
+These are different from the Fuego names for the same objects. The
+first three of these Jenkins objects correspond to the Fuego objects
+of: **boards**, **tests** and **runs**, respectively.
+
+==================
+Main dashboard
+==================
+
+The main dashboard of Jenkins looks like the following:
+
+New Installation
+====================
+
+When Fuego has just been installed, there is nothing in the Jenkins
+interface (no nodes, jobs or views). The interface should look
+something like this:
+
+.. image:: ../images/fuego-1.1-jenkins-dashboard-new.png
+ :width: 900
+
+With a single node (board) added
+===================================
+
+Here is the main dashboard of Jenkins, after a single node (called
+'beaglebone' in this case) has been added. Note the node (board)
+appears in the left sidebar under "Build Executor Status":
+
+.. image:: ../images/fuego-1.1-jenkins-dashboard-beaglebone.png
+ :width: 900
+
+
+
+With beaglebone node and jobs
+=================================
+
+Here is the main dashboard of Jenkins, showing a single node
+(beaglebone) and jobs for this board.
+
+.. image:: ../images/fuego-1.1-jenkins-dashboard-beaglebone-jobs.png
+ :width: 900
+
+
+Dashboard with jobs in Build Queue
+===================================
+
+Here is the Jenkins dashboard with a lot of jobs in the Build Queue.
+Note the list of jobs in the left side-bar, in the "Build Queue" pane.
+
+.. image:: ../images/fuego-1.1-jenkins-dashboard-batch-build-queue.png
+ :width: 900
+
+==============
+Node pages
+==============
+
+If you click on the node in the **Build Executor Status** pane, then
+you can see a list of the jobs associated with a node.
+
+
+Node status page
+=======================
+
+Here is the status for the beaglebone node.
+
+.. image:: ../images/fuego-1.1-jenkins-beaglebone-node.png
+ :width: 900
+
+
+==============
+Job pages
+==============
+
+If you click on a job in the Jenkins interface, you can see
+information about an individual job. This page shows information about
+the status of the job, including a Build History for the job (in the
+left sidebar).
+
+You can start a job by clicking on the "Build Now" button in the left
+menu.
+
+Functional job status page
+=================================
+
+Here is a page showing the status information for a Functional test
+called 'hello_world'. The main area of the screen has information
+about the last successful and failed builds of the test. Note the
+left sidebar pane with "Build History", to see individual test
+execution results.
+
+.. image:: ../images/fuego-1.1-jenkins-hello_world-job.png
+ :width: 900
+
+
+Benchmark job - starting a build
+=====================================
+
+Here is a picture of a job build being started. Note the progress bar
+in the **Build History** pane in the left sidebar.
+
+.. image:: ../images/fuego-1.1-jenkins-dhrystone-start-build.png
+ :width: 900
+
+
+Benchmark job - before successful execution
+================================================
+
+Before a Benchmark job completes it has no data to plot on it's chart,
+and appears similar to a Functional Job status page:
+
+.. image:: ../images/fuego-1.1-jenkins-Dhrystone-job.png
+ :width: 900
+
+
+
+Benchmark job - with plot of metrics
+========================================
+
+Normally, a Benchmark page shows one or more plots showing the values
+for data returned by this benchmark.
+
+.. image:: ../images/fuego-1.1-jenkins-dhrystone-job-plot.png
+ :width: 900
+
+===============
+Build pages
+===============
+
+A build page shows the results of a single execution of a job (test)
+on a board. You can click on the build number in the Jenkins
+interface to see this page.
+
+
+Results from a job build
+==============================
+
+Here are the results from the execution of the "hello world" job.
+This was the results of running the Fuego test
+"Functional.hello_world" on a beaglebone board.
+
+.. image:: ../images/fuego-1.1-jenkins-hello_world-build.png
+ :width: 900
+
+Test log results
+====================
+
+You can examine the different logs for each test. Each test produces
+a log from the program that ran on the board. This is available by
+following a link called "log" from the 'build' page for that test run.
+You can see the console log, which shows the output of commands for
+this test, by clicking on "console log" in the build interface (or the
+build drop-down menu in the **Build History** list).
+
+Drhystone test log
+--------------------------
+
+Here are results from a run of the Dhrystone test on a beaglebone
+board:
+
+.. image:: ../images/fuego-1.1-jenkins-dhrystone-log.png
+ :width: 900
+
+Jenkins Console log
+---------------------
+
+Here is the console log for a test executed on the beaglebone:
+
+.. image:: ../images/fuego-1.1-jenkins-console-log.png
+ :width: 900
+
+==============
+View pages
+==============
+
+A **view** is an area in the main dashboard that shows a collection
+of jobs with a particular set of status columns for each job. They
+appear as tabs in the main dashboard view of Jenkins. You can create
+your own view to see a subset of the jobs that are available in
+Jenkins
+
+Here are some screen shots showing how to add a new view to Jenkins.
+
+
+Screen to add a new view
+==============================
+
+Here is the screen to add a new view to Jenkins.
+
+.. image:: ../images/fuego-1.1-jenkins-add-view-beaglebone.png
+ :width: 900
+
+
+Screen to configure the view settings
+===========================================
+
+Here is the screen to configure view settings. Note the use of a
+regular expression to control what jobs to see in this view. You can
+also control what status columns to display in the view.
+
+.. image:: ../images/fuego-1.1-jenkins-config-view-beaglebone.png
+ :width: 900
+
+=======================
+Other Jenkins pages
+=======================
+
+
+Build History
+==================
+
+The global build history page is available by clicking on the **Build
+History** link in the main dashboard page of Jenkins. It shows the
+execution time and status for a recent time period.
+
+.. image:: ../images/fuego-1.1-jenkins-build-history.png
+ :width: 900
+
+
+Jenkins management
+=======================
+
+You can manage Jenkins using the **Manage Jenkins** page, available
+from the top-level dashboard page in Jenkins. From here you can
+update Jenkins itself, install or remove plugins, and perform other
+management operations for Jenkins.
+
+.. image:: ../images/fuego-1.1-jenkins-management.png
+ :width: 900
+
diff --git a/docs/rst_src/Log_files.rst b/docs/rst_src/Log_files.rst
new file mode 100644
index 0000000..b9caf68
--- /dev/null
+++ b/docs/rst_src/Log_files.rst
@@ -0,0 +1,185 @@
+##################
+Log files
+##################
+
+
+During test execution, Fuego collects several different logs.
+
+These represent different aspects of the system, and are used for
+different purposes.
+
+FIXTHIS - finish documenting log files on the Log files page.
+Specifically, document parsed logs better (describe how they are
+generated in log_compare)
+
+Here are the main logs collected or generated during a test:
+
+ * **console log**
+ * **devlog**
+ * **syslog**
+ * **test log**
+ * **parsed log**
+
+These are located in the 'run' directory (also know as the 'log'
+directory), at:
+``/fuego-rw/logs/<test_name>/<board>.<spec>.<build_id>.<build_number>``
+
+There are additional logs that may be defined for a test, which are
+used in post-processing and checking of results for the test.
+
+ * **pn log**
+ * **reference log**
+
+================
+Results logs
+================
+
+These logs are the results of test execution, and have output from
+different parts of the system.
+
+console log
+==================
+
+The console log is collected by Jenkins during the entire execution of
+the test. It is dominated by the invocation of the test base script,
+which is executed as a shell script, with the 'set -x' parameter set.
+
+During execution of the base script, several "source" statements are
+encountered, nesting the invocation of scripts multiple times. The
+number of '+' signs preceding a line in the console log indicates the
+depth (or invocation nesting level) for that line, in the shell
+execution.
+
+There are "phase" messages added to the log during the test, which can
+help identify which part of the test a particular sequence is in.
+This can help you debug what part of a test (pre_test, build, deploy,
+run, post_test) is failing, if any.
+
+The second major part of the script is the post_test phase, which is
+also executed as a shell script fragment, utilizing the prolog
+generated previously, and documenting the gather of logs and
+determination of final test result.
+
+The location of the console log for a test run is at:
+``/var/lib/jenkins/jobs/<jobname>/builds/<build_number>/log``
+
+If a test was executed by ftc instead of Jenkins (that is, directly
+from the command line), then the consolelog is in the Fuego log
+direcotry and is called: ``consolelog.txt``
+
+
+devlog
+===========
+
+This is a summarized list of operations performed during the execution
+of a test. These entries are created when internal routines use the
+function :ref:`report_devlog <function report devlog>` to add a line to
+this log.
+
+The name of the devlog for a test run is ``devlog.txt``
+
+syslog
+===========
+
+The syslog records the messages from the target system's log. There
+are two of these recorded, one during the pre_test (called the
+"before" log), and one during the post_test (called the "after" log).
+
+This includes messages from the kernel (if a logger is transferring
+the messages from the kernel to the system log), as well as messages
+from programs running on the system (that output to their status to
+the syslog).
+
+The names of the syslogs for a test are:
+
+ * ``syslog.before.txt``
+ * ``syslog.after.txt``
+
+test log
+============
+
+This is the output produced by the test program itself. It is
+collected with the *report* and *report_append* functions, which
+save the standard output from the commands they run into a log file
+which is retrieved from the target at the end of the test.
+
+The name of the test log for a test is: ``testlog.txt``
+
+parsed log
+===============
+
+The 'parsed' version of the log, is one that has been grep'ed for a
+regular expression using log_compare. The parsed log should consist
+of individual lines showing a pass, fail or skip for each sub-test in
+a test suite.
+
+The names of the parsed logs for a test are:
+
+ * *testlog.p.txt*
+ * *testlog.n.txt*
+
+==================
+Reference logs
+==================
+
+Fuego also uses reference logs during test results processing, to
+compare against the logs that are generated during a test:
+
+pn log
+============
+
+A test may provide a so-called 'pn log', which is a log file recording
+the positive and negative results from a previous test run.
+
+If this file exists in the test directory, it is used during
+post-processing (in the :ref:`log_compare <function log compare>`
+function), to see if the positive or negative results from the current
+log match those from the saved pn log.
+
+reference log
+==================
+
+This file holds the operations and threshold values for a Benchmark
+test.
+
+See :ref:`reference.log`
+
+============
+summary
+============
+
+In the Fuego version 1.1 (early 2017), the log directories are as
+follows:
+
+ * Fuego logs:
+
+ * ``/fuego-rw/logs/<testname>/<board>.<spec>.<build_id>.<build_number>``
+
+ * run.json - saved by test itself
+ * devlog.txt - written by report_devlog
+ * syslog.before.txt - saved by ov_rootfs_logread (dump_syslogs)
+ * syslog.after.txt - saved by ov_rootfs_logread
+ * testlog.txt - saved by get_testlog
+ * testlog.p.txt - created by log_compare
+ * consolelog.txt - created by ftc (or link to Jenkins console log)
+ * res.json - created by bench_processing (parser.py for a Benchmark test)
+ * fres.json - created by post_test (parser.py for a Functional test
+
+ * jenkins files:
+
+ * ``/var/lib/jenkins/jobs/<jobname>/builds/buildnum/``
+
+ * build.xml
+ * changelog.xml
+ * log - console log created by Jenkins
+
+ * per-test data files:
+
+ * ``/fuego-rw/logs/<testname>``
+
+ * plot.data
+ * plot.png
+ * metrics.json - list of metrics for this test
+ * <testname>.<metric>.json - list of data values for a metric
+ * <testname>.info.json - list of meta-data for each data line (device, firmware, platform data points)
+
--
2.7.4
--
This
message contains confidential information and is intended only
for the
individual(s) named. If you are not the intended
recipient, you are
notified that disclosing, copying, distributing or taking any
action in
reliance on the contents of this mail and attached file/s is strictly
prohibited. Please notify the
sender immediately and delete this e-mail
from your system. E-mail transmission
cannot be guaranteed to be secured or
error-free as information could be
intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain
viruses. The sender therefore does
not accept liability for any errors or
omissions in the contents of this
message, which arise as a result of e-mail
transmission.
More information about the Fuego
mailing list