Fuze™ Test Executive CLI Specification

Fuze™ Test Executive CLI Specification#

Introduction#

The Fuze™ Test Executive CLI (run-atf.py) provides the user-facing command-line interface for executing automated test cycles.

This tool is used by both human testers and CI systems.

Usage#

Basic Syntax:

python run-atf.py [options]

Configuration Sources#

The Fuze™ Test Executive considers multiple input sources when preparing to execute a new test cycle. Test cycle parameter sources are categorized by how “sticky” they are, i.e., whether they change frequently or infrequently. Test cycle parameters that generally change infrequently are captured in configuration files by the test cycle designer, then read by the Executive at execution-time. Test cycle parameters that can change frequently from one execution (or CI instance) to another are specified on the command line.

The Executive reads test cycle configuration parameters and options from:

Command Line Arguments#

-h|–help

Display usage: a list of recognized options, and if bounded, their accepted values.

-m|–module

Name of the logical software Module Under Test (MUT). This is the logical name for a software package under test, as configured in the Environment configuration file. Required.

-d|–dut

Device (type) Under Test (DUT). Unrestricted string, but must be defined in the Environment Configuration file to be useful.

-r|–runtype

Type of test run to execute. Example values: informal, bvt, periodic. [Default == informal]. Informs the Executive where to find (auto-select) test suites to execute, unless overridden by other means.

-l|–location

Test resources location logical name. This can refer to a physical DUT farm, e.g., a lab, or a local host setup. [Default == local].

-ts|–testsetup

Test setup profile. This has implications for what occurs after an Executive-aware DUT power cycle. For example, ram implies the DUT must be prepared (programmed) after every known DUT power cycle, whereas flash implies the DUT is prepared only at the start of the test cycle, regardless of subsequent Executive-aware DUT power cycles. Options include ram, flash. [Default == ram].

-mv|–moduleversion

Version string of the MUT. Defaults to latest available (by file system time stamp) discovered under the host search path(s) defined in the Resources Configuration file.

-p|–project

Project name or identifier in accordance with test content directory hierarchy, i.e., <project>/<module>/. [Default == helloworld].

-tr|–testrunid

Unique identifier string for this test run. For coordination with external systems and/or teams. Optional, but logged and retained with results during test cycle execution if defined.

-s|–suite

Space delimited list of test suite file path(s) to find and execute (e.g., relative/path/to/ts_feature_1.json). This is a dynamic method to override the test suite auto-select, or statically defined tests list confiuration file (if defined).

-sdp|–skipdutprep

Boolean flag to skip DUT preparation steps, including the initial autonomous DUT power cycle. This is a method to manually prepare the DUT, overriding the default or context selected DUT preparation algorithm, and leave this preparation undisturbed (by the Executive) for the duration of the test cycle.

-i|–include

Space delimited list of DUT names to include in test cycle participation, to the exclusion of any others. This is a method to limit the available DUT list specified in the location-selected Resources configuration file, e.g., forcing all testing onto a single or subset of DUTs for debugging. In case of conflict with the -e|–exclude option, this value takes precedence. Optional.

-e|–exclude

Space delimited list of DUT names to exclude from test cycle participation. This is a method to limit the available DUT list specified in the location-selected Resources configuration file, e.g., avoiding testing on a sick DUT in repair. In case of conflict with the -i|–include option, this value is ignored. Optional.

Example Usage#

All commands are executed from the Fuze™ Test executive/ folder.

Run a test suite using config only:

./run-atf.bin --project demos --module helloworld

Override key parameters:

./run-atf.bin --project demos --module helloworld --runtype bvt --location LAB1 --dut stm32_dut --suite /home/user1/in_progress/ts_feature_1.json

Override module version:

./run-atf.bin --project demos --module helloworld_stm32 --moduleversion 1.2.3

Skip DUT prep:

run-atf.bin --project demos --module helloworld --skipdutprep

Results#

Results are written to a timestamped results directory under the executive/ directory:

executive/results/<timestamp>-<module>v<version tested>-<runtype>/

Includes:

  • Local and Remote Executive Level Logs

  • Test Case oriented Results (CSV format)

  • Full DUT oriented Test Case Results (JSON format)

Contents