Test Content Creation

Test Content Creation#

Introduction#

This page shows you the details of test content creation.

Test Content Organization#

Fuze™ Test test content is organized into test cases and test suites contained within the context of a test execution. Test cases are the atomic units of test. You create one file per test case. A test suite is a single file that contains a logical grouping of test cases. Test cases can be shared amongst test suites, and test multiple test suites can be applied to a single text execution.

_images/c2l-atf-test-exec-suites-cases.png

Test Suites are Logical Grouping of Test Cases Under a Test Execution#

Test Execution#

A Test Execution is a single “run” of Fuze™ Test via the command line (or execution stage in the EmbedScale pipeline). Fuze™ Test orchestrates (i.e., matches test content with correct DUT) test content per execution. You configure which test suites to execute during its run. Therefore, a test execution sequence contains a collection of test suites.

More detailed information on Test Execution is contained in Test Execution.

Test Suites#

A Test Suite contains a collection of test cases and associated configuration and control.

Detailed Test Suite information can be found in Test Suite Specification.

Test Suite Design Considerations#

Fuze™ Test orchestrates and parallelizes test case execution within a test suite across the DUT farm when it can. Therefore, test case execution within a test suite is contextually related to the DUT farm configuration and test suite control.

If > 1 DUT has been configured to execute a particular set of tests, and the test suite configuration allows it, then you must assume each test case can execute on any of the configured DUTs. This type of parallelization results in time efficiency.

_images/fuze-test-parallelization.png

Parallelization of Test Cases within a Test Suite#

However, you may also configure the test suite to ensure that all test cases execute on all parallel DUT platforms for which they are targeted. This type of parallelization results in cross-device testing.

_images/fuze-test-suite-parallelization.png

Parallelization of Test Suite Across DUT Farm#

Fuze™ Test test execution operates on a single test suite at a time and orchestrates the content accordingly. Within a test suite, it applies parallelization according to its configuration. Therefore, you must design your groupings of test cases into test suites with this knowledge.

  • You cannot assume the order of execution when using a test suite that is configured and able to fanout across a DUT farm. The “testcases” section of a test suite contains the list of test cases that can be parallelized.

  • You may specifically configure some or all of your test cases to execute in order and all of the DUTs in the farm by using the “setup” and “teardown” sections.

  • You may group test cases into test suites that are logically related and adhere to the above knowledge

  • You may group test cases into test suites that require the same setup and teardown sequences.

Test Cases#

A test case is a single atomic unit of test. It is the fundamental unit of test for Fuze™ Test and can be assumed to execute from start to finish on a single DUT, regardless of the topology of the DUT farm. Test cases are comprised of a series of procedural steps with configuration and control options.

Detailed Test Case information can be found in Test Case Specification.

Test Case Design Considerations#

Fuze™ Test assumes test cases are atomic and idempotent. Therefore, a single test case should ensure:

  • State Preservation: The initial state before the test execution is the same as the state after the test execution.

  • Repeatability: The test can be repeated multiple times without any side effects or changes in outcome.

  • No Side Effects: The test execution does not alter the system state in a way that affects subsequent tests or operations.

  • No Inter-Case Dependencies: No test case should assume or require any results, states, or operations from another test case.

  • No Inter-DUT Depedencies: No test case should assume or require any results, signals, or operations from another DUT in the farm.

Note

If a DUT requires communication with another DUT for testing, then the DUT pair must be considered as a single DUT system.

DUT Power Cycle, Reset, and Initialization#

Configuration Management of Test Content#

Fuze™ Test is designed to pull test content from a location of your choosing. Therefore, you can keep your test suite and test case files in your repository. You can colocate your test content with your system’s codebase, or you can keep it sequestered in its own repository.

Contents