Developer Console
Sign In
Sign In
Developer Console

Test Summary and Results

On the Tests Summary tab, you can monitor and manage the tests that you submitted. Data contained in this tab can help you with analysis and troubleshooting.

Test runs

After you select Tests Summary, you will see a table containing details about each test that you've submitted. The following table provides a description of the details.

Test summary fields
Field Description
Test Name Name of the specific test that was run. You chose the name when you created the test run.
Test Type Performance/Functionality
Test Run Status The current status of the test run. Test run status categories are:
  • In Progress: Tests that are currently running or are queued to run.
  • Complete: Tests that have successfully finished running on the specified devices.
  • Error: Tests that encountered issues or failures during the test run.
Device Family Devices the test was run on. Device categories are:
  • Fire TV
  • Fire Tablet
OS Indicates the OS the test was run on.
Device Name of the device model.
Submission Date Date and time when the test was started.

To see detailed results for a test run, select the test name.

Device result summary

When you select a test name, you can view the device result summary. From this screen you can gather more detailed insights about your test run. The results list the names of the devices the test was run on, and indicates whether that device is compatible or non-compatible. Select the name of the device to open additional details about the test run.

The following table describes the fields shown in the device result summary.

Device result summary fields
Field Description
Test name Name of the specific test that was run. You chose the name when you created the test run.
App name Name of the app that you selected from the drop-down or name of the APK that you uploaded.
Test type Performance/Functionality
Test status The current status of the test run. Test status categories are:
  • In Progress: Tests that are currently running or are queued to run.
  • Complete: Tests that have successfully finished running on the specified devices.
  • Error: Tests that encountered issues or failures during the test run.
Device family Devices the test was run on. Device categories are:
  • Fire TV
  • Fire Tablet
OS Indicates the OS version the test was run on.
Device Name of the device model.
Submission date Date and time when the test was started.
Duration Time taken to complete the test.
Test result Indicates the overall outcome of the functional testing performed on the device under evaluation. Test result categories are:
  • Compatible: The device has successfully passed all the critical test cases.
  • Non-compatible: The device has failed the critical test cases and therefore hasn't met the necessary criteria for compatibility.
  • Error: The device encountered issues while running the test cases, preventing the completion of the evaluation process.
Overall result Determined by the number of devices that have successfully passed or failed the critical test cases.

Functional tests

With the functionality test results, you can analyze test outcomes in detail and identify potential issues, which can help you optimize your app across different devices and operating systems.

Functional test results provide the following details:

  • Test case status showing pass or fail for App Install, App Launch, App Login, and App Exit. If a device passes all critical test cases, it's marked as compatible. Otherwise, it's marked as non-compatible.
  • Names of the devices on which the test was run and a status for each. For details on test status, see Test result status
  • Screenshots
  • Device logs

Performance tests

Performance tests generate data on key performance indicators (KPIs), which you can use to analyze potential bottlenecks (factors that limit performance) for your app. You can use the performance test results to help you optimize resource usage across various devices and operating systems.

Performance test results provide the following details:

  • Summary of the test results for each KPI. This includes graphs that can help you identify performance trends and anomalies.
  • Data on individual iterations or test runs to help you investigate specific instances of performance deviations or issues.
  • Average response time and TP50 value of each KPI based on the number of iterations or test runs performed.
  • Names of the devices on which the test was run and a status for each. For details on test status, see Test result status

Test result status

The following table describes the possible status a for a test run on a device.

Status Description
Pass The test run completed successfully on the device without encountering any issues.
Fail During the test run, one or more test steps or test cases failed to meet the expected criteria or requirements.

Last updated: Dec 16, 2024