Test Criteria for Amazon Appstore Apps

When you submit your Amazon Fire device app or game to the Amazon Appstore, the app must pass a series of tests to qualify for publication. As you develop your app, use the test criteria on this page as a guide for your own quality assurance testing before submitting the app.

• Content policy
• General qualifications
  • Additional qualifications for Fire TV
  • Additional qualifications for Fire tablets
  • Amazon Fire TV remote and voice remote qualifications
  • Game and other controllers
• Test criteria group 1: Device capability tests
• Test criteria group 2: App behavior on Fire TV device
• Test criteria group 2: App behavior on Fire tablet device (Fire tablets only)
• Test criteria group 2: Key app performance indicators
  • Time to start tests
    • Cold start: time to display first frame
    • Warm start: time to display first frame
    • Cold start: time for ready to use
    • Warm start: time for ready to use
  • Memory tests
    • Foreground memory usage
    • Background memory usage post video streaming
• Test criteria group 3: Core app functionality
• Test criteria group 4: Amazon Fire TV remote behavior (Fire TV only)
• Test criteria group 5: Amazon Fire game controller behavior (Fire TV only)

Content policy

Each app that you submit to us must also adhere to the Amazon Appstore Content Policy. If Amazon determines that an app contains, facilitates, or promotes content that is prohibited by these guidelines, Amazon will reject the app submission and notify you using the email address associated with your developer account.

General qualifications

At a high level, your app should meet the following qualifications:

• Have a simple and well-thought-out user experience that does not cause user confusion.
• Be thoroughly tested and free of bugs, crashes, and obvious defects.
• Provide visual indication that a user's action has succeeded or failed.
• Fail gracefully if the user tries to access unsupported functionality. For example, the app could display an error message that reads "This feature is not currently available."
• Fully utilize the screen area of the Fire tablet or Fire TV screen.
• Be free of visual defects, such as missing/distorted/pixelated images, misaligned UI elements, or illegible text.
• Have UI text that is free of grammatical or spelling errors.

Additional qualifications for Fire TV

Apps designed for Fire TV should meet the following additional qualifications:

• Design your app to be displayed in 1080p resolution (1920x1080px) for the best possible experience.
• Have limited text entry requirements. Large amounts of text entry hinder the user experience.
• Users should be able to read text and UI elements from approximately 10 feet away.
• Be 2GB or less in size (recommended). This size includes any first launch downloads of additional assets or other content. Note that if your app is above or near 4GB in file size it may not install or run on the device.

Additional qualifications for Fire tablets

Apps designed for Fire tablets should meet the following additional qualifications:

• Your app's UI should take into consideration and utilize both the notification bar and dynamic soft key menu behavior.

Amazon Fire TV remote and voice remote qualifications

For your app to be compatible with the Fire TV remotes, it should have these characteristics:

• Feature a simple UI that is easily navigable with Up, Down, Left, Right movements (D-Pad).
• Not require motion events or rapid input for use.
• Be usable with only one hand. Do not require the use of two hands for sustained interactivity.
• If the user needs to hold the remote sideways to use your app, inform the user of that on launch so navigation is handled correctly.
• Avoid simultaneous button inputs, especially for core functionality.

Game and other controllers

For your app to be compatible with game controllers (both the Amazon Fire game controller and other Bluetooth game controllers), it should feature a simple UI that is easily navigable with D-Pad or analog stick input.

If your app requires the use of any other controller, that controller must support the Bluetooth HID gamepad profile and not require additional software for input.

Test criteria group 1: Device capability tests

This group of tests is designed to make sure that your app does not use capabilities that are not present on Amazon Fire tablet or Fire TV devices. If your app does request device permissions that are not present, it should handle the error condition in a user-friendly manner, such as by providing an error message. Your app must not force close or crash in the absence of these device capabilities.

For the best experience, your app should not use any of the following permissions.

• android.permission.SET_WALLPAPER
• android.permission.CALL_PRIVILEGED
• android.permission.CALL_PHONE
• android.permission.MODIFY_PHONE_STATE
• android.permission.READ_PHONE_STATE
• android.permission.READ_SMS
• android.permission.RECEIVE_SMS
• android.permission.SEND_SMS
• android.permission.WRITE_SMS
• android.permission.ACCESS_FINE_LOCATION
• android.permission.RECORD_AUDIO

Additionally, Fire TV apps should not use the following permission:

• android.permission.CAMERA

Test criteria group 2: App behavior on Fire TV device

Your app must interact correctly with Amazon Fire TV devices and the Fire TV platform. To complete this group of tests, install on a Fire TV device and then launch the app.

2.1 Install Size

• Test: Install and ensure that the app's full install size on disk is no greater than 4 GB.
• Expected Results: Apps must not exceed a 4 GB file size when fully installed on an Amazon Fire TV device. For apps which require a media or content package download upon first launch in order to access the app's entire core functionality, install size should be measured after this initial launch and download.

2.2 Installation Time

• Test: Ensure quick installation of app resources.
• Expected Results: Ideally, the app does not take more than 15 seconds to install. Make sure the unused resources are removed from the app. This reduces the app's size and helps in quick installation.

2.3 Quick Loading Times

• Test: Observe that loading screens taking longer than 15 seconds display a "loading" message.
• Expected Results: If loading takes longer than 15 seconds, the app must indicate this to the user (through loading screen, progress bar, etc).

2.4 Login Functionality

• Test: Launch the app and observe the functionality of all available login options provided in the app.
• Expected Result: A user should be able to log in to the app with all available login options. Some examples of login options are logging in using an activation code, logging in manually, and logging in remotely using a mobile phone. Ensure that the login and approval functions work and display as intended.

2.5 Overall Performance

• Test: Ensure that the overall performance of the app sustains a usable and pleasant user experience. See Key App Performance Indicators for more guidance.
• Expected Results: Apps should sustain a high frame rate without dropping below 25 fps for an extended period of time. Apps should not impede the user's ability to interact with that app. For example, users should not experience a noticeable delay when typing. The app should not force close, crash, or hard lock while in use. The recommended average frame rate is 55-60 fps, especially for apps that involve fast-moving graphic objects.

2.6 Overall Memory Use

• Test: Make sure each app version you release on the Appstore writes less than an average of 50MB/hr when app is on the foreground and 2MB/hr when app is on the background.
• Expected Results: Exceeding the eMMC write limit of 50MB/hr will lead to failures in devices using the affected apps over time. Before releasing an app update, verify that any video and/or audio played through the apps is not buffered in the eMMC, as this will likely result in the app writing more than 50MB/hr to the eMMC component. Video and/or audio buffering is usually done on components other than the eMMC.

2.7 Stability

• Test: Verify that the app remains stable and maintains full interactivity while running.
• Expected Results: Apps must not experience visual slowdown (FPS loss), force close, crash, hard lock, or throw error messages while in use. The application must remain stable, even when dealing with conflicting inputs.

2.8 Graphical Issues

• Test: Observe the graphical UI elements within the app and verify that no distorted, pixelated, misalignments, or other graphical anomalies occur.
• Expected Results: The app's graphics should not appear distorted, pixelated, misaligned, or contain any other graphical anomalies. Additionally, all apps featured on Amazon Fire TV devices must display automatically in landscape mode, and scale to fit the larger form factor of a TV/monitor running in a minimum of 480p resolution.

2.9 Screen Real Estate Coverage

• Test: Observe the amount of screen that the app occupies.
• Expected Results: Apps should occupy 90% of the screen to be fully compatible (taking into account overscan issues noted in the Design and User Experience Guidelines. But apps may still pass as long as they fill 80% of the screen and are centered horizontally and vertically on the screen; apps should not be located in one corner of the screen. Also, the unfilled portions of the screen must be free from graphical anomalies.

2.10 Text Entry and Display

• Test: Identify all areas of the app's core functionality which require text entry, as well as observing in-app text from approximately 10 feet away.
• Expected Results: The app must not require a significant amount of text entry as part of its core functionality. UI components (menus, buttons, images) must be large enough and spaced far enough apart to be read from approximately 10 feet away. In-app text must be clearly readable from this distance as well.

2.11 UI Highlighting

• Test: Ensure that UI elements appear highlighted when selected.
• Expected Results: Apps should feature a UI that accommodates button (rather than touch screen) input by indicating to users what the current focus of the UI is. This may be achieved through the default rectangular box, a movable cursor, or some other indicator of what the user is selecting.

2.12 Touch Screen UI

• Test: Ensure that no virtual joystick, pause icon, or other touch screen-related references are displayed in the app UI.
• Expected Results: Apps must degrade touch screen UI elements when controller inputs are detected, or remove all such touch screen references from the app entirely.

2.13 Navigability

• Test: Search for areas where users can become trapped, that is, unable to navigate using either onscreen elements or controller buttons.
• Expected Results: Users should be able to navigate the interface of the app without becoming locked into any one screen. In addition, users should not be forced to exit the app using the Home button.

2.14 External Linking

• Test: Observe whether the app links users to external content.
• Expected Results: External links should not break any functionality of the app. Apps may only link directly to video content within another app if the two apps are associated with one another. See also Deep Linking from within Your App.

2.15 Ultra High Definition (UHD), 4K Playback

• Test: If supported by your app, select and play any UHD or 4K content in the app.
• Expected Result:
  • Audio and video artifacts should not be displayed.
  • Audio and video should be in sync.
  • Resolution should be 3840 x 2160.

• Test Tips:

  The Developer Tools Menu displays information related to the color standard. The menu also provides options that give real-time metrics and other information about your app. Access the Developer Tools Menu in the following ways.

  • Remote: On the directional pad, press and hold the center button for one second. While still holding down center, press the down button too. Hold both buttons for three to four seconds. Then release both buttons and press the remote's menu button. If this doesn't work, try holding center and down simultaneously instead of staggered.
  • Amazon Game Controller: Press and hold A and Y buttons simultaneously for five seconds. Release both buttons and press the controller's menu button.
  • Android Debug Bridge: If ADB is enabled on the Fire TV and a computer is connected, you can run the following command to invoke the Developer Tools Menu.

      adb shell am start com.amazon.ssm/com.amazon.ssm.ControlPanel
    

    To use the ADB command, you must enable ADB debugging in the Fire TV settings, have Android Studio installed, and know your Fire TV's IP address. To find the IP address of your Fire TV, go to Settings > My Fire TV (or Device & Software) > About > Network. Open a terminal window on your computer and connect through ADB with this command: adb connect 123.456.7.89:5555. Replace 123.456.7.89 with your own IP address. Leave 5555 as the port.

    If you want to display multimedia information when Android MediaCodec APIs are in use, toggle on Advanced Options. If you play media with this option enabled, an additional display appears on the screen. In the display, the MUL section shows multimedia (audio and video) information, and the AUD section shows audio focus information.

  The following screenshots show the Developer Tools Menu and the Advanced Options display.

  

  

2.16 Dolby Vision, High Dynamic Range (HDR)

You should be able to play movies recorded in HDR or Dolby Vision format on devices that support Dolby Vision or other HDR formats. To see the display settings on a Fire TV, go to Settings > Display & Sounds > Display.

• Test: Select and play any Dolby Vision or HDR supported title in the app.
• Expected Result:
  • There should be a logo or tag that indicates that the content is available in the Dolby Vision or HDR format.
  • Audio and video artifacts should not be displayed.
  • Audio and video should be in sync.
  • Color Standard should be BT2020.
• Test Tips

2.17 Dolby Atmos

Your app should be able to detect device capabilities and therefore should stream Atmos-supported content in the Atmos audio format. If an audio track is configured with Dolby Atmos encoding format, and the connecting endpoint supports that encoding format, the content should play in Atmos audio format. If Dolby Atmos is not supported at the connecting endpoint, your app must fall back to non-Dolby format. For more guidelines on Dolby integration, see Dolby Integration Guidelines (Fire TV).

To see the Dolby settings on a Fire TV, go to Settings > Display & Sounds > Audio.

• Test: Connect an Atmos-supported Fire TV to an audio/video receiver (AVR) that supports Atmos, or use a TV that supports the Atmos audio format.

  To connect a Fire TV to an AVR, use the following steps:

  1. Connect the Fire TV stick to the AVR input port.
  2. Connect the AVR Audio Return Channel (ARC) port to the TV ARC port, using an HDMI cable.

  To connect a smart TV to an AVR, use the following steps:

  1. Connect the smart TV ARC port to the AVR ARC port.

  After connecting the Fire TV to the AVR, make sure the Fire TV displays video, and audio is routed from the AVR. Because the AVR is connected, audio shouldn't be routed through the TV speakers.

  Play any Atmos-supported content in the app.

• Expected Result:

  • The digital display of the AVR should show the text "Atmos".
  • Audio and video should be in sync during playback.
  • There should not be any frame drops, audio loss, or colored artifacts.

2.18 Audio Focus

All apps that play audio should correctly acquire and abandon audio focus and exactly match the duration of the playback. To avoid music apps playing at the same time, Android introduced the idea of audio focus, which means only one app can hold audio focus at a time. For more information on how to handle audio focus, see Managing Audio Focus (Fire TV).

• Test:
  1. Play any content in the app (app under test).
  2. Press the home button on the remote.
  3. Launch any other music app. For example, Amazon Music.
  4. Exit the music app by pressing the home button on the remote.
  5. Launch the app under test and resume playback.
  6. Press play/pause during playback.
• Expected Result:
  • There should be no audio overlap observed.
  • The app under test should remain in the foreground, and focus should not move back to the music app.

2.19 HDMI Disconnection

When an app receives an HDMI disconnection event, it must pause media playback. Apps should not continue playback without a screen. Playback of a 4K quality video can use a large amount of data, but without a screen the data use is unnecessary. Even audio-only apps should pause playback when an HDMI sink gets disconnected. The app is expected to stay in a paused state when the HDMI cable is reconnected to a display. For more information, see Requirement 7.1: HDMI Disconnection.

• Test:
  • Initiate playback in the app. During playback, switch to a different HDMI input of the TV. Wait for at least three minutes, then switch back to the Fire TV and launch the app.
  • Initiate playback in the app. During playback, turn off the TV by using the power button on the remote. Wait for at least three minutes, then turn on the Fire TV and launch the app.
• Expected Result:
  • Playback in the app should be paused when the HDMI inputs are changed.
  • Content should resume from the same position when Fire TV is reconnected.
  • No crash should be observed.

2.20 Unsupported Software

• Test: Test in-app functionality for dependencies on unsupported software.
• Expected Results: Apps fail this test if they are unusable due to a reliance on software or OS features the Amazon Fire TV platform does not support. The tester should note which limitation is observed here (themes, wallpapers, widgets, virtual keyboards, program launchers, app ratings, and so on).

2.21 Unsupported Hardware

• Test: Test in-app controls for use of unsupported hardware inputs.
• Expected Results: The app must not require a control mechanism unsupported by the device. This may include such features as:
• Gyroscope
• Accelerometer (or 'Tilt' control)
• Other external hardware
• Controllers that do not support the Bluetooth HID gamepad profile.

2.22 Exit Time

• Test: Ensure a smooth and quick exit occurs when the Home button is pressed.
• Expected Results: The app should not take more than 2 seconds to exit when the Home button is pressed.

2.23 Uninstallation

• Test: Uninstall the app from the device.
• Expected Results: The app should be uninstalled from the device.

2.24 Uninstallation Time

• Test: Ensure the app uninstalls quickly and removes all artifacts.
• Expected Results: Ideally, the app takes no more than 2 seconds to uninstall. The app should create minimal artifacts on the disk. This helps ensure faster clean-up while uninstalling the app.

Test criteria group 2: App behavior on Fire tablet device (Fire tablets only)

Your app must interact correctly with the Fire tablet and its operating environment. To complete this group of tests, install the app on a Fire tablet and launch.

2.1 Launch Without Error

• Test: Successfully launch the app from the app library.
• Expected Results: The app launches on the Fire tablet and will advance the user into the app's interface without any errors.

2.2 Quick Loading Time

• Test: Observe that loading screens taking longer than 15 seconds display a "loading" message.
• Expected Results: If loading takes longer than 15 seconds, the app must indicate this to the user (through loading screen, progress bar, etc).

2.3 Fullscreen Mode Usability

• Test: If the app uses fullscreen, observe that the soft keys toolbar is minimized (with the fullscreen handle visible) and that the Settings toolbar is hidden.
• Expected Results: When the app uses fullscreen, the soft keys toolbar should be minimized (with the fullscreen handle visible) and the Settings toolbar should be hidden.

2.4 Soft Keys and Settings Toolbar

• Test: If the app does not use fullscreen, observe that the soft keys toolbar and the Settings toolbar are present.
• Expected Results: When the app does not use fullscreen, the soft keys toolbar and the Settings toolbar should be present.

2.5 Soft Key Menu Button Functionality

• Test: Observe the soft key menu button expands for more soft keys toolbar options.
• Expected Results: Use of the Menu button is not required. But when in use, tapping the Menu button should not cause the app to force close.

2.6 Correct Back Button Behavior

• Test: Observe the behavior of the Back button when pressed within the app, reverting back to a previous screen.
• Expected Results: Use of the Back button is not required. But when in use, the Back button should not cause the app to force close.

2.7 Correct Home Button Behavior

• Test: Observe the behavior of the Home button when pressed within the app.
• Expected Results: Use of the Home button is required and should take the user back to the Home screen of the Fire tablet.

2.8 Correct Search Button Behavior

• Test: Observe the behavior of the Search button when pressed within the app.
• Expected Results: Use of the Search button is not required. But when in use, the Search button should not cause the app to force close and allow the user to search within the app.

2.9 App Performance with Invocation of the Settings Toolbar

• Test: Observe the behavior of the Settings toolbar when invoked, and observe the app's performance when changing the settings (for example, volume).
• Expected Results: Users should have the ability to change Fire tablet settings while running the app. Use of the Settings menu should not cause the user to lose state, cause a force close, or impede the user's experience in any way.

2.10 Soft Key Toolbar Interference

• Test: Verify that the soft keys toolbar does not interfere with the app or cover important UI elements.
• Expected Results: The soft keys toolbar should not obstruct any elements of the app, for example, buttons, text, or important graphics.

2.11 Navigability

• Test: Search for areas where users can become trapped, that is, unable to navigate using either onscreen elements or the soft keys buttons.
• Expected Results: Users should be able to navigate the interface of the app without becoming locked into any one screen. In addition, users should not be forced to exit the app using the Home soft key.

2.12 State Retention After Hibernate Mode

• Test: Invoke hibernation mode of the device by pressing the Power button.
• Expected Results: When the device is in hibernation mode, the app should return to its previous state when it comes out of hibernation mode, saving all user data and progress. To save their state, most apps should pause; some apps, however, such as alarms, radios, or music players, should keep running when the device is hibernating. Also, the app may pass if the loss of user state is insignificant.

2.13 Network Connection Interrupted During Download

• Test: Observe the behavior of the app when the network connection has been lost while a download is in progress.
• Expected Results: Wake lock should be active for large downloads (large downloads >= 5 minutes). If Wake lock has not been implemented and the device has gone into hibernation mode, when the user returns to the app they must be informed via a message that the download has failed.

2.14 Overall Performance

• Test: Ensure that the overall performance of the app sustains a usable and pleasant user experience. See Key App Performance Indicators for more guidance.
• Expected Results: Apps should sustain a high frame rate without dropping below 25 fps for an extended period of time. Apps should not impede the user's ability to interact with an app. For example, users should not experience a noticeable delay when typing in an app. The app should not force close, crash, or hard lock while in use. The recommended average frame rate is 55-60 fps, especially for apps that involve fast-moving graphic objects.

2.15 Force Closes

• Test: Observe any reproducible or non-reproducible force closes.
• Expected Results: The user must be able to interact with the app's UI and features without any force closes.

2.16 Hard Locks

• Test: Observe any reproducible or non-reproducible hard locks.
• Expected Results: The user must be able to interact with the app's UI and features without any hard locks.

2.17 Errors from User Abuse

• Test: Repeatedly press buttons or other graphical elements on the screen.
• Expected Results: The app should not force close when users rapidly press a button.

2.18 Graphical Issues

• Test: Observe the graphical UI elements within the app and verify that no distorted, pixelated, misalignments, or other graphical anomalies occur.
• Expected Results: The app's graphics should not appear distorted, pixelated, misaligned, or contain any other graphical anomalies. The user experience should not suffer because of poor-quality graphics.

2.19 UI Displacement

• Test: Observe key presses for UI displacement.
• Expected Results: Verify that the location of UI graphics correspond accurately to where their interactive areas exist on the touchscreen. For example, in some cases, users may have to press 20px above where a button is displayed—this would be a fail.

2.20 External Links

• Test: Observe whether the app links users to external content.
• Expected Results: External links within the app should not break any functionality of the app. After returning to the app from an external link, the app should have saved the user state and progress. Links may appear to fail because they direct users to apps that are not yet live. This behavior is expected, but the links should work once the apps are live. Apps may only link directly to video content within another app if those apps are associated with one another. See also Deep Linking from within Your App.

2.21 Screen Real Estate Coverage

• Test: Observe the amount of screen that the app occupies.
• Expected Results: Apps should occupy 100% of the screen to be fully compatible. But apps may still pass as long as they fill 80% of the screen and are centered horizontally and vertically on the screen; apps should not be located in one corner of the screen. Also, the unfilled portions of the screen must be free from graphical anomalies.

2.22 Volume Controls

• Test: Observe the behavior of in-app volume and mute controls.
• Expected Results: Any volume and mute functions built into an app should function as expected. If muted, then the sound should no longer play.

2.23 Text Placement

• Test: Observe the placement and text within user input data fields.
• Expected Results: Input boxes should fully display the text that the user has typed without being cropped or unreadable.

2.24 Orientation Change

• Test: Observe the behavior of the app when the device is rotated between landscape and portrait modes. Repeat as needed throughout the app.
• Expected Results: When the screen orientation is rotated, the graphics onscreen should adjust properly to fill the new layout. Text, graphics, and buttons should appear without error while being fully functional from one layout to another. Note that not all apps allow the screen to be rotated. Also note that pictures and videos can be the exception to the rule due to their fixed aspect ratios.

2.25 Social Networking Login Functionality

• Test: Observe the functionality of any social networking login pages.
• Expected Results: Users should be able to interact with social networking login pages, allowing users to log into their account. Ensure that the login and approval functions work and display as intended.

2.26 SMS/MMS Graceful Failure

• Test: Observe the behavior of buttons within the app that submit or attempt to share user data.
• Expected Results: Apps that use SMS or MMS to share user data should not force close. These apps should be able to gracefully fail with a relevant error message.

2.27 Social Networking Sharing

• Test: Observe the social networking functions and their interactions with the app.
• Expected Results: If the app features any social networking integration (Facebook, Twitter, etc.), verify that content is able to be uploaded/shared across these mediums as expected. Account Creation: If an external service provided by the app requires the user to create an account, verify that an account may be created and used with the app.

2.28 Amazon Appstore Links Function Correctly

• Test: Observe external links that link to the Amazon Appstore.
• Expected Results: All links that point to other apps should link correctly to the Amazon Appstore. If a link points directly to an app on the Amazon Appstore and is not currently live on the store, then it is acceptable for that link to fail.

2.29 Uninstallation

• Test: Uninstall the app from the device.
• Expected Results: The app should be uninstalled from the device.

2.30 Exit Time

• Test: Exit the app.
• Expected Results: The app should not take more than 2 seconds to exit when the Home button is pressed.

2.31 Installation Time

• Test: Ensure quick installation of app resources.
• Expected Results: Ideally, the app will not take more than 15 seconds to install. Make sure the unused resources are removed from the app. This will reduce the app's size and help in quick installation.

2.32 Uninstallation Time

• Test: Ensure the app uninstalls quickly and removes all artifacts.
• Expected Results: Ideally, the app will take no more than 2 seconds to uninstall. The app should create minimal artifacts on the disk. This will help ensure faster clean-up while uninstalling the app.

2.33 Orientation Lock

• Test: Ensure that the app's orientation is correct following device orientation setting.
• Expected Results: If device orientation is locked in device settings, app orientation should not change upon rotating the device. The app should open in its default orientation or as per the locked orientation in the device. The orientation should not change automatically on rotating the device if it is locked in the device.

2.34 Battery Usage

• Test: Ensure that the app is not consuming battery excessively.
• Expected Results: Battery-intensive operations should be optimized to ensure that power consumption remains low.

2.35 Temperature

• Test: Ensure that the device temperature does not increase to the point of discomforting the user, even upon prolonged use.
• Expected Results: The app should not heat up the device such that it results in discomforting the user when it is held in both hands. If continuous usage of the app causes the device temperature to reach 40 degree Celsius from normal ambient temperature (33 degrees Celsius) during a period of 1 hour, then it would be considered a fail. Make sure the intensive operations are optimized to ensure that the temperature remains low.

2.36 Gestures

• Test: All possible gestures should be handled as per their standard behavior. Expected Results: Gestures like pinch (in/out), swipe (left/right/up/down), and press and hold, should work as expected.

2.37 State Retention After USB Plug-In / Plug-Out

• Test: Ensure the game preserves its state after USB plug-in / plug-out.
• Expected Results: When the device is in USB mode, the app should return to its previous state when it comes out of USB mode, saving all user data and progress.

2.38 Errors/Exceptions in the Background

• Test: Ensure there are no exceptions thrown from the app.
• Expected Results: We recommend you analyze the device logs to see if the app is throwing any errors or exceptions. Ideally, the app should not throw any errors or exceptions in the background.

Test criteria group 2: Key app performance indicators

Time to start tests

Cold start: time to display first frame

This is the time an app takes to launch until the first frame when it is launching for the first time after the app process is killed or after a device reboot. Fire TV and Fire tablet apps should display the first frame within 2 seconds of cold start launch.

Apps under test (AUT) – Pre-conditions

1. Login to the device using a test account.
2. Download the app under test.
3. Device is in steady state.
4. Ensure that there are no apps in the background.

Test steps

1. Ensure that the AUT is not running in the background. If so, kill the app process using the adb command.
2. Launch the app using the adb command adb shell am start AppPackage/AppActivity.
3. Calculate the time taken to draw the first frame of the app.
4. Force close the app.
5. Repeat step 1-4 for 50 iterations.

• Mac or Linux
• Windows

adb logcat | grep "Displayed APP_PACKAGE_NAME/LAUNCHER_ACTIVITY"

adb logcat | findstr "Displayed APP_PACKAGE_NAME\LAUNCHER_ACTIVITY"

Sample output: ActivityManager: Displayed APP_PACKAGE_NAME/LAUNCHER_ACTIVITY +930ms (total +849ms)

Warm start: time to display first frame

This is the time an app takes to launch until the first frame is displayed when the app process is already running in the background. The system brings the app from the background to the foreground as part of this launch activity. All apps on Fire TV and Fire tablets should display the first frame within 1 seconds a warm start.

App under test (AUT) – Pre-conditions

1. Login to device using a test account.
2. Download the app under test.
3. Device is in steady state.
4. Ensure that there are no other apps in background.

Test steps

1. Launch the AUT and press the Home button. Ensure AUT is in the background.
2. Launch the app by adb command adb shell am start AppPackage/AppActivity.
3. Calculate the time taken to draw the first frame of the app.
4. Press the Home button.
5. Repeat steps 2-4 for 50 iterations.

• Mac or Linux
• Windows

adb logcat | grep "Displayed APP_PACKAGE_NAME/LAUNCHER_ACTIVITY"

adb logcat | findstr "Displayed APP_PACKAGE_NAME\LAUNCHER_ACTIVITY"

Sample output: ActivityManager: Displayed APP_PACKAGE_NAME/LAUNCHER_ACTIVITY +930ms (total +849ms)

Cold start: time for ready to use

This is the time an app takes to launch until it is fully drawn and ready for user interaction when it is launching for the first time after the app process is killed or after a device reboot. Fire TV apps should be ready to use within 10 seconds of cold start launch. For Fire tablet apps, this threshold is 15 seconds for gaming apps, and 10 seconds for non-gaming apps.

Apps under test (AUT) - Pre-conditions

1. Login to device using test account.
2. Download the app under test.
3. Device is in steady state.
4. Ensure that there are no other apps in background.

Test steps

1. Ensure that the AUT is not running in the background. If so, kill the app process using adb command.
2. Launch the app. If your app does not have reportFullyDrawn implementation then start a manual timer same time as you launch the app. For more information see the reportFullyDrawn documentation.
3. Calculate the time taken for the app to be fully loaded – this is the app state where customers can start interacting with the app.
4. Force close the app.
5. Repeat step 1-4 for 10 iterations.

• Mac or Linux
• Windows

adb logcat | grep "Fully drawn APP_PACKAGE_NAME/LAUNCHER_ACTIVITY"

adb logcat | findstr "Fully drawn APP_PACKAGE_NAME\LAUNCHER_ACTIVITY"

Sample output: ActivityManager: Fully drawn APP_PACKAGE_NAME/LAUNCHER_ACTIVITY +930ms (total +849ms)

If app has not implemented reportFullyDrawn method then you may measure the Ready to Use time manually (may not be accurate).

Warm start: time for ready to use

This is the time an app takes to launch until fully drawn and ready for user interaction when the app process is already running in the background. The system brings the app from background to the foreground as part of this launch activity. Fire TV apps and Fire tablet apps should be ready to use within 2 seconds of a warm launch. For apps that require a user to login, this time should be measured for a post login use case.

Apps under test (AUT) – Pre-conditions

1. Login to device using test account.
2. Download the app under test.
3. Device is in steady state.
4. Ensure that there are no apps in background.

Test steps

1. Launch the AUT and press Home button. Ensure AUT is in the background.
2. Launch the application. If your app does not have reportFullyDrawn implementation then start a manual timer same time as you launch the app.
3. Calculate the time taken for the app to be fully loaded – this is the app screen where customers can interact with the app.
4. Press Home button.
5. Repeat step 2-4 for 10 iterations.

• Mac or Linux
• Windows

adb logcat | grep "Fully drawn APP_PACKAGE_NAME/LAUNCHER_ACTIVITY"

adb logcat | findstr "Fully drawn APP_PACKAGE_NAME\LAUNCHER_ACTIVITY"

Sample output: ActivityManager: Fully drawn APP_PACKAGE_NAME/LAUNCHER_ACTIVITY +930ms (total +849ms)

If app has not implemented reportFullyDrawn method then you may measure the Ready to Use time manually (may not be accurate)

Memory tests

Foreground memory usage

This is the memory that an app consumes while it is in foreground and when the video playback is in progress. Fire TV apps should restrict foreground memory (RAM) consumption to 400 MB for 4K video apps, and 300 MB for 1080P or lower resolution. Fire tablet gaming apps should limit foreground memory consumption to less than 1000 MB and non-gaming apps should restrict foreground memory consumption to less than 600 MB.

Apps under test (AUT) – Pre-conditions

1. Login to device using test account.
2. Download the app under test.
3. Device is in steady state.
4. Ensure that there are no apps in background.

Test steps

1. Ensure that the AUT is not running in the background. If so, kill the app process using adb command.
2. Launch the application by adb command adb shell am start AppPackage/AppActivity.
3. Open the app and play a video content for 15 min.
4. Calculate the Foreground Memory usage of application.
5. Press Home button.
6. Repeat step 2-4 for 10 iterations.

• Mac or Linux
• Windows

adb shell dumpsys meminfo | grep “APP PACKAGE”

adb shell dumpsys meminfo | findstr “APP PACKAGE”

Sample output: 81,773K: APP PACKAGE (pid 21917 / activities)

Background memory usage post video streaming

This is the memory an app consumes after it is put in the background immediately following a video playback session when it was in foreground. While there is no defined threshold for Background memory consumption, the amount of memory an app uses in background will be deciding factor for killing a background app when the system is running low on memory (RAM). App with highest background memory consumption would be the first to get killed when system needs more memory for its foreground and other priority tasks.

Apps under test (AUT) – Pre-conditions

1. Login to device using test account.
2. Download the app under test.
3. Device is in steady state.
4. Ensure that there are no apps in background.

Test steps

1. Ensure that the AUT is not running in the background. If so, kill the app process using adb command.
2. Launch the app by adb command adb shell am start AppPackage/AppActivity.
3. Open the app and play a video content for 15 min.
4. Press Home button to send the app to the background.
5. Wait for 60secs.
6. Calculate the Background Memory usage of app.
7. Repeat step 2-4 for 10 iterations.

• Mac or Linux
• Windows

adb shell dumpsys meminfo | grep “APP PACKAGE”

adb shell dumpsys meminfo | findstr “APP PACKAGE”

Sample output: 81,773K: APP PACKAGE (pid 21917 / activities)

For more information, see App Performance Scripts.

Test criteria group 3: Core app functionality

In general, your app should "work the way it is designed." This is a general statement, but core functionality tends to be overlooked while being one of the most important aspects of testing. The following tests relate to such things as games keeping scores correctly, following game rules, and so forth.

3.1 Core App Functionality

• Test: Observe the core functionality of the app.
• Expected Results: Determine the main purpose of the app and whether the app can be used for this purpose on an Amazon Fire tablet or Fire TV device (for example, a music app must be able to play music).

3.2 Missing Features

• Test: Observe any missing functions of the app that may be listed as features under the app metadata.
• Expected Results: The features mentioned in an app's description should be available as part of the app’s functionality.

3.3 Graceful Degradation

• Test: Observe any features used in the app that are supported on the Fire devices targeted by the app.
• Expected Results: Any features of the app that are not available on targeted Fire devices should degrade gracefully with a message rather than a force close.

Apps that do degrade gracefully should still provide value to the end user. If an app's core functions are unavailable and are the only value to the app, the app will fail overall. For example, if a photo-taking app does not allow the user to take photos, even though it notifies the user that the camera is unavailable, the app still fails.

If the main function of an app is still accessible to the user when other functions are not, then the app might still pass. For example, if a bus route app notifies the user that Google Maps is unavailable and the user still has the ability to view the bus routes and the bus time tables, the app could pass.

3.4 Email (Fire Tablets Only)

• Test: Observe the featured use of an external email app as it relates to the core functionality of the app when email has and has not yet been set up on the device.
• Expected Results: If one of the core functionalities of the app is to send emails with an external email client, then it should be tested without a user's email previously set up through the external email client. It has been observed that apps will force close when attempting to call an external email client. The expected result is that the app should not force close. The app can disable the email feature with or without a message notifying the user.

3.5 Software-Related Settings

• Test: Observe the app's behavior when changing the settings of software-related options (for example, game difficulty, filtering, and display settings).
• Expected Results: Tested settings should function as anticipated, changing the related function within the app.

3.6 Hardware-Related Settings (Fire Tablets Only)

• Test: Observe the app's behavior when changing the settings of hardware-related options (for example, volume, accelerometer, and landscape vs. portrait settings).
• Expected Results: Tested settings should function as anticipated, changing the related function within the app.

3.7 App Data Storage

• Test: Observe the app's ability to save user state or data after exiting the app.
• Expected Results: If the app has modes or features that save user data, this data must be stored and remain accessible after exiting and restarting the app.

3.8 Data Persistence After an Update

• Test: Install an old version of the app and save some user data. Install latest version of the app over the old version. Observe whether the data saved in the old version of the app is accessible in new version.
• Expected Results: The app should preserve user data, such as game data, content purchased through In-App Purchasing, saved settings, downloaded content, login credentials, searches, saved chats, bookmarks, and saved pages.

3.9 Accessing External Applications

• Test: Ensure that app dependencies are handled correctly.
• Expected Results: If your app relies on another app to perform certain tasks, such as sending a file, and there are no apps on the device to perform this action, the main app should not force close.

3.10 Layout (Fire Tablets Only)

• Test: Ensure the layout of your app looks appropriate for a Fire tablet screen.
• Expected Results: Your app should not look like the mobile phone version of an app that has been scaled up for a Fire tablet screen. The layout and UI elements such as buttons, text input box, fonts, images, and so forth, should be optimized for a Fire tablet. Also, touch targets should not be too small. The objects should be identified easily. The size should be large and spaced enough for appropriate thick finger touch.

3.11 Hardware Acceleration (Fire Tablets Only)

• Test: Ensure smooth 2D graphics rendering throughout the app.

• Expected Results: Hardware acceleration should be enabled in the manifest file. To enable the hardware-accelerated 2D graphics, you should edit the AndroidManifest.xml file and add the following attribute to the <application /> tag:

      android:hardwareAccelerated="true"
  

3.12 Third-Party Controller Support (Fire TV Only)

• Test: Ensure that any third-party controllers the app supports are compatible with a Bluetooth HID gamepad profile.
• Expected Results: Amazon Fire TV devices support the use of third-party controllers, as long as they adhere to the Bluetooth HID gamepad profile. Proprietary technologies requiring additional driver or launcher app installs are not supported.

3.13 Multiple Controller Inputs (Fire TV Only)

• Test: For apps that support local multiplayer, ensure that the app accepts inputs from multiple controller inputs simultaneously.
• Expected Results: The app must handle simultaneous button inputs from multiple controllers with no adverse effect on game play. If an app supports multiple controller types, it must be possible to engage in local multiplayer through any combination of supported controllers.

3.13 Audio Playback

• Test: Ensure that the app stops playing audio once the user has exited the app.
• Expected Results: Apps must stop playing audio once the user has exited the app. Music apps (when playing content) may continue to play in the background. Once a new source of audio begins playing (by launching another app, for example) the first app must relinquish audio focus.

3.14 Daydream and System Standby (Fire TV Only)

• Test: Ensure that the app stops playing audio once the device enters daydream (displays the screen saver) or enters standby mode.
• Expected Results: Apps should stop playing audio when the device enters daydream (screen saver) or standby mode. There are two exceptions to this rule:
• Video apps (when playing content) must remain on (FLAG_KEEP_SCREEN_ON) and should never enter daydream or standby mode.
• Music apps (when playing content) must continue to play through daydream and standby modes without losing audio focus.

3.15 Metadata

• Test: Ensure that any screenshots and icons submitted for the app are the correct size, do not contain graphical errors, and represent the appropriate version of your app. 
• Expected Results: App submissions must include a large icon (512 × 512 PNG with transparency) and between three and 10 screenshots (PNGs or JPGs). These images must not appear distorted, stretched, or pixelated when viewed in the UI. Note that if you do not provide Amazon Fire TV-specific screenshots for your app in the submission process, any phone or tablet screenshots you have provided are used to represent your app in the Fire TV Appstore. This can be a confusing experience for your customers. If you are targeting Fire TV for your app, provide the correct Fire TV screenshots, or use screenshots that can represent your app generically for all devices. For image guidelines, see Images and videos.

Test criteria group 4: Amazon Fire TV remote behavior (Fire TV only)

Amazon Fire TV uses the remote control for both navigation and user input. The Fire TV platform does not support user input with a mouse or keyboard, nor are touch events available.

Note: All the tests in this section, with the exception of 3.2 (Microphone Button), refer to both the Amazon Fire TV Remote and the Alexa Voice Remote.

4.1 Select Button

• Test: Press the Select button. Test situations where a UI element has been highlighted with the D-Pad, as well as when no UI elements are highlighted.
• Expected Results: Pressing the Select button must initiate a logical action on the highlighted user interface object (analogous to a touch screen 'tap' input on the highlighted area).

4.2 Microphone Button (Fire TV Voice Remote Only)

• Test: Press the Microphone button and perform a voice search.
• Expected Results: Verify that pressing the Microphone button does not cause stability or graphical issues as the result of the voice search activity. The app must implement onPause() to preserve state, followed by onResume() once voice search is dismissed. For music apps, the app should duck the audio (lower the volume) through the use of OnAudioFocusChangeListener.

4.3 D-Pad

• Test: Press the D-Pad in all directions (Left/Right/Up/Down).
• Expected Results: The D-Pad buttons must move the focus and highlight in the user interface in the given direction.

4.4 Home Button

• Test: Press the Home button.
• Expected Results:
• For apps categorized as games in the Amazon Appstore, the Home button displays a "Game Paused" dialog. This behavior must not cause stability or graphical issues within the app. The app must implement onPause() to preserve state when the Game Paused dialog is displayed.
• For all other apps, the Home button navigates the user to the "Home" section of the Fire TV launcher. The app must implement onPause() to preserve state in case of an unanticipated return to Home.

4.5 Back Button

• Test case: When pressing the Back button within the app, the UI should revert to the previous screen.
• Expected Results: Use of the Back button is not required. But when in use, the Back button should not cause the app to force close or enter into an infinite loop within the UI.

4.6 Media Buttons

• Test: Press the Fast Forward, Play/Pause, and Rewind buttons.
• Expected Results: In media playback, the media buttons behave as expected.  In non-media apps, the media buttons should not be used for some other function, as this may interfere with the system's ability to control media playing in the background.  Note that If your app uses a framework such as Unity, you may ignore this requirement , as the ability to pass key events through to the system is not supported in those frameworks.

4.7 All Remaining Buttons

• Test: Press all remaining buttons.
• Expected Results: There are no behavior requirements for these remaining buttons. However:
• Single button presses must not exit the user to the Home screen.
• Buttons may navigate the user backwards in the UI.
• If the user is in the app's main menu (or an equivalent starting point for UI navigation), your app may use a button press to exit.

4.8 Core Functionality

• Test: Observe and test the core functionality of the application.
• Expected Results: After identifying the most common use cases/scenarios for app use, perform a detailed run-through (10–15 minutes) for functionality testing. Determine whether the app is fully compatible with Amazon Fire TV devices when navigating its functionality using this particular controller.** Note:** Certain apps may require simultaneous button inputs (D-Pad input paired with the Select button, for example) to navigate and use the app's core functionality. Because the Amazon Fire TV Remote and Voice Remote are optimized for use with only one hand, this functionality test must determine whether or not the one-handed UX is acceptable for sustained interaction with the app.

Test criteria group 5: Amazon Fire game controller behavior (Fire TV only)

All apps for which game controller compatibility has been declared are evaluated using the Amazon Fire Game Controller as a reference device. Only game controllers utilizing the Bluetooth HID gamepad profile are supported.

5.2 D-Pad

• Test: Move the D-Pad in all directions (left/right/up/down).
• Expected Results: Moving the D-Pad must move the focus in the user interface in the given direction.

5.3 A Button

• Test: Press the A button.
• Expected Results: Use the A button to confirm menu options or accept UI prompt.
• For games, during game play, pressing the A action button should perform the primary action of the game control layout.
• In media apps, the A button should also play or pause the audio/video playback.

5.4 B Button

• Test: Press the B button.
• Expected Results: It must be possible to navigate backwards out of menu options and decline UI prompts by pressing the B action button. If the app is a game, during game play, pressing the B action button should perform the secondary action of the game control layout.

5.5 Home Button

• Test: Press the Home button.
• Expected Results:
• For apps categorized as games in the Amazon Appstore, the Home button displays the "Game Paused" dialog. This behavior must not cause stability or graphical issues within the app. The app must implement onPause() to preserve state when Game Paused dialog is displayed.
• For all other apps, the Home button navigates the user to the "Home" section of the Fire TV launcher. The app must implement onPause() to preserve state in case of an unanticipated return to the launcher.

5.6 Media Buttons

• Test: Press the Fast Forward, Play/Pause, and Rewind buttons.
• Expected Results: In media playback, the media buttons behave as expected.  In non-media apps, the media buttons should not be used for some other function, as this may interfere with the system's ability to control media playing in the background.  Note that If your app uses a framework such as Unity, you may ignore this requirement , as the ability to pass key events through to the system is not supported in those frameworks.

5.7 Remaining Buttons

• Test: Press all remaining buttons and move the Left and Right Analog Sticks (left/right/up/down).
• Expected Results: There are no behavior requirements for these remaining buttons or analog sticks. However:
• Single button presses must not exit the user to the Home screen.
• Buttons may navigate the user backwards in the UI.
• If the user is in the app's main menu (or an equivalent starting point for UI navigation), your app may use a button press to exit.

5.8 Core Functionality

• Test: Observe and test the core functionality of the application.
• Expected Results: After identifying the most common use cases/scenarios for app use, perform a detailed run-through (10–15 minutes) for functionality testing. Determine whether the app is fully compatible with Amazon Fire TV when navigating its functionality using this particular controller.

Last updated: Nov 05, 2024