Use pre-launch reports to identify issues

When you upload an APK or app bundle to the open, closed, or internal test track, you can identify issues for a wide range of devices running different versions of Android.

The pre-launch report on your Play Console helps you identify:

  • Crashes
  • Performance issues
  • Accessibility issues
  • Security vulnerabilities

How the pre-launch report works

After you upload and publish a test APK or app bundle, test devices will automatically launch and crawl your app for several minutes. The crawl will perform basic actions every few seconds on your app, such as typing, tapping, and swiping.

After tests are complete, your results will be available in the Pre-launch report section of your Play Console.

Notes:

  • Test devices are selected based on a wide range of criteria, including popularity, crash frequency, screen resolutions, manufacturers, operating systems, and more. The selection of test devices may vary.
  • The pre-launch report is powered by analysis from Firebase Test Lab for Android.

Collapse All Expand All

Run a pre-launch report

Test your app

To generate your app's pre-launch report, publish an app on the open, closed, or internal test track.

All future versions that are published on a test track will automatically be tested for device compatibility, display issues, performance issues, and security vulnerabilities.

Notes:

  • You'll usually receive test results shortly after uploading your APK or app bundle. In some cases, you'll receive results several hours after upload.
  • You can test APKs or app bundles for multiple apps at the same time.
  • Currently, the pre-launch report devices can't run scripted tests on your app(s). To run scripted tests, follow the instructions under "Create a custom test with Firebase Test Lab for Android."
Test your game

You can use demo loops to test a series of predefined actions in your game. Demo loops make the pre-launch report relevant for testing game play, since the standard crawler doesn't support OpenGL.

To get started: 

  1. Create a demo loop in your game. For details, go to the Firebase Help Center.
    • Note: You don't need a Firebase account to use demo loops in the pre-launch report.
  2. To use demo loops in the pre-launch report, your game must be modified to do the following:
    • Launch the demo loop
    • Run the demo loop
    • Close the demo loop (optional)
  3. Publish the version of your game with the demo loop to the open, closed, or internal test track. The crawler will automatically detect and execute the demo loop in your game.
Optional: Create a custom test with Firebase Test Lab for Android

With Firebase Test Lab, you can pay to select your device type and testing method to create custom tests. Custom test results will be available in your Firebase console.

To run a custom test:

  1. Sign in to your Play Console.
  2. Select an app.
  3. Select Release managementPre-launch report.
  4. At the top of your screen, you'll see "Run Custom Tests" if you're able to run a custom test. To begin, select Get started.

View test results

To review your results:

  1. Sign in to your Play Console.
  2. Select an app.
  3. Select Release managementPre-launch report.
  4. Review the Crashes, Performance, AccessibilityScreenshots, and Security tabs. On each page, you'll see the results for your latest test. To view the results for previous builds, use the "Select" drop-down.
    • Note: If you see "Testing in progress," your latest test hasn't finished yet. If your latest test couldn't run, you may see "Test failed." To run another test, publish another APK or app bundle.

Results

Please keep in mind, Google can’t guarantee that tests will identify all issues.

Overview
Each test summary includes the number of errors, warnings, and minor issues found during testing, separated by issue type. You’ll also see a recommendation based on your app’s testing results.
Here are some examples of the issues that are found during testing:
  • Errors: Crashes
  • Warnings: ANRs, slow startup and load times, sign in or crawl issues, memory issues
  • Minor issues: Missing content labels, color contrast issues, small touch target sizes, implementation issues

View reports by category

Near the bottom half of your app’s Overview tab, you can view individual cards that summarize issues that are related to crashes, performance, security and privacy, and accessibility. Depending on the test results, you may also see insights and recommendations that can help you improve your app.

On each card, you’ll see a description that includes the number and type(s) of issues identified during testing.

View individual issues

On each card, you can click View all issues or the down arrow on the bottom right of the card to view a brief description of each issue. To get more details about a specific issue, click View issue.

Compare to other builds

The chart near the bottom of your app’s Overview tab shows the testing summary compared to previous builds. For each APK or app bundle, you can see the number of errors, warnings, minor issues, and tests completed, along with a timestamp of when the report was generated.

Get tips on using pre-launch reports

At bottom of your app’s Overview tab, you can find tutorials and tips on how you can use pre-launch reports more effectively.

Crashes

Each test summary includes the number of:

  • Devices with issues
  • Devices without issues
  • Devices tested

Compare to other builds

The chart on your app's Crashes tab shows your app's compatibility compared to previous builds.

Each bar represents a recent tested build of your APK or app bundle, along with the number of successful and failed launches found during testing.

View individual reports

Near the bottom half of your app's Crashes tab, you can view individual reports from devices running different versions of Android.

On each line, you'll see the name of the testing device, the device's Android version, and an icon showing whether your app had any testing issues.

  • If a device had issues during testing, you'll see a red icon.
  • If there weren't any issues during testing, you'll see a green check mark.

Select a test to see more details, like the device's name, operating system, test duration, a screenshot and video from testing, demo loop output, and stack traces (if available).

Performance

For each device model, the test summary includes the following metrics:

  • Average frames per second: Average rate at which frames are displayed.
    • Note: Average frames per second data is only available for tests using demo loops.
  • Average CPU: Percentage of average CPU usage by your app on the specific device model.
  • Average network sent: Average number of bytes per second sent by your app over a network connection on the specific device model.
  • Average network received: Average number of bytes per second received by your app over a network connection on the specific device model.
  • Average memory: Average memory used within your app over the selected time period on the specific device model.

Note: Test devices running some older versions of Android may not be able to generate performance data.

View individual reports

Select each device model to see a graph and recording of each metric plotted over time throughout the test.

For example, you can view your app's CPU percentage as the test progresses. If you notice a CPU spike, review the action the crawler took at that time to help resolve the problem.

Accessibility

Each test summary includes the number of accessibility errors, warnings, and minor issues found during testing, separated into the following categories:

View reports by category

Near the bottom half of your app’s Accessibility tab, you can see categorized sections with screen clusters that identify where accessibility issues were found in your app.

  • If any errors were identified, you’ll see a red icon.
  • If any warnings were identified, you’ll see an orange icon.
  • If only minor issues were identified, you’ll see a blue icon.
  • If no issues were identified, you’ll see a green check mark.

View individual issues

You can click a screen cluster to see example screenshots with corresponding device model names, operating systems, screen sizes, screen densities, and languages, along with a recommendation.

Improve your app's accessibility

Screenshots

Test summary

  • Images that show how your app appears across test devices in different languages
  • Metadata about the tested devices (including the model name, Android version, language, screen resolution, & DPI)
  • The number of devices with screenshots
  • The number of devices that couldn't be tested with your APK or app bundle
    • Devices unavailable: Your APK or app bundle is compatible with these devices but there was an issue during testing. If your test had any unavailable devices, you might want to upload another APK or app bundle and test again.
    • Devices incompatible: Your APK or app bundle is incompatible with certain test devices. If you test again, you won't receive results for any incompatible devices.

Note: If you're running a test using demo loops, you won't see any data on the Screenshots tab.

Ways to view screenshots

  • Screen clusters: Select the Screen clusters tab to see how similar screenshots display across different devices. With this view, the pre-launch report groups images together based on the elements or widgets on a screen.
  • Devices: Select the Devices tab to see all the screenshots associated with a specific device. With this view, you can see screenshots taken in chronological order during a test.

Language preferences

To view screenshots from specific languages, you can set up language preferences in the Settings tab. 

Security

Each test summary includes the name and description of any security vulnerabilities found in your APK or app bundle.

Note: We recommend you take action on any security vulnerabilities that are listed before publishing your APK or app bundle to production.

Manage pre-launch report settings

Pre-launch report versions

As of October 2017, the pre-launch report uses a new crawler to provide a more comprehensive review of your app. To do so, the pre-launch report must temporarily re-sign your APK or app bundle within our test environment.

Potential limitations of the new testing process

As a result of the re-signing process, you might notice testing limitations on some apps. Limitations may include:

  • Failure to autologin to Google Accounts
  • Failure to correctly interact with web service RPCs that use the app signing key as an authentication mechanism
  • Fewer crashes being discovered

Note: Most apps won't be affected by these limitations and will see more extensive crawls that find more issues than previous pre-launch reports.

Switch to the previous testing process

If you notice any issues, you can switch back to the previous testing process (which doesn't require re-signing): 

  1. Sign in to your Play Console.
  2. Select an app.
  3. Select Release management > Pre-launch report > Settings.
  4. In the “Pre-launch report version” section, move the Opt-out switch to the right until it turns blue.
Turn off all pre-launch reports

Pre-launch reports are automatically generated when you publish an app to the open, closed, or internal test track. To disable all pre-launch reports for your app:

  1. Sign in to your Play Console.
  2. Select an app.
  3. Select Release management > Pre-launch report > Settings.
  4. Next to "Enable pre-launch reports," move the switch to the left until it turns gray. If you want to turn pre-launch reports on again at a later time, move the switch back to the right until it turns blue.

Additional testing features

If your app has a sign-in screen with required credentials, you can add credentials to the Play Console that are associated with a test account.

Important: The credentials you provide are for testing purposes. Keep in mind, even if these credentials are provided, Google can't guarantee that tests will identify all issues.

How are credentials used?

Any credentials you add as part of pre-launch report testing will be saved and used for all future tests (until edited). If you edit your app's test credentials, we'll use the updated credentials on any new tests. If you delete your app's test credentials, you'll need to add new credentials before any future tests can sign in to your app.

While we do our best to keep test credentials secure, we recommend you don't add any official credentials into the pre-launch report. Instead, create a test account username and password.

Note: Credentials can only be automatically inserted into Android apps that use standard Android widgets. Credentials can't be used in apps that use OpenGL to render custom controls or apps that use a WebView for a web-based authentication flow.

Set up credentials

  1. Sign in to your Play Console.
  2. Select an app.
  3. On the left menu, select Release managementPre-launch report > Settings.
  4. Enter the following:
    • Username Resource Name: The Android resource name of the text field within your app where the given username should be entered (for example, if your text field is @+id/username_field, enter username_field).
    • Username: Username associated with your test account.
    • Password Resource Name: The Android resource name of the text field within your app where the given password should be entered.
    • Password: Password associated with your test account.
    • Sign-In Button Resource Name: The Android resource name of any button in your app that should be tapped after the username and password are entered. This could include a button to activate the sign-in process. You can omit this field.
  5. Save your changes.

Edit or delete credentials

  1. Sign in to your Play Console.
  2. Select an app.
  3. On the left menu, select Release managementPre-launch report > Settings.
  4. On the screen that appears:
    • To edit credentials, type your updated credentials.
    • To delete credentials, select Clear all fields.
  5. Save your changes.
View test results for specific languages

If you'd like the pre-launch report to only show test results from specific languages, you can set up language preferences in the Settings tab. You can select up to five languages. 

Tip: Since the pre-launch report runs automatically when you upload a test APK or app bundle, you can only add language preferences after the initial test completes.  

  1. Sign in to your Play Console.
  2. Select an app.
  3. On the left menu, select Release management > Pre-launch report > Settings.
  4. Under "Language preferences," select Add language .
  5. Select up to 5 languages. For future tests, you'll only see test results from these languages.
    • To remove a language, select the cancel icon .
Control how the pre-launch report explores your app

If you want more control over your tests, you can set up Robo scripts to perform specific actions like testing a common user journey or a new section of your app. In standard tests, the crawler may or may not perform these same actions.

When you run a test with a script attached, the crawler runs your pre-scripted actions first and then explores the app as usual.

To get started:

  1. To record your script, follow the instructions in the Firebase Help Center. Robo script recordings don't capture your actions outside the tested app.
    • Note: You don't need a Firebase account to run Robo script.
  2. After your script is ready, sign in to your Play Console.
  3. Select an app.
  4. Select Release management > Pre-launch report.
  5. Select the Settings tab.
  6. In the "Robo script" section, upload your script.

Collapse All Expand All

Testing tips & support

Geolocation of test devices

Test devices are located in the United States. If your app uses geolocation or has content restrictions based on country, test devices can only show what's available in their location.

If your app requires a particular geolocation outside of where test devices are located, you can publish an APK or app bundle that removes location requirements for testing purposes.

Subscriptions or in-app products

Test devices aren't able to make purchases during testing. If your app offers subscriptions or in-app products to access some parts of your app, testing scenarios may be limited.

Apps without a main launch activity (widgets, keyboards, etc.)

Test devices aren't able to run tests on apps without a main launch activity, including launchers, widgets, keyboards, and watch faces.

Apps with display ads

For details on how to mitigate fraudulent ad revenue with automated app testing, review the Google Developers site.

Apps compiled for x86 devices

Our device set is limited to ARM devices. An x86 APK will be shown as incompatible with all test devices.

Apps for Wear OS by Google, Auto, TV, and older Android versions

Our device set is limited to phones and tablets, so it's not possible to test apps directly on Wear OS by Google, Auto, or TV. Our device set also doesn't include devices with Android versions older than 4.1.1 (Jelly Bean).

Apps based on OpenGL/Unity3D frameworks

Our automated crawler can't identify UI elements on apps using OpenGL/Unity3D. The test results would be limited to a test launching the app and closing it after a few seconds.

To test a series of predefined actions in your game, you can use demo loops.

Pre-loaded media or pre-installed additional apps

The test platform doesn't support pre-loading devices with media or pre-installed apps.

If you want to run tests on an app with pre-loaded data, you can publish a test version of your app with your media files embedded in its APK or app bundle.

Apps using Google Play licensing service

If your app is in an open test or your app has been published to production, pre-launch report testing will use an ID that acts as if it's licensed for your app.

If your app is not in an open test and doesn't have an active APK published to production, your app will fail the licensing check. You'll still receive pre-launch report results, but your app will be in an unlicensed state. To run tests on these apps, you can publish a closed version of your app with licensing services disabled. 

Apps using landscape configuration

The test devices are predefined to run tests on fixed portrait page orientation by default, which may force apps using landscape orientation into portrait mode.

Apps that check for rooted devices

The test platform doesn't support apps that check whether a device has privileged control (root access) on Android.

Sign up for pre-launch report emails

You can receive email notifications for available pre-launch reports by setting up your email preferences in the Play Console. You can choose to receive emails for all tests or only tests with issues.

Was this article helpful?
How can we improve it?