Mobile App QA Testing: Strategies and Tools to Improve Your Process

Over the last decade, the mobile industry has exploded with more than 5 billion mobile device users worldwide. Companies today need to ensure not only that they can deliver beautiful, intuitive experiences on mobile, but also that they can deliver them on every operating system (OS) and device.

For cross-platform development, many have turned to React Native to build and scale mobile apps on multiple OSs from a single codebase. However, this still presents the challenge of testing the app on the wide variety of devices available in the market to ensure consistency and usability.

In this article, we examine why quality assurance (QA) testing is so challenging for mobile apps in particular and provide guidance for organizations looking to improve their testing process. We discuss both automated and manual testing to help companies find the testing strategy that is right for them. We also call out the specific tools that we recommend for testing teams.

What is mobile testing and why is it so difficult?

To start, let's define mobile testing to set a baseline for understanding. Mobile app testing is the process of assessing functionalities of mobile apps. The purpose of mobile app testing is to ensure the quality of the app, meet specified requirements, reduce the risk or errors, and improve user satisfaction.

To understand why mobile app testing is so challenging, it's easiest to think of it in comparison with web app testing. Web apps run in the browser, which means that tests are limited by the types of browsers and different versions of those browsers. Most engineers will test on the top 7-10 browsers in use today, and for cross-platform browsers like Chrome and Firefox, will test additionally on Windows and Mac OS.

Mobile app testing is more complex because of the sheer number of different OSs and different models of devices that exist. Over time, more and more OSs are being built up independently, beyond the traditional OSs like Apple iOS, Android, and Windows. In addition, each OS supports a wide variety of different devices that range in size and capability.

Of the traditional OSs, Android poses the most significant challenge because there are many different original equipment manufacturers (OEM) that develop Android phones and often modify aspects of the Android OS based on their specific needs. As a result, engineers have struggled to determine how to set best practices for ensuring mobile apps work properly on all devices.

Improving your mobile testing process

Having a strong testing process is instrumental to overcoming these challenges and ensuring your mobile app is of high quality. We have outlined our process below in hopes that it will help organizations find clarity in how to meet best practices for mobile app testing.

List all of the relevant tests for your mobile app

In order to create a comprehensive testing plan, we start by documenting all of the tests that need to be evaluated to ensure the quality of the app. Documenting tests allows the team to track, understand, and resolve any quality issues that may arise. We use Testrail for test case management, as it helps to organize the QA process and ensure every case is properly tracked and resolved.

When listing all of our test cases, we always consider the overall goal and experience of the app itself. Be sure to review all requirements and key benchmarks to determine what tests need to be covered.

Determine which test cases should be automated

Once a comprehensive list of tests has been determined, each test should be evaluated to determine which can be automated and which should be performed manually.

In most cases, automated testing is the best way to improve test efficiency and receive instant feedback. However, there are some cases where it is not technically feasible to automate the tests or it may cost a lot of resources to automate a single step. In these cases, we review whether automating the scenario would return a greater ROI compared to manual testing. If there is concern that time and resources would be wasted when automating a complex test, we always choose to perform it manually.

Luckily, Appium makes it easy for us to automate a variety common tests. Some examples include:

• Identify UI elements
• User onboarding
• Login and registration
• Credit card purchase
• Forgot password

We highly recommend Appium as an automation testing tool because its open-source, simple to execute, and able to test cross-platform. You can find some examples of Appium tests here.

Research what devices you will need to test

As mentioned earlier, the sheer number of different devices available on the market presents a significant challenge for mobile testing. While it would be extremely time-consuming and expensive to test every device on the market, it is possible to research and understand which devices need to be tested.

For Apple iOS, testing is a bit more straightforward because it is closed ecosystem with the same OS for phone and tablet. With some research, it is easy to identify the most popular and most recent models that will need to be tested. We typically test on the latest 6 versions, checking both phone and tablet.

Android is much tricker because there are so many manufacturers that modify the Android OS code. To cover this broad range, we pick the most popular devices from each manufacturer to test. AppBrain is a great source of data we use to determine which devices are most popular. We also consider devices with different screen resolutions, memory size, and connectivity to ensure we are able to check a wide variety of devices.

For all other OSs, we also recommend performing research to determine the most popular devices to test, as well as testing on latest 6 versions. Again, it is important to check a variety of screen resolutions as well to make sure the mobile app will display correctly for users on all different screens.

Create a testing plan for testing different models

Once we have determined our list of tests and devices to test, we are able to structure our testing plan.

Testing on real devices is always preferred when possible for the highest accuracy of results. However, procuring mobile devices for testing can be expensive and often impractical depending on the distribution of the team. Instead, we often utilize cloud-based tools and emulators for testing like BrowserStack and Sauce Labs.

To test all models efficiently, we group devices based on their screen resolution and cycle through the needed tests on each group. We typically create testing plans based on the number of groups we will test each week. Depending on the app, we plan for about 3-8 weeks of QA testing prior to launch or new feature release. Most testing is performed concurrently with development to maximize efficiency and ensure features are complete upon release.

In conclusion

With more and more technology innovations appearing on mobile, mobile app testing is more critical than ever. Having the right process in place is instrumental to ensuring your app will delight users and run smoothly.

To learn more about how you can develop and test your mobile app, feel free to send us a message and we would be happy to answer any of your questions.