How to Overcome Challenges Faced in Hybrid App Testing

The mobile boom is self-evident, and it is here to stay. Smartphones are rapidly becoming the primary means of interaction for consumers and businesses worldwide, with thousands of apps created each day. Mobile goes beyond smartphones and tablets. Today, apps are incorporated into cars, wearable tech, and home appliances.

Hybrid apps are being adopted by people of all age groups across industry verticals. Most applications today, including famous ones like Spotify, Slack, Skype, and others, are all hybrid. However, hybrid apps pose some unique challenges when it comes to QA. Here, I will be giving insights into how the nature of hybrid apps requires an additional consideration during the testing strategy.

What are Hybrid apps, exactly?

Hybrid apps provide the best of both native applications and web applications using HTML 5. Hybrid mobile apps are built with a combination of web technologies like HTML, CSS, and JavaScript. A hybrid app is hosted inside a native application that utilizes a mobile platform’s WebView. The native wrapper app uses OS APIs to create an embedded HTML rendering engine, which provides a bridge between the browser and device APIs.

With a native app, you have either an iOS code or an Android code, and you have to write code either for one of them or two sets of code. However, if you are using a hybrid framework, what you could do is build your code on top of that code, and the framework will translate everything into two different platforms. Let’s say, an iPhone takes a picture in a certain way, and Android takes a picture in a different way using two sets of codes. This framework will consider that and will give you one standard set of codes.

Challenges In Hybrid App Development

For native apps, the scope of testing is confined to either iOS or Android, but in the case of Hybrid apps, the extent of testing gets wider. Based on my experience, I highlighted the most common issues mobile application testers face daily. However, many other challenges can be case-specific, and I did not include them on my list.

A Hybrid app needs to be tested on two different platforms:

• iOS
• Android

For Android, the OS-versions we tested was from version 4.4 to the latest version. For iOS, we tested upwards from version 8.0.

For different platforms, testing is required for the following:

Different OS Versions

We sometimes find issues that are specific to one OS version. For example, in my last project, I faced a problem where in one module, the text was truncated at the bottom in iOS8, whereas in iOS9 it was working fine. It was therefore required to be verified on all the different platforms available.

Testing your app on various screen resolutions

The layout of the app needs to be checked on different screen resolutions. This is necessary because we may have a significant number of screen resolutions available in the market for one mobile OS (like Android).

UI/UX Testing

UI/UX testing for hybrid applications involves testing across various devices and operating systems, as all devices carry the conventions that their users expect. This approach ensures that the UI elements, like toolbar, menus, images, lists, and edit fields, in the hybrid application work consistently across multiple target platforms and provide a better usability experience.

Testing your app in varied region and time format settings

The application sometimes has different functionality depending on region and time format settings. For example, changing the region and time format of the device will also modify the time format in your app.

Testing the app with other third-party applications

Some users install third-party apps that are accessible in your hybrid application. Sometimes these third-party apps can crash your application. Therefore, hybrid apps are required to be tested with third-party apps that may be used within your application. For example, when we tried using an app we developed with the Swype keyboard, the app crashed.

Testing your App in various Network condition

Many applications today are network dependent, so the app needs to be tested on different network conditions like signal loss and bandwidth variations. The variability of a network can have different impacts on the application; for example, unpredictable application behavior, user interface related errors, database corruption, performance challenges and functional issues.

To test applications under different network conditions, I throttled the connection using Charles Proxy tool. Charles adjusts the bandwidth and latency of your Internet connection.

Automation and test case writing

Hybrid apps can make automation both easy and hard. If it’s built only on web-based languages and adapted with as little native coding as possible, it is effortless for QA engineers to write automated scripts that function across various OSes, which is indeed a novelty. However, if the app is created with entirely different languages and has very different navigation features, then writing new automated scripts is required for each supported platform.

Beyond functional testing, performance must be assessed on the individual device level as well as from a usability and UX perspective. The performance of a hybrid app is moderate compared to native apps. Therefore, certain features in your app may take a little extra time to respond in comparison to native apps.

One of the advantages with hybrid applications is that development and maintenance can be more straightforward because one fix in the web could mend an issue in all the supported platforms. However, it is not at all easy for testing. This is because even if a fix or newly developed feature is just in the web code, you cannot take for granted that it will work the same on all the platforms. We saw many cases where a fix would work on iOS but not on Android, or worked on iOS 9.0 but not on iOS 8.1. So for each small fix in the web, it still needs to be tested on a large number of devices.

Because of this, I started to look at some automation frameworks and started using Appium, which is a cross-platform test framework that has support for Android, iOS and Windows Phone.

Key takeaways from developing Hybrid apps

• Hybrid apps take less time to develop. If you have less than four months to develop an app, and you want to test a limited private market on the viability of your app, then use Hybrid.
• Hybrid applications provide cross-platform compatibility, with an emphasis on accessibility to native features, which results in good user experience.
• Hybrid apps have a lower total cost of ownership owing to easier upgrades, lower maintenance costs, and development using familiar tools and technologies.
• Hybrid apps provide access to native features like camera, GPS, Bluetooth, contacts, etc.
• You can use your existing web talent and don’t need to bring on additional resources.
• Hidden bug potential – Hybrid apps add an extra layer to counter any bugs that may exist. Frameworks themselves can sometimes have bugs. 
• Hybrid apps can be made available and distributed via a relevant app store, just like native apps.
• Hybrid apps appear to the end user as native apps but run significantly slower than native apps.

Best Practices for Hybrid Testing

The following are best practices that should be kept in mind while testing a hybrid mobile app:

Functional Testing

• Test the app as a Black Box and try to break it
• UI testing should focus on validating how the application handles the touch events, mouse events, keyboard strokes, and gestures against the custom UI guidelines.
• UI/UX features such as page transitions (with slide up, slide down), swipes (with a tap, tap hold, double tap, swipe left, swipe right features), and platform-specific look and feel of controls (like time picker) are required to be tested on different platforms.

Non-Functional Testing

• Performance Testing: It has been observed that 60% of users abandon a mobile application if it does not load within 3 seconds, so performance should be one of the critical concerns for mobile app developers. Hybrid applications are slower than native apps.
• Security Testing: If your app is writing sensitive information to the file system at any point, such as credentials, signature files, usernames, passwords, and API keys, authentication tokens should be verified on all the platforms.
• Don’t just restrict non-functional testing to performance or security. Carry out the following tests:
  • Installation testing
  • Uninstallation testing
  • Idle state testing
  • Offline mode testing
  • Interprocess communication testing such as incoming call, text message, and other app notifications while using app
  • Battery performance while using the app
  • Performance of app in different network conditions

Beta Test App for Early Feedback

For early feedback, we can distribute the app internally through various tools available. Some tools to consider using include: Test Flight for an iOS app, Crashlytics for iOS and Android, and HockeyApp for iOS and Android.

To be Hybrid or not?

Below is a checklist that will help us figure out if hybrid apps are the best way forward given the requirements, context and end users.

• Hybrid app development may become popular for developers and companies that do not have a platform preference.
• We have many clients that want to develop relatively simple mobile apps for several mobile platforms at the same time, and they want to get these apps out there as quickly as possible, before their competitors.

If we look at statistics (such as the recent survey by RedHat), 50% of businesses today need mobile development specialists. However, what’s most interesting is that among these firms, native app developers are not in high demand. As it turns out, front-end web developers and back-end integration specialists take care of a significant portion of mobile development tasks these days. I think this is a good indication that mobile development is shifting from native development towards hybrid app development.

While it is impossible to foresee every possible outcome, we can certainly make a few safe assertions. It is safe to assume that hybrid app development will continue to grow in popularity, for both consumers and developers.

Hybrid mobile apps are here to stay and are certainly a big part of the future of overall mobile applications development. It will be interesting to see the next few generations of hybrid mobile apps, and how they support the next generation of mobile devices as they emerge onto the consumer market.