AWS Device Farm – Mobile App Testing on Real Devices

This is an explanation of Amazon Device Farm and how it works for iOS and Android devices. It allows users to create a project, identify an application, and configure and run a test. You can use it to test mobile apps on real devices.

Amazon Device Farm allows you to test your apps in parallel against a large collection of physical devices that you can choose from the AWS Cloud. This helps to improve the quality of your app.

The test runs that are performed in farm can be one of two types: manual test run and automated test run. Manual test run can be run by remote access; we can select a device and do all of the testing manually. In an automated test run, testing can be done by creating a new test run and performing all of the testing by importing the test suite and application for iOS or Android in the device farm.

The list below contains the test types on various platforms:

For Android:

• Appium Java Junit
• Appium Java TestNG
• Calabash
• Instrumentation
• UI Automator
• Explorer

For iOS:

• Appium Java Junit
• Appium Java TestNG
• Calabash
• UI Automation
• XCTest (including KIF)

For web apps:

• Appium Java Junit
• Appium Java TestNG

How to set up a new project and test run (Manual Test):

Log in to AWS, go to Device Farm, and select the “Device Farm” service.

Click on ‘Create New Project.’

Enter your Project Name in the text box and click on the “Create Project” button.

Click on a newly created project. You will be automatically redirected to “Create new run.” Click on the “Remote Access” tab and click on “Start a new session.”

Now we need to either select a device from the list given or enter a device on which we want to test our application. We can select any iOS or Android device.

After selecting a device, we need to “Confirm and start session.”

It takes a while for Device Farm to make the device available for testing. Then the set up process takes place for requesting a device.

Once the devices become available, we can see the screen and access it through the Console. Now we need to upload the APK file or IPA file from the Upload option – located on the right – to see the application through Console and test it.

We can test the app manually by performing the basic gestures and clicking on the functions.

We can perform a testing of the app by executing all of the test cases and see the results.

Features and Advantages

• AWS Device Farm tests are run on real, non-rooted devices
• The remote Access results contain logs and a video of the session
• The default orientation of devices can be switched between portrait (vertical) and landscape (horizontal) mode
• Device Farm stores reports for 400 days

Disadvantages

• Access of devices is slow
• Sometimes requested devices take a lot of time to become available
• Device connection in between testing gets disconnected
• No option to simulate common gestures like Zoom In, Zoom Out, shake etc. in ADF
• There is a sixty-minute limit length for a remote access session. After sixty minutes, we have to create a new session and add devices to proceed further
• There is no confirmation message displayed if an app is finished installing in a device