Testing Challenges with Beacon Applications

Beacon applications offer a wide variety of opportunities, but there are challenges that come with testing them. Beacons are small, low-cost, battery-operated pieces of hardware that can easily be attached to a wall. This new class of low-powered, cost-effective transmitters uses Bluetooth connections to transmit data regularly. Any mobile device that supports Bluetooth Low Energy can become a Beacon and can detect other beacons when they are nearby.

Beacons use Bluetooth Low Energy to broadcast signals which are gathered up by a compatible applications/OS. Bluetooth Low Energy is a network technology which is used to transmit data over short distances. The identifier can then be looked up over the internet to determine the device’s physical location or trigger an action on the device such as a push notification.

All the devices which are in range with Bluetooth signals on can transmit data packets at an interval ranging from 10 ms to 10 seconds. This interval also impacts the battery life of the beacon. Shorter intervals result in quicker discovery of devices but it means shorter battery life.

Our Proof-of-Concept Beacon Application

We at 3Pillar developed an application for iOS as well as Android devices for tracking attendees in various meeting rooms of an organization.

For the proof-of-concept application, each meeting room was installed with one beacon, and as soon as someone entered a room, if the app is installed and running on that person’s iOS/Android device, then the app would send the information of the person who entered and the beacon (in range) to the server. Thus, the count in that particular room would be increased by one. Similarly, if someone leaves the room and goes out of range from room’s beacon, then again the server is notified and the attendee count is decreased by one. The application was also covering multiple locations, like Noida, Cluj, Romania, and the USA. Individuals entering at different locations are also visible to everyone with this application installed.

To test the beacon application, we followed the following steps:

1. Installed the App on multiple iOS/Android devices.

2. Logged in to the App on all the devices with different user IDs.

3. Placed beacons in different meeting rooms (1 beacon per room) at 4 meeting rooms of

one geolocation and 2 meeting rooms of another geolocation.

4. Moved in to one room with iOS OR Android device.

5. Verified that the count of attendee in that room is increased by 1.

6. Moved out of that room.

7. Verified that the count of attendee in that room is decreased by 1.

8. Moved in to one room with both iOS and Android device.

9. Verified that the count of attendee in that room is increased by 2.

10. Moved out of that room.

11. Verified that the count of attendee in that room is decreased by 2.

12. User with different User Id enters different rooms at different Geo-Locations.

13. Verified that the count of attendee in each room is increased by 1.

Challenges in Testing

Well, testing a beacon based application is not as straight forward as testing any other application. There were many obstacles which we faced while testing this application. Below I have jotted down a few of those:

1. Range of Beacons: Ranging is an important feature of Beacons; the signals that beacons generate are responsible for searching any compatible devices in range. This is where we faced issues, as the detection of devices was not consistent and sometimes even when the iOS/Android device was in range of the beacon it was not detected by the beacon. It happened many times that the range was changed from ‘Near’ to ‘Far’ even when the beacon and mobile devices were placed closed to each other.

2. Battery Drainage: Battery drainage was another issue that we faced during testing. When the application was running on the devices, the battery drainage varied from 14%-19%. However the drainage problem in Android was slightly lower as compared to iOS devices.

3. Internet connectivity: This is another challenge that we faced. If the internet is disconnected for a fraction of second, then application will not respond and the record of the person will not be updated. Once the internet connection is reestablished, then the application will start running again and then only it will send the information to the server to reflect the user count.

4. Placement of Beacon: Well, placement of the beacon is very important in testing any app which is based on beacon technology. Because even If the beacon range is good and there is some obstacle between the beacon and the mobile device, then the detection of the mobile device can be a problem. We saw this frequently during our testing.

5. Multiple Geo-Location verification: As the entire team was in one geolocation, it was quite a challenge to test the application at other geolocations and for this we requested our stakeholders to install the app on their mobile devices and use the same from other geolocation simultaneously.

Prerequisites

Like other applications, our application also has certain prerequisites as follows:

1. Location services (A general class of computer program-level services that use device location data to control features. This is an information service which is accessible with mobile devices through the mobile network and which uses information on the geographical position of the mobile device) should always be up and running.

2. Internet connectivity should be on always.

3. Bluetooth should always be On in mobile device.

4. App should be running on mobile device.

QA Best practices

1. To overcome the beacon ranging issue, the QA team, in coordination with the dev team, started the ranging as soon as monitoring started and the method to stop ranging was set not to be called ever.

2. To reduce battery drainage, the app should be run in the background and more the beacon ranging time is set in settings, lesser will be the battery drainage.

3. Internet connectivity issue can be overcome by a high-speed regular internet connection, which can cover the mobile device in an entire company’s location.

4. Beacons should be mounted on the wall or roof of the rooms so that there will be no obstacles between them and mobile devices.

5. For multiple geolocation testing, we can use the Android SDK to send mock GPS information to a real device. The Android SDK theoretically allows us to do so.