The Google Play has become an overwhelming distribution channel for developers building their next-big-things, either in forms of applications, games or even services. This ever-expanding marketplace gives developers a global reach and target coverage of hundreds of different Android devices on a rapidly growing and changing platform.

However, due to variety in Android OS/platform versions, fragmentation does exist. Android fragmentation is known to cause multiple and incompatible versions of the system specific to the OEM and that causes applications and games to work differently (or wrongly) on many Android devices.

But, ask any original equipment manufacturer (OEM) and they think it’s a great idea to put in a custom resolution with their own custom user interface (UI), middleware, drivers and all those must-have bells and whistles. For OEMs, it’s not about fragmentation, but differentiation. It’s about the way to stand out from the Android masses. For developers, the Android fragmentation is an issue and makes their heads ache.

In this first blog as part of our blog series, Deconstructing Mobile Testing Methods, we’ll look at the most widely used Android testing methods – Robotium and uiautomator. We hope to give you a glance into both of these.

Access and Range of App Testing

The most notable limitation in Android Instrumentation Frameworks – including Robotium – is that it lets click throughout only on the application that’s under testing. For example, if the application opens the camera and tries to take a snapshot, the test ends with a fail for whole test.

Why? Because of a permission to perform a click from one application to another. It’s related to Android’s security model. The uiautomator doesn’t have this limitation, but it allows taking pictures in one application and enable access to change device settings (admin settings) in second application.

UI Elements

For recognizing UI elements, the Robotium is much more accurate because it let’s tests to click on elements by their resource ID that provides a more accurate element identification. In addition to ID, the elements can be recognized by the content.

For example, index, text, etc. In uiautomator the same can be done with resource IDs as uiautomator has a general accessibility on labels, e.g. text, description, etc. But if there are more elements with the same text, there’s a need to add index for the instance. And,  if the UI changes dynamically, it might be a big problem. As uiautomator actually let’s a test to click throughout device and text descriptions, such as “Settings” can cause issues as there are “Settings” and “Option settings”. For this sort of recognition, it’s much harder to write an universal tests in uiautomator.

Application Programming Interfaces

The biggest advantage for uiautomator is its API. It provides five different classes for developers to use. These classes, with interfaces and exceptions, allow developers to capture and manipulate UI components on the target application. These classes are:

com.android.uiautomator.core.UiCollection;
com.android.uiautomator.core.UiDevice;
com.android.uiautomator.core.UiObject;
com.android.uiautomator.core.UiScrollable;
com.android.uiautomator.core.UiSelector;


Let’s take the UiSelector class as an example. This class provides the mechanism for tests to describe the UI elements they intend to target. As the user interface element has many properties associated with it,UiSelector provides methods to create nice looking and easy-to-understand queries. Additionally, theUiSelector class allows targeting of UI elements within a specific display hierarchies to distinguish similar elements based in the hierarchies they’re in. For example:

new UiSelector().className("Button").textContains("the").index(4)

That is very straightforward, clean and easy. Finally, the big benefit of this API is that it’s managed by Google. Sure, it gives a certain type of confidence that the API is maintained and managed in an appropriate way and no single individual will remove its methods.

But let’s compare this to the Robotium API which provides only one all-you-can-eat-type of class – the Soloclass. The difference is significant. Robotium’s API is enormous, containing methods for getting views, clicking views, waiting views, etc. It’s more complex so to speak, but on the other hand, provides better control over the testing. But, there are missing pieces such as clickOnButton(View). This is an interesting missing piece as the API offers methods like clickOnView(View). Another good example of different logic in Robotium API is that it provides clickOnButton(String text) but it doesn’t provide clickOnCheckBox(String name). Generally, the structure of those queries for UI elements is just the normal imperative code (in uiautomator it looks like declarative style),  so the previous example would look like this in the case of Robotium:

int index = 0;
Button buttonToFind;
for (Button button : solo.getCurrentViews(Button.class)) {
    if (button.getText().contains("the")) {
      index++;
       if (index == 5) {
         buttonToFind = button;
      break;
       }
    }
}


On the other hand, Robotium code is trivial for anyone ever programmed C or C++. However, some recent changes in Robotium – yes, we’re talking about removing methods without deprecating those – can represent a huge challenge for testers with existing, written test cases. For example, getCurrentButtons,getCurrentImages and so on, were removed and replaced with getCurrentViews(class) without marking those as deprecated functions.

Backwards Compatibility and Framework Availability

In availability for different API levels, Robotium is clearly beating uiautomator – at least today. The uiautomator is relatively new mechanism and it supports devices with API level 16 (or newer). The Robotium goes way back to API level 7 with its support. Actually, uiautomator doesn’t go too far behind to support older API levels. Today, Gingerbread (Android 2.3 – 2.3.7, API levels 9 and 10), Honeycomb (Android 3.0 – 3.2, API levels 11, 12 and 13) and Ice Cream Sandwich (Android 4.0 – 4.0.4, API levels 14 and 15) have still significant market coverage, in terms of volumes and used apps. Therefore, for developers it is much easier to go back to the older versions with Robotium.

IDE integration

Android Instrumentation Framework provides a complete integration with ADT tools, so developer/tester can just create a test project using ADT and run it using Eclipse special runner. When it comes to uiautomator, developers/testers need to to create regular Java project, manually add JUnit library as well as Android.jar and uiautomator.jar, and then build it using ant. To run it successfully, developer needs to manually copy it to device using ADB.

WebView

One feature that is lacking in uiautomator is there’s no way to directly access Android objects (Views) and there’s a limitation to handle WebView. Because there’s no way to access WebView, testers cannot inject JavaScript – which is clearly the easiest and the best way to handle those tests. Currently, there’s nothing testers could do inside WebView with uiautomator. Futhermore, uiautomator lacks of methods to click long on device (x,y) but it only provides methods that are usable and clickable on a specified view.

For our conclusion, both of these testing frameworks have their pros and cons and it depends on the nature of tested app what is more suitable than the other. In the next series, we’ll show an example how to use Twitter application as an example and construct simple test cases for it using both, Robotium and uiautomator.

See you again next week!