Mobile Test Environment

Real Devices vs. Emulators/Simulators

One of the biggest challenges for mobile testing is device support

The number of models and the types of smartphones are increasing by the day and growing at an accelerating pace. There are differences between devices and which devices to test is not an easy challenge to solve. It is dependent on identifying the compatibility matrix as early as possible in the project, yet that becomes a challenge to collect as it is not static (smile). However, we need to start narrowing down with an initial list

Emulators/Simulators cannot emulate every feature of a real device - pixel perfect, phone hardware quirks etc. Official Android documentation suggests the below.

In general the software rep. of underlying hardware in Android world is called "Emulator" and in the iOS world , called "Simulator". Technically, there are more differences and feel free to search online

Summary: Test Environment for mobile devices involves a mix of both real devices and emulators/simulators to get the test coverage we expect, of course trade-off with the cost parameter as always

Initial Manual Testing on Local iOS Simulator

Manual testing - The first experience of the app is generally viewed on an iOS simulator, that runs on a Mac. In the XCode IDE, building and publishing an app to a simulator is an inbuilt feature. It is also possible to publish the app from command line. The subsequent step would be to re-publish the app to a real device by hooking it up to a Mac, either through USB or by joining the network through WiFi (It is assumed that the Mac and real device can talk to each other in the case of Wifi Communication).

Initial Manual Testing on Local Android Real Device

In the case of a local Android device, below are the steps to experience the first build of an app.

• Android SDK installed on a Windows or Mac
• Insure that SDK is updated with latest device drivers. For Mac, my experience has been relatively easier with the device drivers, however for windows, I had to explicitly install the driver by going to the handset device manufacturer's website
• From the source code perspective, the attribute android:debuggable=true should be set
• On the actual device, navigate to Settings > Developer Options > Enable USB debugging and set that to true. If the setting is not found, then navigate to Settings > About phone and click Build number 7 times. That should bring up the option
• Once the device is connected, a dialog box would appear on the device. Accept the message to allow communication
• A quick verification step would be to type adb devices from the terminal (adb binary is located inside the SDK folder inside platform-tools directory)

Initial Manual Testing on a Local Android Emulator

In the Android world, we refer to emulator, which is equivalent to simulator in the iOS World.

AVD: Use Android Virtual Device Manager to create various android virtual devices with different devices, SDK versions and many more hardware characteristics.

Invoke Emulator

• Navigate to Android SDK folder/tools/ to locate the emulator binary
• emulator -avd avd_name invokes the previously created avd
• I found it relatively easier to first create an AVD configuration from IDE like Eclipse or IntelliJ, because the workflow is easier and complex details are abstracted.

Further Manual Testing in the Cloud

We already know that it is almost impossible to have every device, hardware and software - maintain a full blown heterogenous mobile test environment - and certify all devices, emulators for our mobile app. As we scale, we have to depend on Cloud services .
We already did some competitive analysis on the services available Mobile Infra - Critical Success Factors out there for mobile testing (both manual and automation).

we would like to start using Sauce Labs - And Sauce labs supports Appium (a very popular open source test automation framework/tool)

That said, Perfecto mobile on the other hand is a fast growing service in terms of its coverage for mobile real devices and worth taking a look

Other services will also be given due importance in time

Bottom line: We will invest in services that enable us to do more of test automation as the ROI is much better with automation