Category Archives: mobile

Calabash: Functional Testing For Mobile Apps

Calabash is an automated testing technology for Android and iOS native and hybrid applications. Calabash is a free open source project, developed and maintained by Xamarin.
While Calabash is completely free, Xamarin provides a number of commercial services centered around Calabash and quality assurance for mobile, namely Xamarin Test Cloud consisting of hosted test-execution environments which let you execute Calabash tests on a large number of Android and iOS devices.

How Calabash works in Android:
When a Calabash Android test is executed both your local computer and a device is involved. The device might be an emulator or an actual physical device. The setup looks like this:


Features: The feature files describe the user-stories you want to test. You can test one or more features in one test run.

Step Definitions: Calabash Android comes with a set of predefined step which you can find here. These steps are generic and made to get you up and running fast. When you get more into Calabash you can implement your own custom steps that use the business terms your project uses like I transfer money to my spending’s account or I play “Never Gonna Give You Up”.

Your app: You don’t have to make modifications to your app before testing it.

Instrumentation Test Server: This is another app that will be installed and executed the device. This app is based on ActivityInstrumentationTestCase2 from the Android SDK. It is generated by the Calabash Android framework.

Calabash-android Setup

The First and foremost things is to have all the prerequisites tools installed in your system. The below are the step by step explanation of how to quickly setup your development environment for functional testing of mobile apps (android) using Cucumber and Calabash-android.


  1. You need to have Ruby installed. Verify your installation by running ruby -v in a terminal – it should print “ruby 1.8.7” (or higher).
    If you are on Windows you can get Ruby from

6       2.    During installation check the 2nd and 3rd options i.e.

  • Add Ruby executables to your PATH
  • Associate .rb and .rbw files with this Ruby installation


  1. You should have the Android SDK installed and the environment variable ANDROID_HOME should be pointing to it. Refer here
  2. Install Cucumber
    Navigate to Command Prompt and type “gem install cucumber
  3. Install Calabash-android
    Navigate to Command Prompt and type “gem install calabash-android

Automating an application using Calabash

  • Make sure that the android emulator is running. Refer here.
  • Create a folder on the desktop (e.g. testapp)
  • Download a test app from the internet and place it in the “testapp” folder
  • For this example I have used the GO Contacts EX.apk downloaded from
  • Open the command prompt and navigate to the “testapp” folder8
  • Type  “calabash-android gen” and hit enter twice9
  • features folder will automatically get created in the “testapp” folder10
  • The features folder consists of the following file and directories
    • step_definitions (folder)
    • support (folder)
    • my_first.features (.feature file)
  • 11
  • Edit “my_first.feature” file and replace the content with the following BDD statements

Feature:  As a user of GO Contacts EX.apk
I want to verify the addition and deletion of contacts in the application
Scenario: As a valid user I can add a friend into the contacts list
Given I press button with id “dial_btn_num9”
Given I press button with id “dial_btn_num9”
Given I press button with id “dial_btn_num7”
Given I press button with id “dial_btn_num5”
Given I press button with id “dial_btn_num3”
Given I select new contacts with id “newnumber_title”
Given I should wait for the text “New contact” to appear
When I enter the text “myfirstnumber” into input field number 1
When I enter the text “testnum” into input field number 2
* I press button with id “btn_done”
* I should wait for the text “No Record” to appear
Then I should see the text “No Record”
Then the expected value “No Record” should be equal to the actual value

“my_first.feature” file will now look something like this:


  • Edit the step_definitions folder and edit the calabash_steps.rb ruby file (open in a notepad)

Replace the content of the file with the following code:

When /^I enter the text “([^\”]*)” into input field number (\d+)$/ do |text, number|
performAction(‘enter_text_into_numbered_field’,text, number)

Given /^I press button with id “([^\”]*)”$/ do |button_id|

Given /^I select new contacts with id “([^\”]*)”$/ do |view_id|

Given /^I should wait for the text “([^\”]*)” to appear$/ do |text|
performAction(‘wait_for_text’, text)

Then /^I should see the text “([^\”]*)”$/ do |text|
performAction(‘assert_text’, text, true)

Then /^the expected text “([^\”]*)” should be equal to the actual text$/ do | expected_value |
actual_value =  performAction(‘get_text_by_id’,”recentemppty”)[‘message’]
raise “The current value is #{actual_value}” unless( actual_value == expected_value)

“calabash_steps.rb” ruby file will now look something like this:


  • Navigate to Command Prompt and type calabash-android resign “GO Contacts EX.apk”
  • 14
  • Next type calabash-android run “GO Contacts EX.apk” in the command prompt

Note: Make sure that the mobile emulator is unlocked, before running the test cases

Calabash-Android Pros and Cons


  • It is an Open-Source Tool – No Licensing Fees.
  • Since – The calabash-android is based on the Cucumber framework. The test cases can be easily created in real simple language.
  • Support for all the basic events and movements on the mobile are present in the libraries.
  • It has a thriving forum and Google Group: “Calabash Android”.


  • It takes time to run on an emulator or device as it always installs the app first before starting the first scenario.
  • If a step fails then the subsequent tests in the scenario are skipped
  • It is still in its nascent stage. Support for many complex scenarios or events is not supported. For that either you have to code your way in Ruby or wait for these supports to appear on the scene.
  • We must have the code of the app for identifying the ids of various elements.

Appium: Mobile App Automation Made Awesome

Appium is an open source test automation tool developed and supported by Sauce Labs to automate native and hybrid mobile apps. It uses JSON wire protocol internally to interact with iOS and Android native apps using the Selenium WebDriver.

Automating hybrid and native mobile applications for Android and iOS is a key function handled by Appium, a node.js server. One of the core tenets of Appium is that test codes can be written in any framework or language like Ruby on Rails, C# and Java without having to modify the apps for automation purposes. The interaction between node.js server and Selenium client libraries is what ultimately works together with the mobile application. Appium is open source and can seamlessly run on a variety of devices and emulators making it an apt choice for mobile test automation.

At present Appium only supports Android and iOS based apps but support for Firefox mobile OS is in pipeline.

How Appium works in Android:

Appium runs on real devices and emulators. It takes the Selenium commands from your test code and translates them into a readable format for UIAutomator, using the WebDriver JSON Wire Protocol. UIAutomator is Android’s native UI automation framework which supports running JUnit test cases directly in to the device from the command line. It uses java as a programming language but Appium will make it run from any of the WebDriver supported languages.


Android Requirements

  • Android SDK API >= 16 (SDK < 16 in Selendroid mode)
  • Mac OSX 10.7+ or Windows 7+ or Linux

Appium Setup

Download and Launch the Appium Server

      1. Download Appium for windows from (
      2. Unzip the AppiumForWindows-x.xx.x folder
      3. Open the Appium interface by double clicking on the Appium.exe in the Appium folder 
      4. Run the server by clicking on the Launch button in the Appium interface

2                                5.   Command prompt with the following message will be displayed-info:         Welcome to Appium v x.xx.x


Desired Capabilities

Not all server implementations will support every WebDriver feature. Therefore, the client and server should use JSON objects with the properties listed below when describing which features a user requests that a session support. If a session cannot support a capability that is requested in the desired capabilities, no error is thrown; a read-only capabilities object is returned that indicates the capabilities the session actually supports

· Appium Sever Capabilities 




App The absolute local path or remote http URL to an .ipa or .apkfile, or a .zip containing one of these. Appium will attempt to install this app binary on the appropriate device first. Can also be one of chrome or chromium to launch Chrome or Chromium on Android, or safari to launch Mobile Safari on iOS. Note that this capability is not required for Android if you specify app-packageand app-activity capabilities (see below).


or,chrome, chromium on Android, safari on iOS

browserName (for Selenium compatibility) should always be ”;  this exists because some clients require it to be sent
Device The kind of mobile device or emulator to use iphone, ipad, selendroid, firefoxos, android, mock_ios
Version Android API version, iOS Version

Android —  4.2/4.3

iOS       — 6.0/6.1/7.0

·         Android Only




app-activity Activity name for the Android activity you want to launch from your package MainActivity, .Settings
app-package Java package of the Android app you want to run,
app-wait-activity Activity name for the Android activity you want to wait for SplashActivity



Appium’s Pros and Cons:


  • The beauty of Appium is that, all the complexities are under the hood of Appium server and for an automation developer the programming language and the whole experience would remain same irrespective of the platform he is automating (iOS or Android).
  • The other benefits of Appium is that it opens the door to cross-platform mobile testing which means the same test would work on multiple platforms.
  • Unlike other tools Appium doesn’t require you to include some extra agents in your app to make it automation friendly. It believes in the philosophy of testing the same app which we are going to submit in the app store.
  • It is developed and supported by Sauce Labs and it is getting picked really fast with in the WebDriver community for mobile automation.
  • It can automate Web, Hybrid and Native mobile applications.


  • Scaling up is an important consideration with Continuous Integration and Appium comes across as a great tool to fulfill this expectation. The reason for this is a technical limitation, in iOS we can only run one instance on Instruments per Mac OS so we can only run our iOS scripts on one device per mac machine. So if we want to run our tests on multiple iOS devices at the same time then we would need to arrange the same number of Mac machines, which would be costly affair. But this limitation can be resolved if we execute our scripts in Sauce Lab’s mobile cloud which at present supports running scripts on multiple iOS simulators at the same time.
  • Appium uses UIAutomator for Android automation which only supports Android SDK Platform, API 16 or higher so to support the older APIs they have used another open source library called Selendroid. So I would not say it as a limitation but it is definitely an overhead on the configuration side.