Execute Test - Sauce Labs

As we continue executing tests, increasing the coverage on various devices, form factors becomes a critical path. Sure, we can keep buying devices, however there is not much economies of scale there. Wouldn't it be nice to have that non-core competency of maintaining the devices offloaded to a service provider?

Sure enough, there are many providers out there - Sauce Labs, Perfecto Mobile, ExperiTest etc. I would like to provide an example of Sauce Labs, because they are very aligned with Appium.

What is SauceLabs

Sauce Labs provides both infrastructure and platform cloud services.

Infrastructure: Over 700+ combinations of browsers, os platforms. Mobile emulators and simulators and real devices

Platform: Selenium GRID platform, so that automation tests that talk in WebDriver protocol can execute tests against the underlying infrastructure mentioned above.

User Interface: A nice GUI for manual testing and on-demand availability of browsers, emulators, simulators and real mobile devices.

Sauce Labs features cannot be summed up here, but I would encourage you reading online. You can also read the below link to get an overview:

Cross-Browser Automation - SauceLabs

Running against Sauce Labs

On Real Devices

Upload your app on Sauce Storage. Replace the values with your Sauce credentials

$ curl -u $SAUCE_USERNAME:$SAUCE_ACCESS_KEY -X POST "http://saucelabs.com/rest/v1/storage/$SAUCE_USERNAME/my_app.zip?overwrite=true" -H "Content-Type: application/octet-stream" -data-binary @my_app.zip

Now add the real device configuration in devices.yaml. An example is as below

Feature files (ci_android.feature)

  @ci_add_contact_sauce
  Scenario: Contact on sauce
    Given I open login screen on app
    And I click add contact
    Then I successfully add a contact

Hooks.rb

Before('@ci_add_contact_sauce') do

  caps = sauce_capabilities(app='http://saucelabs.com/example_files/app-debug.apk',appActivity='com.example.android.contactmanager.ContactManager',
                            appPackage='com.example.android.contactmanager')

  @driver= Appium::Driver.new(caps)
  Appium.promote_appium_methods AppiumWorld
  puts "Execute anything before scenario/test case"
  @driver.start_driver
end

Execute

bundle exec cucumber features/ci_android.feature:26 DEVICE=saucelabs_samsung_galaxy_s4

Feedback

Once we kick off the execution, we should instantly see a row on sauce dashboard that represents the execution session.

When the execution is complete, the results should look as below

Clicking the execution session row, we can find information that will help us analyze the results (videos, appium logs, screenshots and detailed logs. Again you can read here to know what saucelabs provides)

Note: The results can also be found in results.html file in the root folder, since we output html results as specified in the cucumber.yml configuration file

On Emulators

Since SauceLabs providers emulators too, the difference is the way we construct Desired Capabilities object, hence replace the values in emulators.yaml. The values can be read from their documentation