Having a good automation framework is fundamental for a quality app. There are several tools available for mobile test automation but not any framework which ties them all together. At Hotwire we wanted to create a test automation framework for Mobile Native apps which will help us to scale & maintain tests as we add more features to the apps. We came up with following requirements for the frameworks:
- Test scenarios should be written in plain English so that it is easy to maintain & most importantly tests should reflect end user perspective.
- Tests should scale. Able to slice and dice tests based on features set & level of testing like smoke, limited, regression, etc.
- Tests should be reusable in multiple platforms like iOS and Android.
- Framework should support multiple programming language.
- Good reporting capabilities.
- Easy to on board test engineers.
After researching multiple tools we settled on using Appium to support multiple platforms, multiple programming languages. Cucumber for Behavior Driven Development (BDD) so that we can write test scenarios in plain English, able to scale tests with the help of tags and for good reporting. Final piece of the puzzle is to write tests once and run them on multiple platform and the reason we want to do this is because we have feature parity across the platforms. Spring came to the rescue to abstract cucumber tests and actual implementation. We also decided to leverage the concept of Page Objects pattern to decouple UI screens from business logic of test validation.
How it works
Below diagram shows the overall architecture we are putting in place. The idea is to use BDD approach to write tests and a make generic framework which can run tests across multiple platforms. Tests are written in plain English using Cucumber Gherkin language, the main programming language used here in Java which is used to write Step Definition, Model & Screen objects. Model enables to run same tests in multiple platforms like Android, iOS. Every fragment or view is represented in a screen object this enables to separate visual elements from the tests. Maven configuration file (pom.xml) helps taking care of tying up all the required external dependency. The cucumber context files have details for specific mobile platforms like iOS / Android.
- Feature file contains the actual tests in the form of scenarios. Cucumber will execute scenarios with the help of step definition.
- Step definition written in Java is the intermediate layer which translates test scenarios to Java tests.
- Model is used to execute tests on different platforms based on the configuration.
- Screen objects are the representation of the screens / view of a App. All the details of the screen like location of UI elements will be part of this class.
- Appium server is used interact with specific platform like iOS, Android or Windows.
- Native instruments is specific to platform which interacts directly with the app.
- Tests can be run on simulator / emulator or on a physical device.
The core implementation technologies are:
- Java (programming language)
- Spring (development framework)
- Cucumber (framework for BDD)
- Selenium (framework for test automation)
Test Report – Test written in plain English
- Test as documentation.
- Test from customer perspective.
- Reduce regression time. Week to days, Days to hours.
- Write once, runs on multiple platforms ( iOS, Android)
- Works on both simulator & physical device.
- Easy to scale tests & runs on any CI.
- Grouping / Tagging of tests made manageable to run with CI.
- Simple and powerful test reports.
- Easy customization and extension. Example : Analytic testing works out of the box.
We have made this framework open source and it is available in this GitHub repository. https://github.com/HotwireDotCom/bdd-test-automation-for-mobile.