Hotwire follows microservices based architecture. However, splitting the core functionality into microservices is a challenge in itself. Integration testing is very important to make sure all the microservices work well together. That is a primary reason why teams here rely more on test automation. Test automation make it easier for teams to be agile and deliver faster. However, just writing lot of integration tests is not enough; making them deterministic really matters. If tests are non-deterministic, SEs/SDETs end up spending more time maintaining and monitoring the test suites.
Hotwire Supply Team recently deployed a new standalone service – ‘Promo’. It has a good integration test coverage and tests are pretty deterministic. That makes the overall deployment process highly reliable and efficient.
Typical Integration Testing Pattern
In order to make integration testing deterministic it is important to have full control over the test data. We insert/update/delete data through our tests to make sure deterministic data objects are returned in the API response.
As shown below, to validate our API response, data is inserted first; then the request is made and finally the response is asserted. Similarly, we manipulate the data using POST/PUT APIs and then assert the database.
Non-Deterministic Test Automation setup
As soon as developers commit their code, the code is compiled and deployed on to the Dev environment by a continuous integration tool(like Jenkins). Integration tests are then triggered against this environment.
Downside of this setup is – Dev environment is not isolated from the outside world. It is open for other applications as well as users for their functional/dev testing. When database is not isolated, it leads to the tampering of test data. Change of test data leads to test failures. Example – If integration tests are running and someone tweaks the test data, the tests would fail. If this happens invariably, it is hard for engineers to distinguish whether the test failures are because of inconsistent data or because of actual development code. This setup is highly non-deterministic, inefficient and error prone.
AWS/DOCKER To The Rescue
Have the following:
- A dedicated EC2 instance
- Two docker images:
- Docker DB image – MySql
- Docker service image – API.
As shown above, on each commit, continuous integration tool compiles and deploys the artifact on to docker. This service image is made to point to a docker MySql instance. Both docker images are within the same EC2 instance. They are completely isolated from the outside world. After the API is deployed tests are triggered. With such isolation, tests are not prone to tampering of data. That makes the overall test automation absolutely deterministic. After the tests pass artifact is then deployed on to Dev environment. With this setup, the amount of maintenance and monitoring required is almost none.
This setup can also provide an efficient way of doing performance testing of microservices.
Integration testing is an important aspect of API/Microservice development. However, making these tests deterministic is a challenge. Running tests against a non-isolated environment is not ideal and it often leads to non-deterministic test failures. Docker images hosting API and its database within the EC2 instance serves a good isolation to the test environment. Running tests against such setup is highly efficient and deterministic.