Our Android application originally started out using AsyncTask to make our api calls. This tightly coupled the networking calls to our Activity lifecycle. Any configuration change would force the Activity to be restarted, and by extension the AsyncTask would be restarted. At Hotwire we wanted to change how we were making and handling our api calls and decouple this from the Activity lifecycle. During the change to a more robust and thought-out architecture for handling our data we wanted to ensure that we built in client-side caching, made the new framework easily expandable to new requirements, abstract away where the source of data was originating, and separate business logic from the data requests. Our solution was to create a module, the DataLayer, which at a fundamental level is made up of three main components: the DataAccessLayer, which implements CRUD operations and is used by the client application for communicating with the module; the DataStoreManager, which houses the control logic for how data is stored and retrieved; and the DataStore, which is a realization of a specific location where data can be created, read, updated, and deleted. These components are not exhaustive, later there will be a more in-depth explanation of each component.
The specific problems we wanted to solve were
- Networking calls using AsyncTasks
- AsyncTasks that were tightly coupled to Activity lifecycle being interrupted by configuration changes
- Data retrieval was happening at different locations throughout the application
Our hypothesis was that by creating a separate module in the application we could consolidate and standardize how and from where data was retrieved. This would abstract the underlying logic and complexity from the presenters and view classes. At the time we also were undergoing an architectural shift to MVP and the Model was missing, making this a good opportunity to create a solution that fit well into the new architecture.
Design & Architecture
The first priority in the framework/module was separation of concerns and adhering to the MVP architecture. Next we wanted the new module to be designed in a way that allowed it easily to be expanded on in the future. Below you can see the high level architecture that was chosen for our data layer module.
This is the interface that should be used by the client to request data from the module. This class can and be tailored to the specific needs of the client(s). Typically this will start with the basic CRUD operations, and can be expanded on as the client sees a need for functionality that is repeated. Ultimately in our application we use Dagger to inject this class into the Presenters allowing the Presenters to request data from the module.
This is the realization of the above interface, IDataAccessLayer.
This class will be interface to the brains of the module, the DataStoreManager, the class that contains all logic for reading/writing/updating/deleting from the DataStores. The interface here should remain relatively unchanged, and should not change very often. The DataAccessLayer class is tailored to the client, but must utilized the methods exposed in this interface to accomplish its task.
The DataStoreManager is where all of the brains of the module live, and where all of the data retrieval decisions are made. We are using a repository-like pattern (although not strictly adhering to it) to fetch data from the multiple DataStores. In our case, these data store types are either from our internal API, the sqlite database on the client, or from one of our third-party APIs. The developer can decide from the presenter when the request is made, the order in which the DataStores should be queried for data. The default will be to check the database for data, then if successful return, else if a failure make an api call from the proper api DataStore for the data.
In this class, RxJava is used to chain observables of the DataStore together, and handles errors that are thrown by the DataStore(s) or from the module itself. There are many errors that are thrown internally that are expected and handled by the manager, but the errors that cannot and are not handled are propagated up the chain back to the presenter where an action can be taken to ignore the error, or display something to notify the user that an action they’ve taken has resulted in an error
DataStore is what we have chosen to call, at a high level, a location/system where data can be retrieve or written. This is the abstract class that all other concrete DataStores should derive. By having a more abstract DataStore as the top level object, it allows for the DataStoreManager to operate on an DataStore the same way for every request without convoluting the manager with complex logic that is specific to a single DataStore. This also houses the content that the application passes (required for database operations), a logger, and the initial DataLayerRequest that was sent by the presenter.
The APIDataStore class is the parent class that all internal api calls must derive from. This class contains all of the boiler plate and common code that every internal api call must have.
The DBDataStore class is the parent class of every high level object we would like to persist to the local database. It contains the SQLiteHelper for setting up and tearing down the database connection, and the Factory class that provides Transformer objects to facilitate mapping of objects that are used for persisting data to the database to the objects that are used by the presenter, and vice versa.
To keep our internal api calls and ThirdParty api calls separate, we have created a third DataStore type. This class currently is pretty sparse, and is only used to help facilitate API calls to Uber, but we anticipate its utilization to increase as more ThirdParty api calls are made from our application.
Example Data Flow
Case 1: Happy path Success
This flow diagram shows the happy path for a data request that can be fulfilled by the Database and is returned back to the Presenter.
Case 2: Error, Success
This flow diagram shows the case where a data request errors when attempting a read from the Database, but is fulfilled by the Api
Case 3: Error, Error
This flow diagram shows the case where a data request errors when attempting a read from the Database, and also errors when making an Api call
- Implementing all of our new api calls as their own respective DataStores, and all networking will be moved out of the main app module.
- We would like to add an in-memory cache for scenarios where we might want to retain data for some time, but not long enough to persist it
- Potentially porting the module to a common library that can be used on both Android & iOS
- Add a domain layer with common business logic
- Refactor the module to be more generic, and open source the module for others to benefit from
Although the module took some time to build. It was the last piece of our new MVP architecture and will finally allow us to finish the migration of all the pages in our application. The module isn’t perfect and is constantly being improved upon as new needs arise, but it is currently serving its purpose and functioning as we intended.