Hotwire Tech Blog

Scribes from Hotwire Engineering

What’s the problem?

Most services handle large data amount. It can come from different places, such as Kafka, RabbitMQ, ActiveMQ. Large amount of data can slow down your application or even make it dead.

We know it, but what is the solution?

We need a realtime computation system focused on distributed processing of large data streams.

  • Apache Stormis a distributed stream processing computation…

    Read more...

Goal

Currently at Hotwire, when developing new mobile features, the business logic and UI is written separately for the Android and iOS app. This is especially time-consuming when developing features that have a lot of overlap. Not only does this take more development time, but also both platforms end up having their own bugs. We felt that it was an interesting time to try out a cross-development tool to build features in a more unified way. One of the promising tools we decided to try out is React Native, a mobile, JavaScript Framework. In this post, I will be discussing the basics and advantages of React Native. In Part 2, I will be going into more detail regarding integrating…

Read more...

Why you should automate your documentation

What is the difference between great web service and ordinary one? In fact, there are a lot of distinctions, but one of the most significant is, of course, documentation.

This is the first thing the users see when start using your web service, and, possibly, they will never go deeper to your source code and never see how cool it is. In this case the first impression is really valuable.

I think, you’ll agree with me, that poor documentation leaves dramatically poor impression, and decrease the pleasure of using any service. It also can erase all of the greatest advantages of your service.

It is pretty straightforward, but at…

Read more...

Overview

We will look into running Jobs on Spark cluster and configuring the settings to fine tune a simple example to achieve significantly lower runtimes. We will also allude to the trade off between setting number of tasks per executors and number of executors per node given a cluster node configuration.

Spark Cluster Abstraction

This diagram (source) shows the key components of the cluster. Spark driver is the program you write. The Driver program demands for executors from the Master (the Worker nodes lets the Master know how much resource they each have). The Master allocates the resources…

Read more...

Problem:

In a service-oriented architecture, often there is a need to cache data. Hotwire Search Team has built a data store for caching static information about hotels such as star rating, neighborhood data, amenities, etc. which is backed by in-memory DB Redis (Elasticache). This cache is exposed via an API called Hotel Static Data Service (HSDS). HSDS was sending a very large response to its clients that needed only a small subset of that information. This caused latencies in critical services that were using HSDS within the search flow.

Current Architecture:

There are a number of services that get called from the time a user enters a search…

Read more...