Learn from our experience developing the Deliveroo product.
We had an incident yesterday. It wasn’t that bad as incidents go, but it got me really worried — it’s something that has the potential to become much worse as we move towards a distributed, multi-service world.
We’re not particularly proud that we are still using Rails 3.2, but we are extremely proud of scaling our traffic over the last few years. Our new services are built in Rails 5, but we still have a hefty chunk of functionality in our original Rails 3.x monolith. A rueful “rails 3.old” hits our slack channels as someone stumbles across an issue as they switch back to the monolith. Growing our business, features and team over the last few years has left little time to do a wholesale upgrade of the monolith platform. Here are some tactical things we’ve done to help it keep pace with our growth.
Counting unique users, checking if a credit card has already been used, or checking if this is a mobile user’s first visit ever — all of these require maintaining a large set of fingerprints (unique visitor ID, card fingerprint, IDFVs depending on the use case).
Because this usually needs to be queried very rapidly, Redis is naturally our store of choice. While using its
SETfeels obvious, what data structure to select? Are there memory/performance compromises?
This shows that while plain key/value is a safe bet, there are possible optimisations with hashes and traps to avoid with sets and sorted sets.