Revenue, Revenue_USD, Revenue_new, rev2, customer_revenue. Do you recognize these? They might be the column names you might find in your BI or analytics tool. Which is the one to use?
You pick Revenue_new (it’s new, after all!) and proceed with your analysis. A few minutes into the meeting with the sales team, the group startles: the data doesn’t match their expectations.
Something’s wrong.
What data are you using?! Revenue_new? Oh, Revenue_new is the old column. The company moved to customer_revenue last quarter when we hired a new VP of Finance and they updated the definition.
Airbnb faced this problem, too.
Circa 2010, there was only one full-time analyst at the company working on data, and his laptop was effectively the company’s data warehouse. Queries were often run directly against the production databases, and expensive queries occasionally caused serious incidents and took down Airbnb.com.
As the company grew, problems worsened:
Years ago, when Brian, our CEO, would ask simple questions like which city had the most bookings in the previous week, Data Science and Finance would sometimes provide diverging answers using slightly different tables, metric definitions, and business logic.
So Airbnb built an internal product Minerva to solve these problems. Minerva is ubiquitous inside Airbnb – it manages more than 12,000 metrics and 4,000 dimensions across 200 data producers.
MetricFlow is an open-source Minerva.
With MetricFlow, a user can search for metrics, find the one approved by the outgoing VP Finance, receive a notification when the new VPF updates it, see the calculation, and identify the metric’s owner.
MetricFlow and Minerva are useful for BI, but they do more. Teams can build on them as a platform.
Imagine the customer success team asks for a revenue chart in their CRM: it’s as easy as siphoning the data from MetricsFlow via a API call. The software is future proofed. If the definition changes, so do the charts. Updates cascade through the organization.
Here’s a demo that goes into a bit more detail.
In the two weeks since launch, MetricFlow has grown to more than 400 Github stars, and is now the most popular metrics framework by that standard.
If you’re curious about metrics stores and how they fit into the modern data stack and data mesh, you might be interested to attend the first Metric Store Summit next week where people from Spotify, Airbnb, Mode, and Hex will be discussing them.
The title of this post is a play on words from Coleridge’s Rime of the Ancient Mariner. Bad metrics are like salt water. Not much use to a sailor or an analyst.