Introducing Facet, A Revolutionary User Interface for High-Dimensional Data

In 2011 Metamarkets launched Druid, a streaming datastore capable of processing billions of events in real-time. Today we’re proud to release Facet, an interface designed to put Druid’s full power at every user’s fingertips.

We built Facet to overcome the inflexibility of traditional reporting and analytics interfaces. While many systems are adept at telling you what is happening, Facet helps you uncover why it’s happening — quickly, intuitively and without the assistance of an analyst or Business Intelligence expert.

One of Facet’s secrets is in how it handles the high-dimensional datasets common to the programmatic marketing space. In this post, I’ll talk about what that means, what makes Facet’s approach different, and where we’re planning to take it from here.

 

Data Exploration and the Curse of Dimensionality
It’s a well-worn trope among analysts that data does not equal insights. Insights require the ability to dig deeper, to filter, sort, and arrange data so you can get to the real story. In short, the key to insights is exploration.

Having developed data exploration tools in finance, e-commerce and genomics, I can safely say that programmatic marketing’s data presents the greatest challenge. It’s not just the billions of events each of our clients handle daily, it’s the number of dimensions associated with each event. A dimension can be anything from a ZIP code to a Device ID, and most programmatic auctions have around 100 of them.

Some systems try and compensate for this complexity by precomputing hundreds or even millions of slices. This approach breaks down with highly-dimensional data. To put a fine point on it: there are more ways to sort, filter, and arrange 10 billion auction events, each with 100 dimensions, than there are transistors in the world. Mathematicians call this the curse of dimensionality, and it’s where most reporting and analytics systems hit the wall.

For example, do you want to see the top performers by ad size or client or campaigns or location? No problem, any reporting system can do it. But what about top performers by ad size by client by campaign by location? Fugetaboutit, you’ve been cursed.

The average account/campaign manager bumps up against the curse a dozen or so times a day, and their only option is to stop what they’re doing and either run their own SQL query or ask an analyst for help. One of our clients estimated that last year, he personally spent 300 hours waiting for SQL queries to return.

 

Lifting the Curse
Facet blasts the curse of dimensionality with two big guns:

The first is its tight coupling with our Druid backend. True to its name, Facet breaks up complex queries into small pieces that are optimized for Druid’s highly-parallelized approach to data processing. The whole process takes milliseconds, and nothing is precomputed.

The second is a smart interface design. Experienced analysts use a common set of verbs to describe data exploration, such as “filter”, “split/pivot”, “sort”, and “compare.”  Facet exposes these primitives in easy-to-use interface elements: autocomplete-style selection of filters; a split bar that allows quick definition of up to five dimensions; context menus for advanced sorting on metrics; and compare functionality built into the date picker.

Anthony Hitchings, Director of Ad Operations at the Financial Times, describes Facet as something akin to a “pivot table on steroids…[a tool that enables his team to] quickly connect the dots and identify new revenue opportunities.”

Russell Glass, Head of Products at LinkedIn Marketing Solutions and Former CEO and Founder of Bizo, said of Facet, “Every programmatic buyer needs an interface as fast and flexible as this.”

But it’s not just about getting back time and money. Facet encourages people to ask questions they’ve never asked before. It makes it easy for anyone (from summer intern to CEO) to satisfy their curiosity, follow hunches, and confirm theories on how to make things better. Ultimately, this practice separates companies that consistently win from the rest of the pack.

 

What’s Next
When it comes to information density and clarity of presentation, few data visualizations can beat the simple effectiveness of tables.  As such, we’ve focused Facet’s launch on exploring multi-dimensional views of data in tables with advanced sorting and compare features.

However, Facet’s architecture lends itself easily to enabling richer kinds of data visualizations, many of which will flow contextually from the table of data being viewed. Watch this space as we roll out new visualizations. In the meantime, below is a sneak peek, and if you’d like a demo of Facet, don’t hesitate to reach out.