Five Tips for a F’ing Great Logo

Filed in Corporate, Druid

This post originally appeared on Druid.io on July 23, 2014. Everyone wants a great logo, but it’s notoriously difficult work—prone to miscommunications, heated debates and countless revisions. Still, after three years we couldn’t put it off any longer. Druid needed a visual identity, so we partnered with the talented folks at Focus Lab for help. Our old logo (left) was...lacking. Much better now, right? Despite our fears, we cranked this out with Focus in a speedy three week sprint. Not only was the process drama-free, it was actually fun. The goal of this post is to give you some insight into how […]

Five Tips for a F’ing Great Logo
Read Post Comments

Open Source Leaders Sound Off on The Rise of the Real-Time Data Stack

Filed in Druid, Technology

In February we were honored to speak at the O’Reilly Strata conference about building a robust, flexible, and completely open source data analytics stack. If you couldn’t make it, you can watch the video here. Preparing for our talk got us thinking about all the brilliant folks working on similar problems, so we organized a panel that same night to continue the conversation. The discussion featured key contributors to several open source technologies: Andy Feng (Storm), Eric Tschetter (Druid), Jun Rao (Kafka), and Matei Zaharia (Spark). It was moderated by VentureBeat Staff Writer Jordan Novet and hosted by Zack Bogue […]

Open Source Leaders Sound Off on The Rise of the Real-Time Data Stack
Read Post Comments

How We Scaled HyperLogLog: Three Real-World Optimizations

Filed in Corporate, Druid, Technology

At Metamarkets, we specialize in converting mountains of programmatic ad data into real-time, explorable views. Because these datasets are so large and complex, we’re always looking for ways to maximize the speed and efficiency of how we deliver them to our clients.  In this post, we’re going to continue our discussion of some of the techniques we use to calculate critical metrics such as unique users and device IDs with maximum performance and accuracy. Approximation algorithms are rapidly gaining traction as the preferred way to determine the unique number of elements in high cardinality sets. In the space of cardinality […]

How We Scaled HyperLogLog: Three Real-World Optimizations
Read Post Comments

The Art of Approximating Distributions: Histograms and Quantiles at Scale

Filed in Algorithms, Data Visualization, Druid, Technology

I'd like to acknowledge Xavier Léauté for his extensive contributions (in particular, for suggesting several algorithmic improvements and work on implementation), helpful comments, and fruitful discussions.  Featured image courtesy of CERN. Many businesses care about accurately computing quantiles over their key metrics, which can pose several interesting challenges at scale. For example, many service level agreements hinge on these metrics, such as guaranteeing that 95% of queries return in < 500ms. Internet service providers routinely use burstable billing, a fact that Google famously exploited to transfer terabytes of data across the US for free. Quantile calculations just involve sorting the data, which can be […]

The Art of Approximating Distributions: Histograms and Quantiles at Scale
Read Post Comments

Real Real-Time. For Real.

Filed in Druid

Danny Yuan, Cloud System Architect at Netflix, and I recently co-presented at the Strata Conference in Santa Clara. The presentation discussed how Netflix engineers leverage Druid, Metamarkets’ open-source, distributed, real-time, analytical data store, to ingest 150,000 events per second (billions per day), equating to about 500MB/s of data at peak (terabytes per hour) while still maintaining real-time, exploratory querying capabilities. Before and after the presentation, we had some interesting chats with conference attendees. One common theme from those discussions was curiosity around the definition of "real-time" in the real world and how Netflix could possibly achieve it at those volumes. This post is […]

Real Real-Time. For Real.
Read Post Comments