Downloading Reports with Metamarkets – Now Even Better!

Filed in Corporate, Technology

The Reporting Challenge The single biggest challenge faced by most people who work with data is, quite simply, how to get answers to their questions.  These questions may be routine and regular (how much have I spent today?) or unique and one-off (why did CTRs for campaign X just spike in Germany?). Answers are usually obtained through reports – either standardized reports generated routinely or custom reports generated as-needed to answer specific questions. Reporting sounds simple enough, but the process of generating and accessing reports can be surprisingly difficult, time-consuming, and just plain annoying. Some of the challenges include: Extracting […]

Downloading Reports with Metamarkets – Now Even Better!
Read Post Comments

How To Integrate with Metamarkets in Under 72 Hours

Filed in Corporate, Technology

Integrating a service with a new tech vendor can be a long and painful process -- but it doesn’t have to be. When potential customers are evaluating Metamarkets, we are often asked about the integration process and how long it takes. The answer is easy: we can put data at your fingertips within 72 hours. Metamarkets runs the largest Druid analytics stack in the world, enabling us to process hundreds of billions of events per day and deliver real-time results through an intuitive, point-and-click interface.  As a result, our customers can quickly and accurately understand their business through data while […]

How To Integrate with Metamarkets in Under 72 Hours
Read Post Comments

Druid Query Optimization with FIFO: Lessons from Our 5000-Core Cluster

Filed in Druid, Technology

Druid's Horizontal Scale A large strength of using Druid as a data store and aggregation engine is its ability to horizontally scale. Whenever more data is in the system, or whenever faster compute times are desired, it is simply a matter of throwing more hardware at the problem, and Druid auto-detects, and auto-balances its workloads. At Metamarkets we are currently ingesting over 3M events/ second (replicated) into our Druid cluster and have multiple hundreds of historical nodes serving this data across multiple tiers. Part of the power of this horizontal scale is how Druid breaks up data into shards. Each […]

Druid Query Optimization with FIFO: Lessons from Our 5000-Core Cluster
Read Post Comments

From the Engineering Frontlines: Rewriting Nested Table

Filed in Technology

Nested table is Facet’s main visualization. It consists of many features that provide various ways to explore the hierarchical data returned by Druid (Druid is an open source, distributed analytics data store — more about Druid here). The first version was built during Facet’s early days, back when it was still called by its internal codename. It had several flaws, the biggest one being that it was relying heavily on D3 for DOM generation without offering enough abstraction between the high levels of Facet (powered by AngularJS) and the D3 bits themselves. This had a significant impact on our ability […]

From the Engineering Frontlines: Rewriting Nested Table
Read Post Comments

Five Principles for Designing Real-Time Data Pipelines

Filed in Announcement, Technology

In the course of onboarding dozens of clients and petabytes of data to Metamarkets’ real-time solution, our Data Engineering team has learned a handful of best practices for ETL.  That’s why today we’re announcing the release of our Real-Time Integration SDK, a set of open source tools that incorporate these learnings, and make it even easier to get running with Metamarkets.  In the last month alone, it has enabled several clients to accelerate through the complex process of building a real-time data integration at petabyte scale. The SDK includes a Java client library for posting data to our real-time API, […]

Five Principles for Designing Real-Time Data Pipelines
Read Post Comments