Managing Changing Data with Dynamic Lookups

The world’s top media exchanges, buyers and publishers use Metamarkets’ interactive analytics solution to drive their business performance through intuitive access to real-time data. Having the ability to explore and visualize the data puts the power of big data into the hands of the people who need it most. However, the data is only powerful if it’s reliable. As a company evolves, so does its data and its underlying attributes. In many cases these changes are unavoidable, such as company name changing due to a merger or acquisition, account assignments or campaign name changes. Metamarkets’ dynamic lookup solution allows users to make updates to the data attributes within our tool so they can manage these changes easily.

Metamarkets Dynamic Lookup Technology

Our real-time ingestion technology lets you stream event data that is self-contained and self-describing. In cases when there are well-defined relationships between one or more data attributes, it can lead to repetition in the data and bandwidth wastage. A more efficient approach to this problem is to let users separate the description of the event stream into multiple files: an essential log file, and a supporting static lookup containing key-value data. Typically a code or ID (the key) is mapped to a longer string of characters (the value). Our ingestion uses this static lookup to expand certain values in the raw data. For example, instead of sending Metamarkets the values for the name of a publisher, it is more efficient to send a publisher ID. This allows for more compact representation of data since only the IDs need to be provided.

Once the lookup operation is done, the transformed data is stored permanently in the data store. Any changes to the mapping would require reprocessing the entire data set to ensure consistency. Since static lookup transforms and stores the data permanently, we recommend using static lookups for cases when the dimension values do not change in time.

To address the case of a fast changing lookup, we are pleased to introduce dynamic lookups that operate on the data set during retrieval of dashboard results.

Benefits of dynamic lookups include:

• Historical continuity for dimension values that change frequently without requiring reprocessing the entire data set.
• Time savings, because there is no data set reprocessing required to complete the update.
• Dynamic lookups that are kept separate from the data set. Thus, any human errors introduced in the lookup do not impact the underlying data set.
• A validation check process that ensures lookups with errors are identified and isolated early. In such cases, the system reverts back to using the most recent error-free lookup.
• The ability for users to create new dimension tables from metadata associated with a dimension table. For example, advertiser or publisher account ownerships can change during the course of a quarter. In such cases, a dynamic lookup can be updated on the fly to reflect the most current changes.

If you’d like to learn more about how dynamic lookups can be applied to your data, please download this use case overview here.

If you are interested in the dynamic lookups beta feature, please contact your account manager.