Data Scientist Interview: Amy Heineike, Director of Mathematics at Quid

Amy Heineike is the Director of Mathematics at Quid, where she has been since its inception, prototyping and launching the company’s technology for analyzing document sets. She studied mathematics at Cambridge University. She previously worked at Volterra Consulting, an economics consultancy in London. Above is a Quid cluster map showing the connections across sectors for technology companies in 2010.

Metamarkets: Could you give an overview of what Quid does and what your role is there as the Director of Mathematics?

Amy Heineike: At Quid, we build software that helps the world’s most advanced organizations answer the billion dollar question, “What should we do?”  We use natural language and machine learning models, under a visual presentation layer, in order to allow them to make sense of the complex world surrounding the organization.

Where I fit in is figuring out what the analytical details of the products we want to build are – that means everything from defining new data types we want to collect, to prototyping analytical models, to speccing and taking new products into production.

Metamarkets: As much as anyone can actually be said to have a typical day, what is your typical day like at Quid?

Amy: I spend quite a bit of my time writing code.  This is very fun code to write because it normally involves starting with some dataset – say documents about ‘mobile payments’ – and piping it through some analyses and exploring what comes out the other end, and then trying again with some other dataset, and seeing what the process did on that set, and then iterating again on the methodology.  I’ll also try out slicing the data in different ways to figure out what else you could squeeze from it, and figure out how to incorporate those views into our visualization tools.  It’s a buzz as you hone in to something that not only works again and again, but lets you discover interesting facts you didn’t know about which ever topic you’re looking at.

After this I spend time a lot of time talking with the team. I spend time with our sales and client facing team to understand the questions that clients have, and how they’ve been responding to the software.  I spend time with engineers debating how to plug different analyses into the production stack, or understanding what improvements are being made that I can leverage.  I spend time with our product managers developing roadmaps for the roll out of new products. Sometimes I meet with our clients directly to get more detailed feedback on what we’re producing.

Metamarkets: What is a client of Quid like? What kind of information are they looking for?

Amy: A typical client would be the strategy arm of a large corporation or a governmental agency that wants to understand what’s happening in an area of innovation.  They’d use the software to discover emerging technologies, new companies that might become potential partners or targets, or explore their what’s happening outside of their organization.

The normal approach to answering those kinds of questions relies on asking experts for opinions, doing a lot of googling and phoning around, and building analyses in excel and powerpoint.  Quid’s software lets them get to higher quality answers faster. 

Metamarkets: Does Quid have a team of data scientists that you oversee, or what’s the structure like?

Amy: So Quid, in a way, is a pure data science company. The company is focused on transforming data into insight.  We have people at the engineering end of the spectrum who focus on building the infrastructure to enable everything to happen, and people at the client facing end, who help our users explore that insight – in between are the algorithm and product people who are figuring out how to bring it together, and that’s where I sit.  So we have this whole selection of skills that all touch on the big themes in data science.  In a world where increasingly the most important information about your organization might be created or tracked outside of it, this kind of model is going to become much more common.


Metamarkets: What is your own background? What did you study in school?

Amy: I studied Mathematics at Cambridge University in the UK.  While I was there, I particularly enjoyed studying non-linear dynamics and network theory – these are methods that let you examine complex systems, and they got me fascinated with the question of how we should think about complex human systems in particular.

I worked for a number of years in London with some behavioral economists who were looking at new methodologies for answering big public policy questions, including network analyses and simulation modeling. One of the directors, Paul Ormerod, just released a new book called “Positive Linking” which explores the implications for how we understand the economy as we realize that interactions between people are a big driver for economic actions, building on some work we collaborated on together.  We also did some work on, for instance, how cities evolve, and what that means for transportation policy.

What’s really interesting right now is that there’s a growing understanding that a lot of standard economic models that have been used extensively for making policy decisions are very flawed.  They assume a uniformity and equilibrium that is very far from our experience of the world. There’s still a lot of work to figure out how you replace that, or how you evolve that. And what got me into Silicon Valley was wanting to see what data is available to support some of these analyses. We were often a bit hamstrung by the fact that we were relying on a lot of aggregated and high level governmental data. It came out very infrequently, and geographical boundaries of the data changed, so it was very hard to do time series analysis with them.

I think it’s exciting now, both seeing the explosion of data sets that are available, but also seeing the development of methodologies that make that data useful – especially for unstructured and natural language content.

Metamarkets: What are some challenges that you face at Quid that you’d say are unique to your work there?

Amy: One of our core ideas is that our users can use unstructured data to help them explore complex questions.  There aren’t many tools out there for this already, which means we have a lot of tough questions on both the UI and the analytic side to make that happen.  Most of the tools for unstructured data give you a list – whether that’s search results or recommendations or news feeds – and we want to give you a map instead.  That means that coverage (how much data did we surface) and visual texture (did we show the right amount of information so the structure is exposed usefully) are critical, but they can be very hard to measure and quantify.  On the UI side, we use network visualization extensively to organize the data, but there are relatively few examples of consumer tools that use this approach to build on.  So our challenge is to make the user experience feel very intuitive for people who are used to Google and Excel.

What’s exciting about this is that once you get into the tool, it’s hard to imagine not having it any more.  How else would I review every news article and major blog post about synthetic biology from the last 2 years?  It’s got to be in Quid.

Metamarkets: Where do you see the field of data science heading in the next decade? Is data scientist a career path that’s going to become a lot more coveted?

Amy: Firstly, the tools available are going to get better and better.  It’s amazing to see the progress in the last four years even. There are ever more rich datasets available behind easy to use APIs, companies that help you spin up and manage computational clusters, software libraries, blogs and tutorials that make it fast to prototype and experiment in languages like R and Python.  There’s a lot further for these to go, but the progress is all in the right direction.

What that means is that it’ll get easier to do more things with more data.   As that happens, we’ll hopefully see much more interesting and transformative things being built.  The big advances won’t come from a black box though, they’ll come from incredibly curious people who make the creative leaps forward bridging what is possible with what is worth doing. There is currently a lot of scope for big advances to be made, and so the demand for people who can fill those roles and the people who want to fill them is going to increase.

Metamarkets: What’s a current project that you’re excited about?

Amy: I’ve been focused on optimizing how we analyze news recently, and that is very exciting.  News is fascinating because it reflects both what is happening, and also the direction of the conversation that is emerging about that – which makes it a very rich dataset for strategic analysis.

By mapping out all of the news that matches your search term, rather than just a list of the top matches, you get a bird’s eye view of what’s been happening, and from there start slicing and exploring the data to dig into what that means.  We did a really interesting piece for NPR recently in which we explored content about “Occupy Wall Street”.  We could show visually how the conversation in New York, which started around bank bailouts, differed from that in Oakland, which focused on the police, and from everything in DC, which centered on tax policy.  We could jump from this hard-to-define concept into views of the main themes in the conversation and how they emerged over time, to comparing the coverage of different sources, to finding the most shared articles, and clusters of stories that somehow never made it big.   It’s exhilarating to consume the combined content of 20,000 articles in just a few hours.