A lawyer friend was saying that he often wondered how law ever gained the lofty status of being classed as a ‘profession’. Yes, they have to know case law to know how to make their pitch, but most are happy to accept briefs defending clients that are clearly guilty in return for their fee. But I’ll say this for them; whenever you ask them a question, they always give you a balanced answer such as ‘If you do X then the chances are this will happen, but if you do Y, then it’s likely that the other will happen’.
I respect them for their equanimity, because it’s something you rarely get from researchers and analysts. I have yet to read a piece of scientific or management research that ends with the admission that after spending all the research grant, the authors still haven’t got any explanation into the topic or that the data was inconclusive; I guess such research just never published.
The same could soon be happening in business. We’ve all sat in business meetings where various folk have presented different sets of numbers, but with the new generation of business intelligence tools and the increasing use of predictive analytics, everyone can be their own analyst and we could quickly find ourselves in meetings where folk are presenting conflicting views on exactly what is causing variance in financial or operational performance this quarter. What’s more, they will swear by the data and the rigor of their analysis. How do you prepare for such events if you’re just an Average Joe when it comes to working with numbers? Seems to me, there are only two options; either cram up on analytic techniques with a short course or a good primer – or get smart and always be the one asking the questions.
Other more qualified people than I are making the same recommendation. Thomas H. Davenport, (the President’s Distinguished Professor of Information Technology and Management at Babson College, a senior adviser to Deloitte Analytics, and the director of research at the International Institute for Analytics) is about to launch a book he co-authored calling ‘Keeping Up with the Quants’ (Harvard Business Review Press, 2013). Recognizing that we live in an era where big data prevails and when analytics are a source of competitive advantage, he suggests that companies need general managers who can partner effectively with “quants” to ensure that their work really does result in better strategic and tactical decisions. That’s right, we ourselves may be totally innumerate, but it’s time we showed our true talent by becoming questioning sceptics.
Professor Davenport argues that many analysts lack sufficient knowledge about the business to identify hypotheses and relevant variables and to know when something fundamental has happened. He suggests the job of managers and executives is to generate hypotheses and determine whether their results and recommendations actually make sense. It’s an important role and he recommends six key steps to follow when making decisions based on analytics and predictive techniques.
- Because the ‘quants’ tend to focus on the data and the analysis, Davenport recommends that non-quants should focus on the first and the last steps of the process, namely using your business experience and intuition to help formulate the hypotheses for analysts to test and then helping present and communicate the results to other executives in a compelling and credible way—something many quants ignore.
- Always review previous findings by identify people who have tried to solve this problem or similar ones—and the approaches they used.
- Formulate a detailed hypothesis about how particular variables affect the outcome and systematically test them – rather that jumping on to the first correlation that shows up in the data.
- Ensure that the analyst has used data that allows the hypotheses to be tested – rather than data that just happened to be readily available.
- Run the statistical analysis, assess its appropriateness for the data, and repeat the process until a good fit is found.
- Use the data to tell a story to decision makers and stakeholders so that they will take action.
Davenport points out that effective quantitative decisions are not just about the math and companies generally have better outcomes where quants and non-quants form deep, trusting ties that allow them to exchange information and ideas freely. It’s about being able to look over their shoulders and constantly be the one asking the questions. His favourites being:
- What was the source of your data?
- How well do the sample data represent the population?
- Does your data distribution include outliers? How did they affect the results?
- What assumptions are behind your analysis? Might certain conditions render your assumptions and your model invalid?
- Why did you decide on that particular analytical approach? What alternatives did you consider?
- How likely is it that the independent variables are actually causing the changes in the dependent variable? Might other analyses establish causality more clearly?
He recommends establishing a culture of inquiry rather than advocacy always working to seek the truth rather than data to support someone’s favourite idea. Warren Buffett once famously said, “Beware of geeks…bearing formulas.” But in today’s data-driven world, we cannot ignore our quants and need to combine their analytic skills with our business experience and sense of intuition. So become a manager who knows the geeks, understand the methodologies they use, question their analytic processes so they get better, sharpen up their presentations for them and Davenport suggests you’ll get better decisions as a result.