Blog

Erick Burchfield, managing director of research

It’s exciting to think of data and analytics as a harbinger of revelatory big new insights, ushering in a new era of data-optimized, predictive, automated decision-making to address big, ugly problems. But data analysis can also empower us in a perhaps less sensational but equally important way: by debunking the conventional wisdom that often drives our decision-making.

Some time ago I worked for an economic development and policy consulting firm that served Native American Tribal Nations. Our work involved fighting government resistance to the establishment of Tribal casinos. As is often the case in policy debates, public opposition to gaming was anchored not on data analysis but on arguments that appealed to “common sense” or “prevailing wisdom.” These arguments often reflected feelings and beliefs, not facts. For example, one “public harm” argument in opposition of Tribal gaming was premised on the following reasoning: “Property values fall in areas near casinos, reducing assessments and property tax collections and therefore harming the State.”

On its face, the argument has intuitive appeal. Casinos can certainly create social problems, including crime, addiction, and homelessness. Those social problems could logically be assumed to radiate to surrounding communities, reducing the value of homes in the area. Absent data to contradict the assertion, such reasoning becomes the path of least resistance for NIMBYism – it’s easy to think this must be true, despite the fact that there’s no evidence to prove it.

In the case above, we made it our mission to put the government’s property tax hypothesis to the data test. As researchers, we rejected intuition in favor of quantitative validation. We doggedly pursued state tax assessment and collection records, mapped casino opening dates to properties within five and ten miles, and compared time series’ of assessments and tax collections before and after casino openings using fairly complex regression models.

Our analysis revealed no relationship between casino openings and property values or tax collections. Whether a given property was within five or within ten miles of a new casino, assessed property values and tax collections were unaffected by the presence of a casino, or even multiple casinos. The intuitive but untested appeal of the government opposition didn’t withstand analysis. Ultimately, the state dropped the argument.

I learned an important lesson: a valuable but underappreciated role of data analysis is its potential to debunk that which we anchor on as “temptingly true” – either because we want to believe it (i.e., confirmation bias) or because it’s easier to believe it (i.e., it simplifies decisions). Not unlike politics, in professional services industries, instinct and prevailing wisdom still drive much decision-making. That’s not surprising in industries that so highly value the intellect and judgment of the individual professional. But heuristic decision-making not validated by data can be dangerously misleading and/or significantly over-simplified.

As a researcher at ALM, I’m commonly confronted with broad-based and widely-held “industry truths” based on anecdotal foundations, conventional wisdom, or idiosyncratic personal experience. “Most lateral partner hires lose money;” “Most law firm mergers fail;” “Growth in big law is hard to come by post-downturn.”

Some of these assertions may have merit. Some have been subjected to analytical study (often in survey form). But they remain in large part untested – with outstanding questions that only deep data analysis can answer authoritatively. In its absence, all-too-dangerous prevailing wisdom fills the void.

We seek to confront this conventional thinking through rigorous data analysis – analysis that leads to insights that challenge prevailing beliefs, rules of thumb, and simplified decision-making short-cuts. When unable to act based on data, we regress to acting on imperfect and often inaccurate instincts, anecdotal experience, and what feels like common sense. One of the most powerful lessons I’ve learned as an analyst is to avoid the temptation to accept the given and to test traditional thinking by looking through a data lens. You may be surprised by what you discover.