why do organizations hate qualitative research?

close up of stained blood cells

Because the business world shuns uncertainty, qualitative research gets twisted so that the conclusions sound like they were deduced, and their validity unimpeachable. Business research adheres to its cousin in the laboratory, where validity is determined by empirical evidence—which is a positivistic view. But, positivism is not embraced universally in the social sciences, and it is certainly not compatible with inductive reasoning. So why do businesses automatically turn to positivism when trying to understand human behavior and reasoning?

two separate continuums - quantitative and qualitative

Both kinds of research have their “fuzzy” studies and their precise studies.

The State of Things in Academic Research

When positivism was first extended to the academic social sciences, it met with opposition. The social sciences researchers supported constructivism or relativism instead, which mean many versions of the truth can be valid at once. Through the last three decades of the 1900’s, this “Science War” raged. Dr. Karl Fast, former professor of User Experience at Kent State University in Ohio, describes an early clash—a book written by Bruno Latour, Science in Action, which chronicled field studies about the way people work in science labs. Karl says, “The studies concluded that since laboratory experiments were conducted by teams who talked about the questions to ask and how to collect the data, science was ‘socially constructed’—therefore not positivistic, itself. Social scientists were overjoyed to have scored a point against their more well-funded cousins doing research in physics, biology, etc.”

Then another landmark event happened in the late 1990’s. A pair of physicists wrote a “faux” paper and got it published in a peer-reviewed social science journal called Social Text. This paper said that in postmodernism, the idea is that our senses rely on interpretation, and the universe is something we construct from these interpretations. To postmodernists, laboratory science is about interpretation of what scientists sense, and conclusions are made through a social process. The paper the two physicists wrote was deliberately absurd, yet they made serious points, and the writing was internally consistent. Scandal, controversy, and embarrassment ensued. The authors eventually wrote a book about this event, called Fashionable Nonsense. In the end, a new theory gained support: post-positivism, which gives allowance for the fact that scientists doing empirical work can be influenced by other factors. Post-positivism holds that a single set of laws exist, but that we have imperfect knowledge of them. (See Dr. Anthony Yeong, independent researcher, Introduction to Business Research Methods.)

A summary:
  • Positivism – the only valid knowledge comes from verified, empirical evidence
  • Constructivism – each person builds their own understanding of reality, so many versions of the truth can be valid at once
  • Relativism – there is no one truth, but it depends on context and the person beholding it; people behold and interpret reality differently (See Podcast by Oliver Kim, “On Relativism and Constructivism.”)
  • Postmodernism – our senses rely on interpretation, and the universe is something we construct from these interpretations
  • Post-Positivism – a single set of laws exist, but we have imperfect knowledge of them
Qualitative Research Is Not Post-Positivistic

Cognitive empathy work falls under constructivism and relativism. (See paper by Valerie Malzer and Sarah von Schrader of Cornell University.) You gather data, which lead to patterns, which give you insights, which are probably correct, but which will have exceptions. In turn, these insights help you guide your choices toward a few better directions than what you had before. These choices can’t be proven as “right,” but have a high probability of improving different dimensions of how you support people.

Practice: Know & Teach the Difference

Now you have the explanation for why business seems to abhor qualitative research, and you can see it follows the historic tendency that natural sciences to get better funding and respect in the academic world. Social sciences tend to be the poor cousin–but not due to less rigor or validity in their work. It’s due to different underlying philosophies.

If you can clarify this difference to your peers, stakeholders, and leaders, it will help them understand. Hopefully it will pave the way to less resistance to your requests to spend time on qualitative research.

Understanding the people you support is a valuable way to improve your offerings and the role you play in the market. Split your budget between researching your solution and researching the problem space; I can help.

sign up for the newsletter; click here

2 Comments on “why do organizations hate qualitative research?

  1. Nice read.

    Perhaps some merit in exploring what stops a business wanting to uncover a deeper sense of need and truth to help get clarity/understanding? i.e. what are the social/organisational pressures to seek fast answers?

    rgds,
    Dan

  2. Great article Indi — and thanks for the opportunity for me to brush up on my academic theory!

    I agree that stakeholders, managers, executives and the like remain generally unconvinced by qualitative data, often at the expense of truly powerful insights. I have two suggestions for how to combat this tendency towards positivism:

    1. Stakeholders and leaders need the significance of qualitative data explained to them in their language. We need to put a story to the data that they can’t ignore. We could start with the fact that they’re enthralled by positivism and ask: what kinds of stories will therefore convince them?

    2. Wherever possible, back up our qualitative findings with numbers — give them the quantitative data they are so obsessed with. They so often think in big numbers. If in-depth conversations with four users won’t convince them, then take the key findings and see if you can find numbers to back them up — either by doing the research yourself, or by citing reputable research by others.

    Obviously, these things are far more complex in practise than they are in my head…mainly we just need to give them what they want (or convince them they want what we’ve got…)