Defence against the Dark Art of Management Consultancy "Research"

Work sucks for millions of us and we are primarily in this unenviable workplace situation due to a spectacular lack of critical thinking. However, most of the misguided elements of our work-life belief systems crumble under the slightest glance of critical review.

Photo by steffen wienberg on Unsplash

Belief perseverance can’t be cured by critical thinking alone, but the ability and willingness to question accepted wisdom has a place along the path to more meaningful change. This is particularly so in the corporate version of ‘defence against the dark arts¹’, i.e., defending against pseudoscience research published by management consultants and others.

Unlike the research published by career academics, research from consulting firms is typically not peer-reviewed or published in respected journals. Careers are not at risk for spurious claims; the opposite seems true.

This research is often completely biased, entirely conflicted and poorly constructed. Every senior manager and executive should be trained in critically reviewing this kind of marketing dressed up as research instead of just unquestioningly accepting it and using it to justify spurious initiatives (no matter how well-intended). At a very minimum, any research that’s being cited to justify an investment or intervention should go through a basic set of questions:

1. Who did the research? Academics or a bunch of junior management consultants? If it were academics, what other papers have they published? How many times have they been cited?

2. Was the research peer-reviewed and published in a respected independent publication?

3. Is there evidence of bias in the starting assumptions? For example, research papers starting with a sentence like “Culture is the primary determinant of performance…” or “Values are the moral code that shapes behaviour…”, other biased and silly entry points should be ignored entirely.

4. What methodology was deployed? Hint: Any methodology that includes analysing earnings calls or annual reports, relying on self-reported data, cherry-picking organisations to include, etc., should be immediately treated with suspicion.

5. Is there any evidence that indicates causation and not just correlation? Would the reverse hypothesis be just as valid a conclusion? For example, does higher employee engagement make companies successful, or could working for already successful companies make higher engagement more likely?

6. In a complex world, how likely is it that one variable can be isolated from the multitude of others?

7. Who stands to gain from this research? Where are the conflicts of interest? Who funded the study? What are we being sold?

8. Does other published independent research support or contradict these findings?

More broadly, CEOs and executives should mandate that any proposed HR-related investment based on ‘best practice’ or ‘accepted wisdom’ will be examined for evidence that the proposed approach is worth testing.

In the unlikely (but highly desirable) situation that an employee proposes an entirely novel and untested practice, CEOs should briefly pause to celebrate an original idea mysteriously appearing in a sea of conformity. Then, they should challenge their teams to construct a series of small experiments to test the idea, maybe even in partnership with an academic whose area of interest converges. That would be something worth celebrating, regardless of the outcomes.

To support and assess levels of critical thinking, there are a number of helpful academic models of cognitive processes. The best-known of these is Bloom’s Taxonomy (revised 2001)². In this model, cognitive processes are arranged and described in order, starting with ‘remember’ and progressing to ‘create’. The six levels are:

1. Remember (recall facts and basic concepts)

2. Understand (explain ideas or concepts)

3. Apply (use information in new situations)

4. Analyse (draw connections among ideas)

5. Evaluate (justify a stand or decision)

6. Create (produce new or original work)

Most teams I have encountered in organisations operate at levels 1-3. Ways to encourage better thinking at the team level include pre-mortems (a session where you imagine the project has already failed, and try and figure out why), planned disagreements (setting the expectations of finding reasons to argue against a course of action), and the combination of idea generation and idea evaluation into team meeting agendas.

Do your own research to seek answers to questions such as:

• What can I learn about the exceptions, the mavericks, the ones who’ve abandoned ‘best practice’ in the search for original practice?

• How can I surface and challenge my biases and most deeply held beliefs about life at work?

• What’s the latest thinking emerging from diverse fields such as neuroscience, behavioural economics, social science, psychology, technology, etc., and what might some of the implications be?

A word of warning: once you start on this path, it may be hard to stop. But that’s no bad thing.

1. For any Harry Potter fans

2. Armstrong, P. (2010). Bloom’s Taxonomy. Vanderbilt University Centre for Teaching