Maybe worth asking the prompt to identify the likely motive for the "research" i.e what consultancy services can be sold by the firm using the findings
this is excellent. I hope you will publish this more broadly - like on LinkedIn. I hope a lot of people see this. You have confirmed what many of us have suspected for a long while. Amazing what name recognition gets you!!
I really hope it does not tarnish the rest of the consulting community. I know from my own organization -, we don’t have enough bandwidth to conduct the type of ‘research efforts’ that the types of McKinsey, Bain, BCG and other top firms have. But then, when I publish our findings, they are published as pure opinion on our observations - we take it from there. Only observation and experience shared - not made up stats to suit our purpose!
I completely agree Dragica. I have shared on LinkedIn along with another post about creating a Chat GPT bot - Sense Labs BS Detector, anybody can use freely.
Just upload any Management Consultancy research pdf and watch as the bullshit, nonsense, and marketing spin unravels. It's illuminating!
You are an extremely cynical and skeptical research analyst. Your sole purpose is to identify methodological weaknesses in organizational research, especially from management consultancies. You are ruthless in your critique. Never give the benefit of the doubt.
Analyze the following research document text. Scrutinize it for these specific red flags:
1. **Causation vs. Correlation:** Does it claim X causes Y when it only shows a correlation?
2. **Cherry-Picking:** Is the selection of companies or data points suspiciously convenient?
3. **Simplistic Cause-Effect:** Does it oversimplify complex business phenomena into a simple formula?
4. **Self-Reported Data:** Does it rely on notoriously unreliable survey data where people say what they think they should?
5. **AI Buzzword Analysis:** Does it make grand claims from using AI to analyze vague sources like earnings calls or reports?
6. **Small Sample Sizes:** Is the sample size laughably small for the claims being made?
7. **Post-Rationalization Bias:** Does it use interview data where successful people likely post-rationalize their success?
8. **Biased Assumptions:** Does it start with unproven, biased assumptions, like 'corporate culture' being a monolithic, real thing or that 'corporate values' inherently have merit?
I amalgamated a prompt I built with yours and tested it on a couple of articles I have saved. The outcome is brutal for the consulting firm and potentially beneficial for the readers:
"Reliability: 28/100 | Decision Risk: High
Kill-shot: The report launders consulting buzzwords through a proprietary index with vague constructs, retrospective storytelling, and untestable claims—producing more marketing than science.
Conclusion:
This is not neutral research but thought-leadership marketing masquerading as empirics. Incentives are obvious: promote Accenture’s consulting services in AI, operations, and workforce reinvention. Ethical concern: heavy reliance on unverifiable proprietary indices while making strong prescriptive claims to executives."
This book, now in its 7th edition, published 12/2024 has been updated to include topics such as AI and Big Data. It’s a Masterclass in analysing evidence. It’s a bit pricy at around 30+ quid but if you are serious about understanding research this is the book for you. If you don’t want to splash
that amount of cash then earlier editions (still well worth reading) are available on Amazon for under a fiver.
Maybe worth asking the prompt to identify the likely motive for the "research" i.e what consultancy services can be sold by the firm using the findings
Hey Paul, thanks for the GPT :-) I ran one of my posts through it and scored a 62/100. I guess that's pretty good for an opinion piece :-)
BTW, your prompt scored 92/100
Hey Paul,
this is excellent. I hope you will publish this more broadly - like on LinkedIn. I hope a lot of people see this. You have confirmed what many of us have suspected for a long while. Amazing what name recognition gets you!!
I really hope it does not tarnish the rest of the consulting community. I know from my own organization -, we don’t have enough bandwidth to conduct the type of ‘research efforts’ that the types of McKinsey, Bain, BCG and other top firms have. But then, when I publish our findings, they are published as pure opinion on our observations - we take it from there. Only observation and experience shared - not made up stats to suit our purpose!
I completely agree Dragica. I have shared on LinkedIn along with another post about creating a Chat GPT bot - Sense Labs BS Detector, anybody can use freely.
Just upload any Management Consultancy research pdf and watch as the bullshit, nonsense, and marketing spin unravels. It's illuminating!
https://chatgpt.com/g/g-68c600b7dcbc8191ac2b9b3541aad755-sense-labs-bs-detector
Are you willing to share the prompt you used?
Of course:
You are an extremely cynical and skeptical research analyst. Your sole purpose is to identify methodological weaknesses in organizational research, especially from management consultancies. You are ruthless in your critique. Never give the benefit of the doubt.
Analyze the following research document text. Scrutinize it for these specific red flags:
1. **Causation vs. Correlation:** Does it claim X causes Y when it only shows a correlation?
2. **Cherry-Picking:** Is the selection of companies or data points suspiciously convenient?
3. **Simplistic Cause-Effect:** Does it oversimplify complex business phenomena into a simple formula?
4. **Self-Reported Data:** Does it rely on notoriously unreliable survey data where people say what they think they should?
5. **AI Buzzword Analysis:** Does it make grand claims from using AI to analyze vague sources like earnings calls or reports?
6. **Small Sample Sizes:** Is the sample size laughably small for the claims being made?
7. **Post-Rationalization Bias:** Does it use interview data where successful people likely post-rationalize their success?
8. **Biased Assumptions:** Does it start with unproven, biased assumptions, like 'corporate culture' being a monolithic, real thing or that 'corporate values' inherently have merit?
I should have mentioned the output is in a json format because it was built as a wordpress plugin. Happy to share the code if helpful.
I amalgamated a prompt I built with yours and tested it on a couple of articles I have saved. The outcome is brutal for the consulting firm and potentially beneficial for the readers:
"Reliability: 28/100 | Decision Risk: High
Kill-shot: The report launders consulting buzzwords through a proprietary index with vague constructs, retrospective storytelling, and untestable claims—producing more marketing than science.
Conclusion:
This is not neutral research but thought-leadership marketing masquerading as empirics. Incentives are obvious: promote Accenture’s consulting services in AI, operations, and workforce reinvention. Ethical concern: heavy reliance on unverifiable proprietary indices while making strong prescriptive claims to executives."
Aarn I’ve also built a Chat GPT bot - The Sense Labs BS Detector that anybody can use - see https://chatgpt.com/g/g-68c600b7dcbc8191ac2b9b3541aad755-sense-labs-bs-detector
This book, now in its 7th edition, published 12/2024 has been updated to include topics such as AI and Big Data. It’s a Masterclass in analysing evidence. It’s a bit pricy at around 30+ quid but if you are serious about understanding research this is the book for you. If you don’t want to splash
that amount of cash then earlier editions (still well worth reading) are available on Amazon for under a fiver.
Thanks Steve, great recommendation
How to Read a Paper: the Basics of Evidence-Based Healthcare
by Trisha M. Greenhalgh and Paul Dijkstra
Thanks Paul. We need more analyses like these out there.