I base my advice to clients on three things:
- Peer-reviewed research
- Non peer-reviewed research
Too many consultants base their advice on personal experience. Personal experience is, of course, necessary and valuable but not sufficient. As the put-down goes: the plural of anecdote is not data. You must have a scientific reason to support the advice you provide.
I use standard methodologies when I manage projects. Typically this is the PMBOK framework, or Scrum for Agile projects, or the INCOSE V model for systems engineering, etc. I learned to do this by doing the opposite at the beginning of my career when I was working for one of the Big Four consulting firms. At that firm, we used proprietary methodologies. To a degree, clients hired us because of that – before the internet, best practices were not widely known and you had to hire a consulting firm to get that knowledge. But it also locked our clients into working with us – breaking up or switching vendors would be very difficult. That didn’t seem right to me so when I started Terrapin in 2003, I committed to only using standard, open methodologies. It makes it very easy for me to transition work to my clients.
It is good to base advice on standards because of the vast amount of data that surrounds the standard. The standard itself has been developed and vetted by many experts in the field. And then companies that adopt the standard produce data that is used to improve the standard. For example, PMBOK is on its fifth edition.
Using standards also helps on-board new employees or vendors. If you use PMBOK then you know for a fact that every PMP in the country (600,000+) knows how to manage projects within your organization.
Peer Reviewed Research
I use peer-reviewed research to underpin my advice too. Most peer-reviewed research is scientifically valid. The methodology and analysis are well thought-out and, by definition, it has been reviewed by others in the field. The biggest problem with peer-reviewed research in project management is there is so little of it. PM is not treated well by most universities. Economics, finance, operations, marketing and other disciplines receive massive attention and funding. I suspect there are just a handful of active researchers in PM.
I will be highlighting the most impactful research I have ever read on this blog.
Non Peer Reviewed Research
Non peer-reviewed research can be as valuable as peer reviewed. With peer-reviewed research, you know someone has vetted it. With non peer reviewed research, you have to do it yourself. So I tend to rely on research from established sources that I trust. The biggest source is Gartner – they produce excellent research. I also enjoy the annual CHAOS Report from the Standish Group.
The large consulting firms such as McKinsey, Deloitte and PWC also produce research. Their research tends to be survey-based. They usually survey their clients, Fortune 1000 companies, so the results are skewed towards big company topics. But the results are beneficial and can be useful when understanding the opinions of CIOs in certain areas.