What is hoped to be ‘The greatest Coaching Outcome Research ever’ is now underway, say Erik de Haan and Andy Copeland. And we need you and your clients to help us obtain the largest ever sample of coaching relationships
As our children teach us, often the most valuable questions in life are very easy to ask yet amazingly hard to answer, such as: “Why did I do that?” or “What did I mean by that?”
In executive coaching, examples of questions that are obvious and essential yet profoundly difficult to answer are:
- Does our coaching work?
- Does it help clients with their critical objectives?
- Which aspects of coaching work?
- What are the ‘active ingredients’?
- Under what circumstances do they work best?
- What intervention would work best here and now, with this client at this moment?
Thousands of coaches have asked these questions; all are curious about effectiveness or outcome. However, it is rare to encounter serious attempts at answering these in the literature with anything more than a coach’s opinion or a few carefully selected case studies.
We estimate there are probably fewer than 20 robust quantitative outcome studies in the literature, none of which satisfy the ‘double-blind randomised control trial’ standard of medicine and psychotherapy. One reason for this is the costly and cumbersome requirements of a rigorous outcome study. Another is that rather than studying, with detachment, their own effectiveness, a coach’s priority is usually to meet their coaching commitments.
However, if we don’t address these questions we may find it difficult to justify our fees; to assert unequivocally that coaching conversations are indeed beneficial and to avoid the potential risks of executive coaching, such as misjudging the situation, aggravating the status quo or abusing our influence (Berglas, 2002).
Statistical power
Contrary to other helping professions, such as counselling and psychotherapy, executive coaching is commissioned and paid for by a wide range of individual contractors, sometimes at board level, sometimes from within the HR function, and often also more locally in large corporates. This has repercussions for research. Whereas in psychotherapy most of the services are centrally commissioned by large health insurance companies or National Health services, this is not so in executive coaching.
As executive coaches we find ourselves with very little pressure to have rigorous outcome research and a dearth of funding for it. At the same time we know from psychotherapy outcome research (see the historical overview in Wampold, 2001) that we are likely to need very high sample sizes, possibly well above 10,000, and a rigorous design with randomised control trials, to demonstrate beyond doubt that executive coaching is effective – with even greater statistical power needed to differentially explore active ingredients in effectiveness. For these same reasons – no pressure from customers and no funding for research – there are as yet no rigorous randomised-control-trial studies available in the coaching literature.
Our aspirations
We believe the way to tackle this is through an ‘open source’ approach, inviting experienced coaches with an interest in doing solid research to join forces and gather high-volume data collectively. A collaborative research project between Ashridge Business School (Ashridge Centre for Coaching), VU University Amsterdam (Department of Management and Organisation) and The University of Sydney (Coaching Psychology Unit), aspires to address the dearth in reliable data in this way. The hope is to obtain the largest sample of coaching relationships in the coaching literature: many hundreds of completed online questionnaires from coaches and clients and from their organisational sponsors (line manager or director).
Building on two previously published peer-reviewed research papers (De Haan et al, 2011; 2012), this outcome research goes further than ever before. Using an innovative methodology, all key stakeholders in the coaching journey (coaches, clients and sponsors) are engaged by completing short and independently verified online questionnaires. The data from these questionnaires will hopefully provide new insights into the coaching relationship from three perspectives.
Well underway
At the moment, only five months after starting data collection and eight months before closing the ‘open source’, we have more than 950 completed questionnaires from coaching clients, plus more than 450 from their coaches (with some confidence that these will grow to a similar number), and 50 from the organisational sponsors, who are very busy people, one has to imagine.
This means that the research project is well underway to becoming the largest ever quantitative coaching outcome study: the largest study to date (Smither et al, 2003) involved 1,202 senior managers in a single large organisation, but not all of these were coached as some made up the control group.
How it works
The process for participation is as follows: first the coach invites his or her clients to participate in the research. As there is an existing relationship, this approach is effective in attaining a high response rate from clients of coaching.
Once the coaching client has completed the client questionnaire, this triggers an invitation to the coach and coaching sponsor to complete their own questionnaires.
The questionnaires themselves are not designed to demonstrate effectiveness – as this cannot be done convincingly without a proper control group. Instead, the questionnaires look into factors that may contribute to effectiveness, such as personality differences between coach and client, the client’s self-efficacy, initial expectations and the quality of the relationship as experienced by the various parties.
Inspired by the very convincing demonstrations of effectiveness in psychotherapy and some early indications in coaching research as well, we decided to look specifically into what exactly the ‘active ingredients’ are, in other words, what are the factors that might make executive coaching more effective?
By engaging with the highest number of coaches, clients and sponsors ever, this coaching outcome research hopes to break new ground in the coaching profession.
Join our research
If you are interested in becoming a part of this exciting research, please visit: www.ashridge.org.uk/centreforcoaching
You need only ask your clients to complete the client questionnaire, which will take them just 10 minutes.
As we honestly believe that larger-scale quantitative outcome research will be essential to establish this burgeoning field and to find answers to our fondest questions, we also offer the following rewards for participation:
Feedback on your average effectiveness scores as seen by your clients
A coaching book after 20 completed questionnaires
Co-authorship of the peer-reviewed article that will come out of this research
As you become part of this effort, you will get the statistical results as soon as they’re in and be among the first to know what this large-scale research programme brings to light about the active ingredients in executive coaching.
Erik de Haan is lead researcher and Andy Copeland is research analyst for this research.
References
S Berglas, ‘The very real dangers of executive coaching’, in Harvard Business Review 80(6), pp86-92, 2002
E De Haan, V Culpin and J Curd, ‘Executive coaching in practice: what determines helpfulness for clients of coaching?’, in Personnel Review, 40(1), pp24-44, 2011
E De Haan, A Duckworth, D Birch and C Jones, ‘Executive coaching outcome research: the predictive value of common factors such as relationship, personality match and self-efficacy’, in Consulting Psychology Journal: Practice and Research (in press)
J W Smither, M London, R Flautt, Y Vargas and I Kucine, ‘Can working with an executive coach improve multisource feedback ratings over time? A quasi-experimental field study’, in Personnel Psychology, 56, pp23-44, 2003
B E Wampold, The Great Psychotherapy Debate: Models, Methods and Findings. Mahwah, NJ: Lawrence Erlbaum Associates, 2001
Coaching at Work, Volume 7, Issue 4