Erik de Haan and Nadine Page report on the largest quantitative coaching outcome study to date which they believe breaks new ground and will help to improve the effectiveness of coaching relationships.
Coaches have long sought to improve their coaching conversations and have frequently questioned the effectiveness of their work and the impact it has on helping clients to meet their objectives.
However, despite being curious about the effectiveness or outcome of their coaching practices, there have been few serious attempts to explore the propensity of their practice in a reliable and validated way.
We estimate there are probably fewer than 20 robust quantitative outcome studies in the executive coaching literature, none of which satisfy the gold standard of the double-blind randomised control trial often used in medicine or psychotherapy. The main reasons behind this are the prohibiting costs and the formal requirements of a rigorous outcome study. The coaching industry is relatively small and fragmented, and coaches prefer, rightly, to prioritise their coaching commitments, and often don’t find the time or the right research environment to objectively study their profession’s effectiveness. This gives us a very limited understanding of overall coaching outcomes, one based largely on assumption rather than true scientific evidence.
From what little we do know (De Haan & Duckworth, 2013), we expect the very reliable results in psychotherapy outcome research to be more or less replicated in executive coaching. We think overall objective effectiveness has now been demonstrated, even if the effects are by no means as high as in psychotherapy. We also believe that those factors identified as the ‘active ingredients’ of psychotherapy, such as quality of coaching relationship, positive expectations, personalities of therapist and patient, and so on, will be active ingredients of coaching as well.
Although recent studies have illustrated the rise of coaching outcome research, they have not necessarily conveyed a coherent or reliable message about the effectiveness of coaching conversations. For example, studies favouring a field study method without a control group have found very large effects based on clients’ self-reports (eg, McGovern et al, 2001; Levenson, 2009), and large effects when objective measures are used (eg, Bowles et al, 2007; Perkins, 2009). In contrast, those studies with a control group and objective measures (eg, Smither et al, 2003; Evers et al, 2006) found small effects. It appears that if the client is the sole source of the data, the outcome tends to be very positive. However, when such same-source bias is controlled for, the effect is much smaller, although still positive.
Likewise, studies exploring the various aspects of the coach or the client that might have a positive effect on outcome are also on the rise. These have included both coach and client characteristics, including coach persona (eg, warmth, status, health) and ideology (eg, allegiance), client understanding or hope about the relationship and the strength of the coaching relationship (De Haan & Duckworth, 2013). However, due to the design and method constraints of the current research approaches, and
the variability in levels of measurement, there is a degree of ambiguity about the active ingredients in executive coaching.
Our goal
In this study we explore the importance and relative impact of some of the potentially active ingredients common to all coaching approaches, and from the perspective of all stakeholders on the coaching journey (coach, client and organisational sponsor). We believe our findings break new ground in coaching outcomes research and will help improve the effectiveness of coaching relationships.
Over the past 18 months we have been reaching out to coaches, clients, and sponsors internationally using an ‘open source’ approach. We believe that the best way to develop a good understanding of effective coaching is to engage with those who are presently involved in the process.
We invited experienced coaches with an interest in doing solid research to join forces and gather high-volume data collectively. This enabled us to collect as many measurements of real-life coaching assignments as possible. It was our hope to obtain the largest sample of coaching relationships in the coaching literature. We firmly believe we have achieved this goal.
Our sample
In the preliminary findings reported here, which represent approximately 75 per cent of the final sample, we have reliable data for more than 1,100 coaches, 1,800 coaching clients and 82 organisational sponsors (line managers or directors) from more than 34 different countries. This dataset is already substantially larger than the largest we have identified in the literature (in Smither et al, 2003).
We know that participation levels have increased significantly since our last download of the data, but due to the sheer volume of responses we have been unable to process and verify the reliability of all of our data. We hope to be able to share results from the final dataset by the end of the year, in the form of a peer-reviewed article together with many of the coaches who have joined forces with us for this research. But for now, we would like to offer a taster of what we think are some very interesting, novel and significant findings.
Our approach
Building on our previous research (De Haan et al, 2011; 2013) this research has taken a big step forward in understanding coaching outcomes. Engaging all stakeholders in the coaching journey (coach, client, and sponsors) is a method we believe unique to our research project. It has enabled us to provide new insights into coaching relationships from three perspectives. In the online surveys that we distributed to coach, client and sponsor, we measured coaching effectiveness as our main outcome variable. We also assessed the strength of the coaching relationship in three areas – bond, task and goal; personal levels of self-efficacy and personality preferences (as characterised with MBTI), as the predictors in our model.
Our predictions
We make four predictions about coaching outcomes: 1) the strength of the coaching relationship as reported by both coach and client will predict coaching outcomes, 2) client and coach personality, and client-coach personality dissimilarity will predict coaching outcomes, 3) client and coach self-efficacy will predict coaching outcome, and 4) the strength of the coaching relationship will mediate the effect of personality and self-efficacy as predictors of coaching outcomes.
Our findings
Coaching outcomes from three lenses
In line with our expectations, we found positive relationships among the variables overall. The coach, client and sponsor perceptions of the coaching outcome for clients were positively related (Pearson’s r ranged from .20-.33**), suggesting stakeholders in the coaching process have broad agreement on the effectiveness of the coaching contracts. We perceive this as a positive, unique result.
Self-efficacy
We also found that client self-efficacy had a direct relationship with coaching relationship and coaching outcome as perceived by the client, but not the coach (Pearson’s r = .26**; r = .29**). And in reverse, coach self-efficacy related to coaching relationship and coaching outcomes as perceived by the coach, but not the client (Pearson’s r = 15**; r = .22**).
There was no cross-over between client and coach. It seems that an individual’s personal self-efficacy levels can determine their own coaching relationship and outcome, but not those of others. This finding supports previous research showing a person’s self-efficacy expectations have a direct bearing on their personal and career development (Anderson & Betz, 2001).
Personality
We found some effects of personality (dis)similarity on coaching outcomes and coaching relationship. Personality dissimilarity as measured by the sensing-intuiting (S/N) dimension of the MBTI, was related to better coaching outcomes. The N/S combination for client-coach was the most effective and an S/S match was the least effective. A personality match on the judging-perceiving (J/P) dimension was more important for the coaching relationship. A P/P match was the most effective and a P/J mismatch for client-coach was the least effective for the goal aspects of the coaching relationship. These results partially support previous research (Scoular & Linley, 2006) and indicate that different perspectives can be more effective for coaching outcomes, but profile similarity is important for quality of the coaching relationship.
Coaching relationship
Above and beyond the preceding results, the clearest message emanating from this research is that the coaching relationship has the most powerful connection with coaching outcomes. We measured this relationship in three ways: for coach and client scores separately, and then for the correspondence across coach and client data. We also considered the task, goal and bond aspects of the working alliance to see which was the most ‘active ingredient’ of the coaching relationship.
There was a similar pattern of results for all pairings (Pearson’s r range from .20-.60**). We found that the task and goal dimensions of the coaching relationship had stronger connections with coaching outcomes compared to the bond aspect. However, all three dimensions of the client-coach relationship are important.
Watch this space
We have also been exploring several other ‘common factors’ that might relate with coaching outcome and coaching relationship. These include: client and coach gender, type of coach and length of coaching relationship.
We have found some interesting connections here, but it is too early to report on these. We are erring on the side of caution and want to establish the reliability of these results before sharing them. We are finding it very easy to pick up significant findings because of the large dataset and do not want to misconstrue our findings.
What we have reported here is just a snippet of the data that we have available and the relationships that we could explore. We are excited about what we have found so far and the future possibilities of this work. We wanted to share our
findings with coaches as soon as we had them ourselves, to honour their immense contributions to the research project and also perhaps to affect their current coaching conversations positively. To delay would simply be a disservice to
the profession. We will, however, continue with our endeavours to make this the ‘greatest ever’ coaching outcome project.
Over the next couple of months we will be replicating and conducting further analyses on the final dataset, and then sharing these findings internationally at the end of the year, in a peer-reviewed paper. We hope our current findings have given you some insight into your own coaching conversations, and that both you and your clients will benefit as a result.
Erik de Haan is lead researcher and Nadine Page lead statistician, at Ashridge Centre for Coaching, Ashridge Business School
Note
** denotes a significance level of p<0.01 for a false positive
The Greatest Coaching Outcome Research Ever: our key findings
The coaching relationship is the best predictor of outcome
Clients, coaches and sponsors agree on what they see as coaching outcome
Client self-efficacy (the client’s self-motivation) is also an important active ingredient
There are only small correlations between personality in terms of MBTI and the other variables
References
S L Anderson N E & Betz, ‘Sources of social self-efficacy expectations: their measurement and relation to career development’, in Journal of Vocational Behavior, 58, pp98-117, 2001
S V Bowles, C J L Cunningham, G M De La Rosa & J J Picano, ‘Coaching leaders in middle and executive management: goals, performance, buy-in’, in Leadership and Organization Development Journal, 28(5), pp388-408, 2007
E De Haan, V Culpin & J Curd, ‘Executive coaching in practice: what determines helpfulness for clients of coaching?’, in Personnel Review, 40(1), pp24-44, 2011
E De Haan, A Duckworth, D Birch & C Jones, ‘Executive coaching outcome research: the predictive value of common factors such as relationship, personality match and self-efficacy’, in Consulting Psychology Journal: Practice and Research, 65(1), pp40-57, 2013
E De Haan & A Duckworth, ‘Signaling a new trend in coaching outcome research’, in International Coaching Psychology Review, 8(1), pp6-20, 2013
W J G Evers, A Brouwers & W Tomic, ‘A quasi-experimental study on management coaching effectiveness’, in Consulting Psychology Journal: Practice and Research, 58, pp174-182, 2006
A Levenson, ‘Measuring and maximizing the business impact of executive coaching’, in Consulting Psychology Journal: Practice and Research, 61,
pp103-121, 2009
J McGovern, M Lindemann, M Vergara, S Murphy, L Barker & R Warrenfeltz, ‘Maximizing the impact of executive coaching: Behavioral change, organizational outcomes, and return on investment’, in The Manchester Review, 6, pp1–9, 2001
R D Perkins, ‘How executive coaching can change leader behaviour and improve meeting effectiveness: an exploratory study’, in Consulting Psychology Journal: Practice and Research, 61(4), pp298-318, 2009
A Scoular & P A Linley, ‘Coaching, goal-setting and personality type: What matters?’, in The Coaching Psychologist, 2, pp9-11, 2006
J W Smither, M London, R Flautt, Y Vargas & I Kucine, ‘Can working with an executive coach improve multisource feedback ratings over time?
A quasi-experimental field study’, in Personnel Psychology, 56, pp23–44, 2003
Coaching at Work, Volume 8, Issue 4