There’s value in mixing it up when it comes to research into mentoring effectiveness within universities, argue Joanna Molyn and Judie Gannon

More and more universities are investing in mentoring schemes (Cornelius, Wood and Lai, 2016), but the question is whether the schemes deliver what these universities hope to achieve.

The literature has widely explored the benefits of mentoring, but as universities are facing increasing economic, political and environmental pressures to improve students’ employability outcomes it is imperative that such mentoring programmes are carefully executed and assessed for their effectiveness.

Mentoring schemes can help bridge the gap between students from the UK’s respected Russell Group universities and widening participation students as, currently, the choices of vocational professions and employability opportunities offered to students differ vastly. The former often take up positions in the well-paid professions whereas the latter study more vocational courses and struggle to find graduate employment (Molyn, 2018).

Mentoring research in higher education

Mentoring relationships and interventions are notoriously difficult to evaluate (DuBois et al, 2011). Law, Hales and Busenbark (2020) discuss poor-design issues in mentoring research due to:

• A small number of experimental or quasi-experimental designs
• The absence of control groups
• A lack of an operational definition of mentoring that would allow replication
• A failure to test or report the validity of survey items
• Self-reported outcome measures
• Only one-time point in data collection
• Over-reliance on descriptive methods as the main analysis
• Issues with demonstrating how the sample was representative of study population
• A lack of theoretical frameworks

The absence of theoretical or conceptual frameworks seems to be another major limitation in outcome-focused mentoring research. This is particularly relevant for those commissioning and developing mentoring interventions with specific outcomes in mind. In the higher education sector, where students are recruited under a widening participation agenda, the issues of social capital and socioeconomic status have to be seriously considered by universities in order for mentoring schemes to address effectively barriers to students’ chosen careers (Molyn, 2018).

Mentoring schemes need to leverage social capital and social networks (Gannon and Maher, 2012) to surmount issues around students’ social class (Mekolichick and Gibbs, 2012) and address students’ cultural and ethnic backgrounds and their socioeconomic status (Molyn, 2018).

A quasi-experimental mixed-method longitudinal research into the effectiveness of a mentoring scheme

Molyn (2018) conducted a study which examined the link between mentoring, self-efficacy and the employability efforts of undergraduate students from a university with a strong widening participation agenda based in the UK, between 2014 and 2016. The experimental group received six mentoring sessions over a period of six months from professionals working in corporations in London. The control group received no mentoring.

The research design attempted to overcome the previously mentioned methodological issues. It was a quasi-experimental, mixed-method, longitudinal design with large sample sizes: N=955 students at Time 1; N=245 students at Time 2; N=15 students, N=9 mentors and N=3 senior managers at Time 3 (interviews). The quantitative part used established scales, with Cronbach’s alphas of .83 and above.

The qualitative part explored students’ self-efficacy beliefs and employability efforts using Social Cognitive Career Theory as its framework. This theory incorporates factors such as gender, ethnicity, socioeconomic status, cultural and gender role models and environmental factors (Lent, Brown and Hackett, 1994).

Results snapshot

Despite the careful research design the study didn’t find evidence that the mentored students outperformed the control group in terms of increased career decision self-efficacy or employability efforts. Perhaps further longitudinal randomised controlled trial studies with many data points in time are needed in order to capture the effectiveness of any intervention, as it was done in Molyn’s most recent study (de Haan, 2018).

However, the qualitative data offered the richness of insights that wouldn’t otherwise have been attainable with only quantitative data. It identified many benefits of mentoring as well as many barriers to students’ chosen careers.

Mixing it up

Using mixed methods created the overall picture that was “more than the sum of its parts” (Moffatt et al., 2006:7). Combining both types of data resulted in the illumination of the findings and in conclusions that would have been unattainable if only one method was used. For example, although quantitative analyses showed that ethnicity mediated students’ self-efficacy, outcome expectations and employability efforts, the qualitative data did not capture these points, despite many accounts of perceived ethnic discrimination.

On the other hand, qualitative data showed that students’ employability efforts were influenced by socioeconomic status, cultural and gender role models, and absence of social networks. Overall, Molyn’s (2018) study clearly highlights the value of mixed method research in helping to tackle the complexity of evaluating mentoring interventions and shows the synergistic value of mixed methods, particularly when results diverge or negate each other.

References

• V Cornelius, L Wood and J Lai, ‘Implementation and evaluation of a formal academic-peer-mentoring programme in higher education’, in Active Learning in Higher Education, 17(3), 193-205, 2016
• D L DuBois, N Portillo, J E Rhodes, N Silverthorn and J C Valentine, ‘How effective are mentoring programs for youth? A systematic assessment of the evidence’, in Psychological Science in the Public Interest, 12(2), 57-91, 2011
• J M Gannon and A Maher, ‘Developing tomorrow’s talent: the case of an undergraduate mentoring programme’, in Education & Training, 54(6), 440-455, 2012
• E de Haan, ‘Key influencer’, in Coaching at Work, November/December, 13(6), 33-35, 2018
• D D Law, K Hales and D Busenbark, ‘Student success: A literature review of faculty to student mentoring’, in Journal on Empowering Teaching Excellence, 4(1), Article 6, 2020
• R W Lent, D S Brown, and G Hackett, ‘Toward a unifying social cognitive theory of career and academic interest, choice, and performance’, in Journal of Vocational Behavior, 45(1), 79-122, 1994
• J Mekolichick, and M K Gibbs, ‘Understanding college generational status in the undergraduate research mentored relationship’, in Council on Undergraduate Research Quarterly, 33(2), 40, 2012
• S Moffatt, M White, J Mackintosh and D Howel, ‘Using quantitative and qualitative data in health services research: What happens when mixed method findings conflict?’, in BMC Health Services Research, 6(28), 1-10, 2006
• J Molyn, The role and effectiveness of coaching in increasing career decision self-efficacy, outcome expectations and employability efforts of higher education students, (unpublished doctoral dissertation), University of Greenwich, London, 2018