Science

Anna Blackman, Gianna Moscardo, & David E. Gray (2016). Challenges for the Theory and Practice of Business Coaching: A Systematic Review of Empirical Evidence. (Link)

Until recently, there has been little published systematic empirical research into business coaching. This article reports on a systematic, critical review of 111 published empirical papers investigating business coaching theory, processes, and outcomes. The present article identifies a significantly larger body of empirical research than covered in previous reviews and uses a Systematic Review methodology (SRm) to conduct a comprehensive review of the available empirical evidence into business coaching effectiveness focusing on implications for theoretical development, practice (within human resource development) and further research in this area. This review identifies convergence around factors that contribute to perceived effective coaching practice but nevertheless highlights a number of issues to be resolved in further research. These include determining the primary beneficiaries of coaching, the factors that contribute to coach credibility, and how the organizational and social context impacts on coaching. Weaknesses in coaching research methodology and research gaps are also noted.

Tim Theeboom, Bianca Beersma & Annelies E.M. van Vianen, The Journal of Positive Psychology (2013): Does coaching work? A meta-analysis on the effects of coaching on individual level outcomes in an organizational context, The Journal of Positive Psychology: Dedicated to furthering research and promoting good practice, DOI: 10.1080/17439760.2013.83749. (PDF)

Whereas coaching is very popular as a management tool, research on coaching effectiveness is lagging behind. Moreover, the studies on coaching that are currently available have focused on a large variety of processes and outcome measures and generally lack a firm theoretical foundation. With the meta-analysis presented in this article, we aim to shed light on the effectiveness of coaching within an organizational context. We address the question whether coaching has an effect on five both theoretically and practically relevant individual-level outcome categories: performance/skills, well-being, coping, work attitudes, and goal-directed self-regulation. The results show that coaching has significant positive effects on all outcomes with effect sizes ranging from g = 0.43 (coping) to g = 0.74 (goal-directed self-regulation). These findings indicate that coaching is, overall, an effective intervention in organizations.

Rebecca J. Jones, Stephen A. Woods, & Yves R. F. Guillaume (2015). The effectiveness of workplace coaching: A meta‐analysis of learning and performance outcomes from coaching. (Link)

This study presents a meta‐analysis synthesizing the existing research on the effectiveness of workplace coaching. We exclusively explore workplace coaching provided by internal or external coaches and therefore exclude cases of manager–subordinate and peer coaching. We propose a framework of potential outcomes from coaching in organizations, which we examine meta‐analytically (k = 17). Our analyses indicated that coaching had positive effects on organizational outcomes overall (δ = 0.36), and on specific forms of outcome criteria (skill‐based δ = 0.28; affective δ = 0.51; individual‐level results δ = 1.24). We also examined moderation by a number of coaching practice factors (use of multisource feedback; type of coach; coaching format; longevity of coaching). Our analyses of practice moderators indicated a significant moderation of effect size for type of coach (with effects being stronger for internal coaches compared to external coaches) and use of multisource feedback (with the use of multisource feedback resulting in smaller positive effects). We found no moderation of effect size by coaching format (comparing face‐to‐face, with blended face‐to‐face and e‐coaching) or duration of coaching (number of sessions or longevity of intervention). The effect sizes give support to the potential utility of coaching in organizations. Implications for coaching research and practice are discussed.

Grant, Anthony & Cavanagh, Michael. (2007). Evidence-based coaching: Flourishing or languishing?. Australian Psychologist – AUST PSYCHOL. 42. 10.1080/00050060701648175. (PDF)

Coaching and coaching psychology offer a potential platform for an applied positive psychology and for facilitating individual, organisational and social change. Experts from around the world were invited to comment on the emerging discipline of coaching psychology and the commercial coaching industry. Several key themes emerged including the potential of coaching to contribute to health promotion, social change and organisational development. There was unequivocal consensus for the need for an evidence-based approach to coaching. A review of the psychological coaching outcome literature found there have been a total of 69 outcome studies between 1980 and July 2007: 23 case studies, 34 within-subject studies and 12 between-subject studies. Only eight randomised controlled studies have been conducted. This indicates that coaching psychology is still in the early stages of development, and can be understood as an emerging or protoscientific psychological discipline. A languishing-flourishing model of coaching is described. To flourish, coaching psychology needs to remain clearly differentiated from the frequently sensationalistic and pseudoscientific facets of the personal development industry while at the same time engaging in the development of the wider coaching industry.

Joy McGovern, Ph.D., Michael Lindemann, Ph.D., Monica Vergara, M.A., Stacey Murphy, Linda Barker, M.A., & Rodney Warrenfeltz, Ph.D. “Maximizing the Impact of Executive Coaching: Behavioral Change, Organizational Outcomes, and Return on Investment” The Manchester Review, Volume 6 • Number 1, 2001 (PDF)

Executive coaching, which surfaced as a leadership development practice over a decade ago, is now among the most widely used executive development techniques. Yet, despite its growing tenure as a leadership development practice, executive coaching is still used sparingly in many organizations, and has remained underutilized. The authors believe this is because the paucity of empirical research into its effectiveness leaves the field open to speculation and subjective opinion.

De Meuse, Kenneth P., Dai, Guangrong and Lee, Robert J. (2009) ‘Evaluating the effectiveness of executive coaching: beyond ROI?’, Coaching: An International Journal of Theory, Research and Practice, 2:2, 117 — 134 (PDF)

The popularity of executive coaching has increased dramatically in both the practitioner world and academia during the past decade. However, evaluating the effectiveness of coaching has lagged behind. Executive coaching is a multi-disciplinary practice, and professionals from many different scholarly back-grounds provide coaching services. The paucity of empirical research may be attributed to the lack of a consensus among these divergent professionals regarding whether and how to evaluate the effectiveness of coaching. In this article, we conducted a meta-analysis of the empirical research as well as reviewed the retrospective studies evaluating coaching effectiveness. Subsequently, we discussed six areas that impact the way researchers evaluate coaching effectiveness and the conclusions they may draw from their studies. Although the Return On Investment (ROI) index provides a straightforward, overall measure of effectiveness, its veracity and usefulness is questioned. It is hoped that the clarification of these areas will help guide the future of coaching evaluation research and practice.

Lawrence, Paul & Whyte, Ann. (2013). Return on investment in executive coaching: A practical model for measuring ROI in organisations. Coaching: An International Journal of Theory, Research and Practice. 7. 4-17. 10.1080/17521882.2013.811694. (PDF)

Research and anecdotal evidence suggest that the majority of organisations make no attempt to evaluate their investment in executive coaching services. Although a body of work espousing the value of financial return on investment (ROI) methodologies exists, few organisations appear to have faith in the value of these frameworks in persuading senior management as to the efficacy of coaching. This paper reports the findings of a survey conducted with purchasers of executive coaching services (purchasing clients) using a semi-structured interview. The survey was designed to explore how purchasing clients currently evaluate their investment in coaching. Using a grounded theory approach, a new model for evaluating investment in coaching services is presented. The new model places a primary emphasis on establishing the purpose of coaching, and on establishing an ongoing, formative approach to evaluation. The potential value of the new model to both organisations and executive coaches is discussed.

Anderson, Merrill C. (2001). Executive Briefing: Case Study on the Return on Investment of Executive Coaching. (Link)

The Bottom Line: Coaching produced a 529% return on investment and significant intangible benefits to the business. The study provided powerful new insights into how to maximize the business impact from executive coaching.

• Each UP is designed around the participant(s) and adapts based on what they need over time. This makes it somewhat difficult to grasp exactly what UP is and to compare between UPs with different people.
• Theoretically, every task completed for or with the participant is the participant's highest value task. In practice, this is somewhat messy. A large part of the UP is to help the participant move closer and closer to allocating their resources to their highest value projects on a moment-to-moment basis.
• A very rough analogy of what the UP Coach is to a participant is the relationship between Alfred Pennyworth and Bruce Wayne. We’re their “right hand person”. We do whatever is necessary to help the participant achieve their goals, within reason. 

• Most of our participants do not wish to reveal their identities. This is principally due to the fact we attempt to provide a comprehensive review of their results, including their failures. It’s stressful for participants to pursue their often ambitious goals without extra public scrutiny.
• For our preliminary case studies, we’re primarily providing a high-level snapshot of the participant's life during their UP. We have a great deal of additional data on each participant, including dozens to hundreds of pages of notes, data from apps, doctor’s reports, self-assessments, etc. But it's difficult to publish more data and retain the participant's privacy. We explicitly remove some data–for example, goals with specific names or metrics–to further ensure participants privacy. It depends on the participant's preferences. The participant's privacy is more important than our desire for transparency.

• We attempt to collect a substantial amount of baseline and follow up data on every area of the participant's life, but we are limited by what they choose to track and reveal. There’s a great deal of trust between the participant and Upgradable, but there isn’t usually much time or interest from the participant to fully participate in a research study. They’re usually extremely busy and providing us consistent data is sometimes distracting. Part of their UP generally includes developing effective Life Dashboards to measure what’s most important to them, but these take time to build. They also evolve over time as the participant learns what they find useful enough to track and analyze. If permitted by the participant, we will eventually publish their de-identified Life Dashboard data.
• UP as an intervention generally includes all the resources a participant uses to achieve their goals, including time spent by coaches, assistants, doctors, nutritionists, etc. But given the difficulties of comprehensively measuring their time contributions, we do not include that data even if we recommend, hire, and/or manage those individuals for the participant. We also don’t estimate the participant's total time contributions. Including everything would substantially increase the amount of time input needed to achieve the outcomes presented.

• Given the relatively weak research methods of many organizations and practitioners–including Upgradable–it’s hard for the general public to trust the usefulness of a lot of published data. Their results are either untrue due to researcher integrity or capability or they’re not easily generalizable to you and your particular situation for often good reasons. In ~99% of cases in the self-development space, the data is simply nonexistent or little more than cherry-picked testimonials or generally unreliable self-report data. We try to do better than this, but it is extremely hard without a large budget for research.
• Please remember that individual differences play a huge role in individual outcomes.
• We fully acknowledge that selection effects account for a substantial amount of the positive results you see in our data. We suspect this is the case for most other behavior change programs, as well as with Ivy league schools, prestigious fellowship programs, elite training programs, best-in-class startup accelerators, etc. Ultimately we wish to determine our impact on people of all backgrounds, not just the people that currently are interested in and able to do an UP.
• We don’t find it exceptionally useful if organizations and practitioners only include selected success stories for their interventions. We would like to include data–flattering and unflattering–for all of our full UP participants in time. But we need a much larger sample size to ensure participants are de-identified first. We unfortunately cannot guarantee we will always publish all data given some participants will refuse to participate, but we will come as close as we realistically can. The first 10 case studies we published represent 100% of the first 10 full UPs we did--no cherry-picking. We welcome third-party audits of our data to ensure we communicate our results faithfully.
• Aside from some much needed replications, we don’t find it exceptionally useful for organizations and practitioners to publish data that is obvious to behavior change experts and astute customers. Unless absolutely necessary, we do not advocate for publishing relatively uninformative data just to “check a credibility box” and ostensibly inform customers of something useful.
• At this stage of our company development, we primarily focus on what makes sense for participants on an individual level. We do wish to generate broadly useful conclusions about how to live better, but that is a multi-decade research effort that is surprisingly hard to do and communicate well.
• We intend to do a deeper, multi-decade investigation of each participant and carefully release data once we secure enough capital to do this research. This includes doing pre-registered randomized controlled trials (RCTs) with a variety of participants and UP types, both with internal researchers on our team and external researchers in an adversarial collaborative environment. If you’re willing to help, please reach out.
• We would advise against making any strong conclusions about our work until multiple RCTs are published. You cannot easily infer much about your likely outcomes with UP from the case study outcome data we've released so far. We've been at this for many years now, but we still keep the word "experimental" in our communications for a reason.