Download

Original Research

Implementation of a Structured Morbidity and Mortality Rounds on an Internal Medicine Clinical Teaching Unit: A Quality Improvement Project

Bennet Schwartzentruber, MD FRCPC1*, Penny Tam, MD FRCPC1, Allison Chiu, MSc HQ2

1Department of Internal Medicine, University of British Columbia Faculty of Medicine, Vancouver, BC, Canada;

2Vancouver Coastal Health Authority, Vancouver, BC, Canada

Abstract

Morbidity and mortality rounds (MMRs) are widely used in a variety of medical settings; however, their implementation and quality are highly variable. In this quality improvement project, we implemented the Ottawa Morbidity and Mortality Model (OM3) at a quaternary teaching hospital inpatient internal medicine clinical teaching unit (CTU). We assessed adherence to the model using a locally developed scale and surveyed participants regarding acceptability of the change. We used several measures to improve adherence, including regular involvement of allied health professionals, screening of cases for appropriateness, providing a template to the residents who prepare cases for presentation, and limiting the number of presentations at each session. Adherence to OM3 improved over time and was consistently high by the end of our data-collecting period. The intervention was also widely accepted by participants, and rounds were found to be valuable to participants. Implementing a validated, structured format, such as OM3, can improve the quality of MMRs while being accepted by participants in an internal medicine teaching hospital.

Résumé

Les revues de morbidité et mortalité (RMM) sont largement utilisées dans divers contextes médicaux, mais leur mise en œuvre et leur qualité varient considérablement. Dans ce projet d’amélioration de la qualité, nous avons implanté l’Ottawa Morbidity and Mortality Model (OM3) dans une unité d’enseignement clinique (UEC) de médecine interne en milieu hospitalier universitaire de soins quaternaires. Nous avons évalué l’adhésion au modèle à l’aide d’une échelle conçue localement et interrogé les participants sur l’acceptabilité de ce changement. Nous avons utilisé plusieurs mesures pour améliorer l’adhésion, notamment la participation régulière de professionnels paramédicaux, la vérification de la pertinence des cas, la fourniture d’un modèle aux résidents qui préparent les cas à présenter et la limitation du nombre de présentations de cas à chaque séance. L’adhésion à l’OM3 augmente au fil du temps et est toujours élevée à la fin de notre période de collecte des données. Cette intervention est aussi largement acceptée par les participants, et les revues se sont révélées précieuses pour les participants. La mise en œuvre d’un format structuré et validé comme l’OM3 peut améliorer la qualité des RMM tout en étant acceptable pour les participants en médecine interne dans un hôpital universitaire.

Key words: quality improvement, quality and safety, patient quality and safety, medical education, internal medicine’ morbidity and mortality

Corresponding Author: Bennet Schwartzentruber: bits37@gmail.com

Submitted: 17 October 2021; Accepted: 24 January 2022; Published: 20 June 2022

Doi: http://dx.doi.org/10.22374/cjgim.v17i2.585

All articles published in DPG Open Access journals
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)(https://creativecommons.org/licenses/by-nc/4.0/).

Introduction

Available knowledge

Morbidity and mortality rounds (MMRs) are widely used in many academic institutions, especially since the advent of the patient quality and safety movement.1,2 These are usually conducted by multidisciplinary health professionals with the two aims of (i) quality assurance and (ii) medical education. However, the implementation of MMRs differs greatly between institutions, thereby raising questions about their consistency and effectiveness.1,3,4 A growing body of literature is available on attempts to measure and improve their effectiveness.57

A framework for high-quality MMRs, the Ottawa Morbidity and Mortality Model (OM3), has been validated and published by a team from the University of Ottawa.2,5 OM3 emphasizes a structured analysis of cases where a preventable harm or “near miss” resulted from cognitive errors or system’s issues, and from which lessons can be learnt to prevent recurrence of adverse outcomes. It also encourages interdisciplinary discussion, which has been recognized as a key component of high-quality MMRs.4 Rounds are conducted in a confidential and blame-free environment, and the facilitating chief medical resident orients participants to this approach each time. The aim of this project is to implement OM3 to improve the quality of our MMRs.

In Canada, competency in patient quality and safety processes is an explicit goal of CanMEDS Competencies of the Royal College of Physicians and Surgeons of Canada.8 For trainees in internal medicine, the Royal College, which is the main licensing body for specialist physicians, has included these competencies in both the objectives framework that applied during our study period, and the Competency-Based Medical Education (CBME) framework that applies currently.9,10

Specific aims

The aim of this project was to improve the quality of our MMRs by increasing our adherence to the OM3 elements by 20% within 10 months.

Methods

Context

Vancouver General Hospital (VGH) is a quaternary care teaching hospital and one of the core training sites for the University of British Columbia Internal Medicine residency. The internal medicine clinical teaching unit (CTU) admits roughly 100–120 patients at any given time. Each CTU team usually consists of a senior resident (second- or third-year resident), two first-year residents, up to three medical students, a clinical pharmacist, and an attending physician.

Morbidity and mortality rounds have been a regular educational activity at VGH CTU for over 10 years. MMRs are held near the end of each 4-week academic block. The chief medical resident (CMR) facilitates, and the senior resident from each team presents a case drawn from the block. There are five or six CTU teams at any given time. In addition to trainees, at least two faculty members are also present who are on service and involved in at least one of the cases presented.

Issues identified at MMRs are brought to the notice of the CTU director and/or quality improvement (QI) physician lead, both of whom regularly attend MMRs. These issues are then further discussed at the monthly CTU QI committee, where managers, directors, head nurses, physicians, and CMR discuss and try to resolve quality issues. If necessary, issues are further escalated to the hospital quality council, where more senior administration is present.

Prior to this project, no consistent guidance was provided to residents in preparing their cases, and the process for acting on any identified safety issues was unclear. Further, the appropriateness of cases was highly variable, and often rare diagnoses were chosen rather than cases identifying opportunities for systemic changes to improve quality and safety of patients. While both are important and legitimate activities, neither fulfills the goal of MMRs, which is to identify systems issues or cognitive errors leading to preventable adverse outcomes or near misses. Further, rounds were usually attended by medical trainees and faculty only, without substantial input from nursing, pharmacy, or other stakeholders involved in patient care.

In order to standardize rounds, in 2016, the OM3 framework was adopted for CTU MMRs at VGH. However, it was not formally assessed prior to this project.

Interventions

Several interventions were introduced to improve adherence to OM3 elements through a series of plan-do-study-act (PDSA) cycles. The project timeline is depicted in Figure 1. Each year consisted of 13 four-week academic blocks with academic year beginning on July 1 and ending on June 30. The interventions were as follows:

Figure 1. Project timeline from initial adoption of OM3 through formal data-collecting period, and intervention to improve adherence to OM3.

PDSA 1 – 2019, Block 1: Interprofessional involvement (nursing, pharmacy, dietary, and social work)

Previously, residents were encouraged to invite interprofessionals involved in the case to attend MMRs. However, this was ad hoc and interprofessional attendance was infrequent and inconsistent. We formalized their involvement by sending a standing digital invitation with the year’s schedule of MMRs. Rounds were also discussed with our hospital’s CTU QI committee, which is attended by physicians, nursing leadership, pharmacy, and other allied health and hospital administrators.

PDSA 2 – 2019, Block 7: Templated presentation for presenters

Given ongoing heterogeneity in presentations and concern about the time burden on residents for preparing cases, a presentation template adapted with permission from co-lead of the OM3 model (personal communication of E. Kwok with Penny Tam, May 8, 2018) was circulated to residents with an introductory email from CMR. This provided “fill in the blank” slides with simple instructions on what information to include in the presentation. The goal of this template was to simplify and standardize presentations (see Appendix 1).

PDSA 3 – 2019, Block 11: Increasing time for discussion

Previously, rounds were of 1-h length, with up to six cases being presented. Now, senior residents are asked to email a brief case summary to CTU QI physician lead to assess appropriateness based on the OM3 criteria and provide feedback on aspects of the case to highlight. The number of cases presented in 1-h duration was limited to four because of concern that too many cases limited time for productive discussion around implications of the case and potential changes to address the preventable harm.

Measures

The following two measures were used to assess our aims:

  1. A locally developed scale was used to grade the adherence of each presentation to OM3. Each presentation was assigned a score from 0 to 6 for adherence to each of the six OM3 criteria (Table 1), with each criterion being assigned a score of 0 (criterion not satisfied), 0.5 (criterion partially satisfied), or 1 (criterion satisfied). The six criteria included: appropriate case, structured case analysis, interprofessional involvement, facilitator presence, action item, and bottom line. This analysis was performed on presentation slides submitted by senior residents and adjudicated by authors Bennet Schwartzentruber and Penny Tam, with any discrepancy resolved by consensus discussion. Data were collected from block 8 of 2018 to block 11 of 2019 (January 2018–May 2019).

  2. Surveys were circulated to residents and faculty (Appendices 1 and 2, respectively) after rounds to assess satisfaction with and perceived usefulness of MMRs, with each statement graded using a 5-point Likert scale from “Strongly agree” to “Strongly disagree.” These were completed on a voluntary basis. This served in part as a balancing measure to ensure that our intervention did not have unintended consequences on participants. Survey data from residents were gathered from block 11 of 2018 to block 5 of 2019 (October 2018–May 2019). Survey data from attending physicians were collected once in May 2019. No survey for allied health professionals was developed at the time of writing this paper.

Table 1. Mean adherence to OM3 before and after PDSA1 with absolute change

OM3 element Pre PDSA1 (2018-8 to 2019-1) Post PDSA1 (2019-2 to 2019-11) Absolute change
Appropriate case 0.96 0.90 –0.06
Structured case analysis 0.67 0.94 0.27
Interprofessional involvement 0.13 0.97 0.84
Facilitator 1.00 1.00 0
Bottom line 0.74 0.90 0.16
Action item 0.63 0.92 0.29

Analysis

Each presentation was analyzed using a scale from 0 to 6 on adherence to OM3. Mean adherence scores were calculated for all presentations allowed in each block. In addition, mean scores were calculated for each subcategory of adherence.

Case presentations were submitted to the authors by email in pdf or PowerPoint format prior to MMRs. After MMRs, cases were analyzed by two of the authors (Schwartzentruber and Penny Tam). Initially, both authors analyzed each case to establish agreement. After this, cases were analyzed by one or the other author, and in case of uncertainty, scores were resolved by consensus. Each case was assigned a score of 0–6 as described above. Mean scores were calculated for all presentations in each academic block. Additionally, mean scores for each subcategory of adherence were calculated pre- and post-PDSA cycle 1, with values from 0 to 1. Given high mean scores of above 5 out of 6 after PDSA1, subsequent analyses between PDSA1 and PDSA2 and between PDSA2 and PDSA3 were not performed.

Numerical values based on the Likert scale were assigned to survey results: from 1 for “strongly disagree,” 3 for “neutral” to 5 for “strongly agree.” Mean values for each question as well as total mean scores were calculated for each block and trended over time.

Ethical considerations

This was a QI project, which under Article 2.5 of the Tri Council Policy Statement was not subjected to institutional ethical review. Therefore, ethical approval was not required.

Results

Adherence to OM3

Adherence to the OM3 model increased over the 17-month period of data collection. The adherence score for the pre--intervention period was 3.93 out of a possible score of 6.00, and the mean score for post-intervention increased to 5.51, which was an improvement of 40.5%. This exceeded our aim of 20% improvement.

Comparing the subcategories of adherence, regular interprofessional involvement had the greatest impact impact on overall adherence. Prior to our intervention, allied health professionals were rarely present at MMRs, but after PDSA1, rounds were attended regularly by at least one allied health professional. This accounted for most of the inflection point seen in Figure 2.

Figure 2. Run chart indicating adherence to the OM3 model over a 17-block period. Scores range from 0 to 6 depending on adherence to the six criteria of OM3. Total score in each block is represented by a blue line. Median scores are averaged for each PDSA cycle. PDSA1 had regular interprofessional involvement, PDSA2 had templated presentations, and PDSA3 had increasing time for discussion.

The absolute change for each of the OM3 elements is shown in Table 1, where interprofessional involvement had an absolute change of 0.84.

Resident and faculty surveys

Surveys were circulated to participating residents for each block for 8 months. Between three and seven responses were received for each block, with a mean score of 5.25. MMRs were found to be useful, receiving a mean score of 4.33/5.00 over eight blocks of data. Residents also expressed a high degree of comfort with selection of cases (mean score 4.19/5.00), identifying systems or cognitive issues (mean score 4.13/5.00), and forming bottom lines and action items (mean score 4.05/5.00). There was also a high degree of comfort discussing the cases openly with a mean score of 4.20/5.00. The question with the lowest score was residents’ knowledge that bottom lines and action items were acted on at a systems level with a mean score of 3.60/5.00. Figure 3 shows the trend in these scores over time. The resident survey is provided in Appendix 2.

Figure 3. Resident survey data. Mean scores are depicted over time. A 5-point Likert scale was used (1 = strongly disagree, 3 = neutral, and 5 = strongly agree).

Surveys were collected from nine staff internists at a single time point, mid-way through our data collection. These also demonstrated a high degree of satisfaction with the OM3 model for MMRs, as shown in Table 2. The question with the lowest score was their awareness of actions taken in the hospital in response to previous MMRs, with a mean score of 3.56. The faculty survey is provided in Appendix 3.

Table 2. Attending physician survey data, using 5-point likert scale at a single time point. Nine responses were received.

Question Mean
The Ottawa M&M model is useful for M&M rounds 4.33
The cases selected are appropriate 4.11
The case analysis done by residents is appropriate 3.78
1 am comfortable facilitating discussions at M&M 3.78
1 would like training in facilitating discussions using the Ottawa Model 4.22
1 am aware that bottom lines/action items are acted on 3.56
Inter-professional involvement (RNs. Pharmacy) at M&Ms is useful 4.44
The templated M&M presentations since January 2019 is useful 4.13

Discussion

This QI project demonstrated that implementation of a validated structured approach to MMRs for internal medicine at a teaching hospital was feasible with a high degree of adherence to the OM3 format. Furthermore, this change was widely acceptable to participants, many of whom were trainees in internal medicine. These findings agreed with the results of Kwok and colleagues.5 While their intervention was in -hospital-wide MMRs, the current project showed successful implementation in an internal medicine teaching service.

A strength of our adherence scale was its simplicity, which made post hoc analysis of presentations easy with a high degree of agreement between authors. Acceptability and perceived usefulness to trainees were important findings, given the burden of preparing these presentations on busy senior residents, and the concern that additional reading of the OM3 framework could add to this burden. In hindsight, it might have been informative to ask residents whether use of the framework led to a perceived increase or decrease in their workload. There was a slight decrease in “appropriate case” selection from 0.96 to 0.90 out of 1.00 (–0.06). We suspected that this small change represented sampling bias, rather than a trend, but we would need further data to clarify whether this was a negative unintended effect of our intervention.

Regular interprofessional involvement in MMRs had the most significant impact on adherence during our data-collection period, although improvements were also observed on other adherence criteria. This is a critical element of patient quality and safety, as engagement of multiple stakeholders enriches discussion and is important to making systemic changes.

Our surveys demonstrated that for both residents and staff, knowledge of the actions taken in the hospital resulting from previous MMRs could be improved. Formerly, feedback on the actions taken was not a regular part of rounds, and these results encouraged us to add regular feedback on actions taken to the introductory slides presented at the start of rounds.

Of note, in January 2019, there was a brief decrease in satisfaction scores across most categories, which correlated with a change in chief medical resident at the end of calendar year, and also with the busiest time of year for medical inpatient services. It is unclear which of these factors may have been contributing. Satisfaction scores did not otherwise change significantly over time. Given consistent average scores of above 4/5, we interpreted this as sustained acceptable satisfaction with rounds despite changes.

One limitation of our data was that the first intervention undertaken—that of adopting OM3—was done before the start of data collection, and it could have been informative to analyze presentations before OM3 was implemented. This might have demonstrated a more pronounced improvement across the criteria over time starting from overall lower levels of adherence. Another limitation was that while our locally developed scale performed well in assessing the adherence of presentations to OM3, there were other aspects of MMRs that were not captured, such as quality of the discussion, and ability of MMRs to effect changes to the cognitive and systems issues they identify. Further, the binary nature of variables lacked granularity. For example, a discussion where nurses, pharmacists, physiotherapists, and physicians were involved could be more productive than when only a single nurse was present with physicians, but both situations would receive a score of “1” on this criterion.

Future studies would be useful to validate our scoring of adherence to OM3. Other studies could evaluate the use of OM3 in specialties other than internal medicine or across other training sites, as many residents work at multiple academic sites that use different formats for MMRs. Although not the direct aim of this project, results of our rounds were presented to the faculty CTU QI committee for further discussion and implementation. More recently, data were extracted from our MMRs for hospital-wide quality and safety processes. Additional work locally could examine the impact of MMRs on systems level changes.

Conclusions

The OM3 framework is validated for MMRs that was successfully adopted in an inpatient internal medicine teaching hospital setting. We exceeded our aim of achieving 20% improvement in adherence to OM3 by achieving 40% improvement. Regular interprofessional participation was easily achieved and contributed significantly to meeting our aim. The change was sustainable as it did not require extra work for participants, and in fact may have simplified the process for senior residents by providing a framework and template for presentation. It was also widely accepted by participants. Further research could extend this approach to other specialties and settings, such as surgical specialties or other training sites. Communicating the outcomes of MMRs back to participants could be an important and neglected element of their success.

Conflict of Interest

The authors have no competing interests to declare.

Funding

This project was supported by the Doctors of BC Specialist Services Committee, Physician Quality Improvement initiative.

REFERENCES

1. Smaggus A, Mrkobrada M, Marson A, Appleton A. Effects of efforts to optimise morbidity and mortality rounds to serve contemporary quality improvement and educational goals: A systematic review. BMJ Qual Saf. 2018 Jan;27(1):74–84. 10.1136/bmjqs-2017-006632

2. Calder LA, Kwok ESH, Adam Cwinn A, et al. Enhancing the quality of morbidity and mortality rounds: The Ottawa M&M model. Acad Emerg Med. 2014 Mar;21(3):314–21. 10.1111/acem.12330

3. Katz D, Detsky AS. Incorporating metacognition into morbidity and mortality rounds: The next frontier in quality improvement: Metacognition to reduce medical error. J Hosp Med. 2016 Feb;11(2):120–2. 10.1002/jhm.2505

4. Epstein NE. Morbidity and mortality conferences: Their educational role and why we should be there. Surg Neurol Int. 2012;3 (Suppl 5):S377-88. 10.4103/2152-7806.103872

5. Kwok ESH, Calder LA, Barlow-Krelina E, et al. Implementation of a structured hospital-wide morbidity and mortality rounds model. BMJ Qual Saf. 2017 Jun;26(6):439–48. 10.1136/bmjqs-2016-005459

6. Wong BM, Etchells EE, Kuper A, Levinson W, Shojania KG. Teaching quality improvement and patient safety to trainees: A systematic review: Acad Med. 2010 Sep;85(9):1425–39. 10.1097/ACM.0b013e3181e2d0c6

7. Benassi P, MacGillivray L, Silver I, Sockalingam S. The role of morbidity and mortality rounds in medical education: A scoping review. Med Educ. 2017;51(5):469–79. 10.1111/medu.13234

8. Frank JR, Snell L, Sherbino J, editors. CanMEDS 2015. Physician competency framework. Ottawa, Canada: Royal College of Physicians and Surgeons of Canada; 2015.

9. Royal College of Physicians and Surgeons of Canada. Objectives of training in the specialty of internal medicine. Ottawa, Canada: Royal College of Physicians and Surgeons of Canada; 2011.

10. Royal College of Physicians and Surgeons of Canada. Internal medicine competencies. Ottawa, Canada: Royal College of Physicians and Surgeons of Canada; 2018.

Appendix 1

MMR presentation template

Appendix 2

Resident survey

Appendix 3

Faculty survey