Post a screenshot of the article’s frequency table and/or graph. ? Example: Frequency Distribution -OR- Graph Answer the following questions about your table or graph. ?? What ty

Maslach Burnout Inventory and a Self-Defined, Single-Item Burnout Measure Produce Different Clinician and Staff Burnout Estimates Margae Knox, MPH, Rachel Willard-Grace, MPH, Beatrice Huang, BA, and Kevin Grumbach, MD

Center for Excellence in Primary Care, Department of Family and Community Medicine, University of California, San Francisco, CA, USA.

BACKGROUND: Clinicians and healthcare staff report high levels of burnout. Two common burnout assess- ments are the Maslach Burnout Inventory (MBI) and a single-item, self-defined burnout measure. Relatively lit- tle is known about how the measures compare. OBJECTIVE: To identify the sensitivity, specificity, and concurrent validity of the self-defined burnout measure compared to the more established MBI measure. DESIGN: Cross-sectional survey (November 2016–Janu- ary 2017). PARTICIPANTS: Four hundred forty-four primary care clinicians and 606 staff from three San Francisco Aarea healthcare systems. MAIN MEASURES: The MBI measure, calculated from a high score on either the emotional exhaustion or cynicism subscale, and a single-itemmeasure of self-defined burn- out. Concurrent validity was assessed using a validated, 7-item team culture scale as reported by Willard-Grace et al. (J Am Board Fam Med 27(2):229–38, 2014) and a standard question about workplace atmosphere as reported by Rassolian et al. (JAMA Intern Med 177(7):1036–8, 2017) and Linzer et al. (Ann Intern Med 151(1):28–36, 2009). KEYRESULTS: Similar to other nationally representative burnout estimates, 52% of clinicians (95% CI: 47–57%) and 46% of staff (95% CI: 42–50%) reported high MBI emotional exhaustion or high MBI cynicism. In contrast, 29% of clinicians (95%CI: 25–33%) and 31% of staff (95% CI: 28–35%) reported Bdefinitely burning out^ or more severe symptoms on the self-defined burnout measure. The self-defined measure’s sensitivity to correctly identify MBI-assessed burnout was 50.4% for clinicians and 58.6% for staff; specificity was 94.7% for clinicians and 92.3% for staff. Area under the receiver operator curve was 0.82 for clinicians and 0.81 for staff. Team culture and atmosphere were significantly associated with both self-defined burnout and the MBI, confirming concurrent validity. CONCLUSIONS: Point estimates of burnout notably differ between the self-defined andMBImeasures. Compared to the MBI, the self-defined burnout measure misses half of high-burnout clinicians and more than 40% of high- burnout staff. The self-defined burnout measure has a

low response burden, is free to administer, and yields similar associations across two burnout predictors from prior studies. However, the self-defined burnout and MBI measures are not interchangeable.

KEY WORDS: burnout; measurement; health services research.

J Gen Intern Med 33(8):1344–51

DOI: 10.1007/s11606-018-4507-6

© Society of General Internal Medicine 2018

BACKGROUND

The National Academy of Medicine 1

and Agency for Health- care Quality and Research

2, 3 have spotlighted concerning levels of burnout among clinicians and healthcare staff, par- ticularly in primary care. High levels of burnout are concerning not only for clinician and staff well-being. A burntout workforce may adversely affect clinical quality, pa- tient experience, and costs of care.

1

Several burnout-related initiatives aim to support the Bquadruple aim^ of a sustainable clinician and staff work experience in addition to improved patient experience, quality, and lower costs.

4

The American Medical Association’s BSTEPS Forward^ modules offer guidance on practice trans- formation and clinician and trainee well-being.

5

The Society for General Internal Medicine made burnout a theme of its 2017 Annual Meeting.

6

The American Board of Family Med- icine added work experience questions to its 2016 recertifica- tion registration to better understand and track burnout,

7

and CEOs of leading healthcare organizations have issued a call to action that argues for regular measurement of physician well- being.

8

Within this context of heightened attention to clinician and staff well-being, greater understanding of the instruments used to measure burnout is essential. Two common measures are the Maslach Burnout Inventory (MBI) and a five-choice, single item based on self-defined burnout. The MBI, consid- ered an industry standard, has been fielded across large sam- ples of diverse occupations in multiple countries. It is com- posed of three dimensions: emotional exhaustion, cynicism (or depersonalization), and personal accomplishment (or profes- sional efficacy). The 16-item General MBI survey uses the terms cynicism and personal accomplishment while the 22- item Health Services Personnel survey uses the analogous terms depersonalization and professional efficacy, with results

Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11606-018-4507-6) contains supplementary material, which is available to authorized users.?

Received November 10, 2017 Revised March 26, 2018 Accepted May 16, 2018 Published online June 4, 2018

1344

consistent across survey versions. 9

MBI instruments require a license fee to administer. The self-defined burnout measure is one of the ten survey

questions on the BMini-Z^ work experience instrument devel- oped by Linzer and colleagues.

10

The item originated from a study of burnout in HMOs

11

and has subsequently been in- cluded in several major studies.

7, 12–14 It is free to use.

Estimates of Clinicians and Staff Burnout

The most recent national estimate of burnout from MBI sub- scales identified high emotional exhaustion among 46.9% of physicians and high cynicism among 34.6% of physicians, with 54.4% of physicians reporting one or both symptoms. Primary care physicians from that study reported even higher burnout levels; more than 60% expressed high emotional exhaustion and/or cynicism.

15

Other studies indicate burnout is also high among healthcare staff, with high emotional exhaustion ranging from 30 to 40%.

16, 17 However, studies using the self-defined burnout measure have found lower point estimates: about one in four family physicians and general internists report Bburning out,^ Bpersistent burnout symptoms,^ or Bcomplete burnout.^

7, 12, 18 These divergent estimates of burnout prevalence raise questions as to whether study populations differ in their well-being, or how MBI and self-defined measures perform differently.

Published Comparisons of Burnout Measures

Few studies directly compare how the MBI and self- defined burnout measures perform in the same sample of physicians. One physician survey found the self-defined burnout measure strongly correlated with the MBI emo- tional exhaustion subscale (r = 0.64, p < 0.0001); the cyni- cism subscale was less strongly correlated (r = 0.324, p value not reported).

18

A second study of Australian oncol- ogy workers also found the self-defined burnout measure and MBI emotional exhaustion subscale strongly correlated (r = 0.68, p < 0.0001).

19

Only one study used an analytic approach other than correlations. That study, a survey of rural physicians and advance practice clinicians, examined whether the self-defined burnout measure predicted high and low MBI subscale burnout categories using multivar- iate linear mixed models. The self-defined burnout measure significantly predicted high emotional exhaustion but did not predict low emotional exhaustion or any category of cynicism.

20

Prior studies have also analyzed sensitivity and speci- ficity for MBI single-item measures (one item from each MBI subscale’s five items). A correlation of 0.76–0.83 was found for the MBI emotional exhaustion single-item vs. subscale. A correlation of 0.61–0.72 was found for the MBI cynicism single-item vs. subscale.

21

Compared to the self-defined burnout single-item, the MBI emotional ex- haustion single-item had a high correlation (r = 0.79) and sensitivity and specificity over 80%.

17

These results suggest that the single-item self-defined burn- out measure and MBI subscales have strong agreement. How- ever, no comparison to our knowledge has described sensitiv- ity and specificity for the self-defined burnout measure and MBI subscales. Moreover, we are not aware of any previous study examining concurrent validity of the self-defined and MBI responses with related, validated work environment measures. Given many national surveillance efforts and pro- gram evaluations using the self-defined burnout measure, there is need for greater understanding among policy makers, researchers, and healthcare leaders on how results compare with the more established MBI burnout subscales.

OBJECTIVE

We compared the self-defined burnout measure and MBI in the same sample of primary care clinicians and staff. Our aims were to (1) compare the prevalence of burnout from the different measures, (2) test the sensitivity and specificity of the self-defined burnout measure to identify individuals expe- riencing high burnout compared to standardMBI benchmarks, and (3) determine if the self-defined burnout measure andMBI have similar associations with a clinic team culture survey measure previously found to be significantly associated with MBI scores

22

and a workplace atmosphere survey measure previously found to be significantly associated with the self- defined burnout measure.

23

METHODS

Design

This study was a cross-sectional survey, approved by the Institutional Review Board of the University of California, San Francisco (protocol numbers 11-08048 and 17-23324).

Participants

We surveyed clinicians and staff working in primary care clinics in three San Francisco area health systems: a university-run clinic network, a network of neighborhood and hospital-based clinics administered by a county health department, and a large private medical group. All clinicians and staff at the university and county clinics and all clinicians at the private group were eligible to participate. Clinicians consist of physicians of family and internal medicine, physi- cian assistants, and nurse practitioners. Staff members include registered nurses, medical assistants, and administrative sup- port. The survey was fielded between November 2016 and January 2017, and was primarily administered electronically. Each person received an e-mail invitation to complete the survey, with up to five reminder e-mails to non-respondents. Paper surveys were administered during staff meetings at some county health network sites based on leadership request. Respondents at two systems were entered into a $25 gift card

1345Knox et al.: Clinician and Staff Burnout MeasuresJGIM

raffle; the third system elected to give each respondent $50 for participation.

Measures

The survey included the 16-item MBI General Survey sub- scales for emotional exhaustion and cynicism as well as a single-item, self-defined burnout measure. MBI subscales were each composed of five burnout symptoms. Respondents rated how often they experience each symptom from 0 (never) to 6 (every day), and responses were summed for each sub- scale (composite score of 0–30 points). High, medium, and lowMBI burnout cut points were based on a distribution of the composite score into terciles from a reference population.

24

High emotional exhaustion was defined as a composite score greater than or equal to 16; high cynicism was a composite score greater than or equal to 11.

9

We primarily analyzed MBI scores based on the presence of high emotional exhaustion or high cynicism, as done in commonly cited national prevalence estimates.

15, 25

Self-defined burnout was a single question that assessed burnout on a scale from 1 to 5.Most studies using this measure define high burnout as answering positively to option 3, 4, or 5.

7, 12 Response options were as follows: (1) BI enjoymywork. I have no symptoms of burnout^; (2) BOccasionally I am under stress, and I don’t always have as much energy as I once did, but I don’t feel burned out^; (3) BI am definitely burning out and have one or more symptoms of burnout, such as physical and emotional exhaustion^; (4) BThe symptoms of burnout that I’m experiencing won’t go away. I think about work frustrations a lot^; and (5) BI feel completely burned out and often wonder if I can go on. I am at the point where I may need some changes or may need to seek some sort of help.^ The survey also included a validated 7-item measure of

team culture previously found to be associated with the MBI. Team culture included agreement with statements such as, BThe group of staff and providers I work with most regularly work well together as a team^ and BI can rely on other people at my clinic to do their jobs well.^ Respondents rated each item from 1 (strongly disagree) to 10 (strongly agree). A composite score was calculated as an average across the seven items.

22

Workplace atmosphere was assessed on a scale from 1 to 5 in response to: BWhich number best describes the atmosphere in your primary work area?^ Response anchors included 1 (calm), 3 (busy but reasonable), and 5 (hectic, chaotic).

10, 23

Data Analysis

All analyses were conducted using Stata 13 26

and stratified by clinician or staff respondent. We stratified clinician and staff analyses to be consistent with other reportings

14

and based on prior findings of differences in burnout between clinicians and staff.

22

We also conducted sub-analyses by gender and part- time work status to confirm whether results were consistent. Correlations for comparing our results with other studies were

calculated using Pearson’s correlation coefficients to measure the association between the self-defined measure with the MBI emotional exhaustion and MBI cynicism subscales.

17,

19, 27

We used a self-defined burnout cut point of 3 (definitely burning out) or above to test sensitivity and specificity for detecting respondents with high burnout compared to standard MBI classification. We also explored cut points of 2 (under stress) and 4 (persistent symptoms) to assess sensitivity and specificity trade-offs and produced area under receiver opera- tor curves (AUC). An AUC of 1.0 indicates a perfect diagnos- tic test; above 0.9 indicates excellent discrimination; 0.8–0.9 is good; 0.7–0.8 is fair; and 0.5–0.7 is non-discriminating to poor discrimination.

28, 29

Table 1 Characteristics of Survey Respondents and Burnout Levels

Clinicians Staff

n Column %

n Column %

All respondents 444 606 Respondent characteristics System System 1 (university

operated) 114 26 181 30

System 2 (county administered)

175 39 425 70

System 3 (private medical group) a

155 35 n/a n/a

Gender b

Male 73 26 101 17 Female 216 75 497 82 Transgender/other 0 0 5 1 Tenure with health system < 1 year 32 7 68 12 1–5 years 106 24 133 23 6–10 years 91 21 113 19 11–15 years 76 17 107 18 > 15 years 135 31 165 28 Clinic sessions per week 1–2 half-days (clinicians) 98 22 n/a n/a 3–5 half-days (clinicians) 226 51 n/a n/a 6 or more half-days

(clinicians) 117 27 n/a n/a

Less than 20 h per week (staff)

n/a n/a 57 10

More than 20 h per week (staff)

n/a n/a 535 90

Burnout levels MBI (exhaustion) Low (0–10) 141 32 252 42 Medium (11–15) 100 23 121 20 High (16+) 197 45 226 38 MBI (cynicism) Low (0–10) 202 46 277 46 Medium (6–10) 89 20 123 21 High (11+) 147 34 198 33 MBI (high exhaustion or cynicism)

231 52 277 46

Self-defined burnout measure 1—no symptoms 90 21 161 28 2 217 50 237 41 3—burning out 99 23 110 19 4 21 5 55 9 5—burned out, seeking

help 5 1 16 3

MBI Maslach Burnout Inventory, n/a not applicable aStaff in system 3 were not surveyed bGender was not asked in system 3

1346 Knox et al.: Clinician and Staff Burnout Measures JGIM

We assessed concurrent validity by calculating the propor- tion of burned out and not burned out respondents for both the self-defined and MBI measures in connection with (1) strong team culture and (2) hectic or chaotic work atmosphere. Un- adjusted odds ratios were calculated to compare associations across the two burnout measures.

RESULTS

The response rate was 74%. Four hundred forty-four of 592 clinicians and 606 of 826 staff responded. Respondents were predominantly female (Table 1). About half had worked with their health system more than 10 years. Almost all staff (90%) worked more than 20 h a week while fewer clinicians (27%) worked six or more half-days in clinic. High burnout based on the MBI—high emotional

exhaustion or high cynicism—was reported by 52% of clinicians (95% CI: 47–57%) and 46% of staff (95% CI: 42–50%). Burnout levels for emotional exhaustion and cynicism subscales individually are reported in Table 1. High self-defined burnout based on a score of 3

(Bdefinitely burning out^) or greater was reported by 29% of clinicians (95% CI: 25–33%) and 31% of staff (95% CI: 28–35%). The lower proportion of self-defined burnout was statistically significant among both clini- cians and staff compared to the overall MBI measure and MBI emotional exhaustion subscale (McNemar’s test, p < 0.001) but not compared to the MBI cynicism subscale (Fig. 1). The correlation between the self-defined burnout measure

and MBI exhaustion subscale was 0.63 for both clinicians and staff (p value < 0.001). The correlation between the self- defined burnout measure and MBI cynicism subscale was 0.57 for clinicians and 0.48 for staff (p value < 0.001). Using the common cut point of 3 or greater for self-defined

burnout, sensitivity was 50.4% among clinicians and 58.6% among staff—i.e., the proportion of respondents with MBI- assessed burnout whose self-defined response also identifies burnout. Specificity—the proportion of respondents without MBI-assessed burnout who also did not report self-defined burnout—was 94.7% for clinicians and 92.3% for staff. A higher cut point of 4 on the self-defined burnout measure dropped sensitivity to 11.5% among clinicians and 24.3%

Figure 1 High burnout based on MBI subscales and the self-defined burnout measure.

1347Knox et al.: Clinician and Staff Burnout MeasuresJGIM

among staff and increased specificity to 100% for clinicians and 98.1% for staff. A lower cut point of 2 greatly increased sensitivity to 98.2% among clinicians and 91.4% among staff, however decreased specificity to 41.8% for clinicians and 44.4% for staff (Table 2). The AUC was 0.82 for clinicians and 0.81 for staff (Fig. 2). Additional sensitivity, specificity, and AUC estimates for the individual MBI emotional exhaus- tion and cynicism subscales are provided as an online appen- dix. Sub-analyses stratified by gender and clinician half-days per week yielded similar results. In an assessment of concurrent validity, strong team culture

was significantly associated with lower burnout for both the MBI (clinician OR 0.34, 95% CI 0.23–0.51) and self-defined burnout (clinician OR 0.33, 95% CI 0.22–0.51). A hectic or chaotic environment was significantly associated with greater burnout for both the MBI (clinician OR 3.56, 95% CI 2.36– 5.36) and self-defined burnout measure (clinician OR 3.07, 95% CI 1.99–4.72) (Table 3).

DISCUSSION

The most important finding from our study is that the preva- lence of burnout among primary care clinicians and staff differed considerably depending on survey instrument. For example, burnout prevalence among clinicians was more than 50% higher using the MBI compared to the self-defined

measure’s cut point of 3 (52 vs. 29%). Similarly, the self- defined measure’s sensitivity to detect individuals with burn- out in our sample missed about 50% of clinicians and 41% of staff that MBI symptoms had classified as experiencing burn- out. A lower self-defined cut point of 2 increased sensitivity to above 90%, but at the expense of substantially decreased specificity. One explanation for the lower self-defined burnout point estimate may be reluctance to self-identify as burned out given an allusion to depression or ineffectiveness. In contrast, theMBI allows individuals to identify with burnout symptoms without directly identifying as burned out. Consistent with other studies, we found a strong, significant

correlation between self-defined burnout and the MBI exhaus- tion subscale and a modest, significant correlation between self-defined burnout and the MBI cynicism subscale. Our analyses add to prior studies by reporting an AUC of 0.81– 0.82, indicating moderate to good discrimination between the self-defined burnout measure and MBI.

29, 30

Our results are also the first to demonstrate concurrent validity for both burnout measures in association with team culture, which had only been examined with the MBI,

22

and workplace atmosphere, which had only been examined with the self-defined measure.

10, 23 Strong team culture was signif- icantly associated with about one-third lower burnout for both burnout measures. A chaotic workplace atmosphere was sig- nificantly associated with about three times higher burnout for both burnout measures. The similar magnitude and variance of

Table 2 Sensitivity and Specificity of Self-Defined Single-Item Burnout Measure Cut Points for Detecting High Burnout as Measured by MBI Subscales

Clinicians Staff

Sensitivity Specificity Sensitivity Specificity

Self-defined single-item % 95% CI % 95% CI % 95% CI % 95% CI

Cut point = 4+ 11.5 7.7–16.4 100.0 98.2–100.0 24.3 19.3–29.8 98.1 95.9–99.3 Cut point = 3+ (standard) 50.4 43.7–57.1 94.7 90.7–97.3 58.6 52.4–64.5 92.3 88.7–95.0 Cut point = 2+ 98.2 95.5–99.5 41.8 34.9–48.8 91.4 87.4–94.5 44.4 38.8–50.1

Figure 2 Area under the receiver operator curve (AUC) for the MBI (high exhaustion or high cynicism) vs. self-defined burnout measure.

1348 Knox et al.: Clinician and Staff Burnout Measures JGIM

associations for both burnout measures suggests that the two burnout measures would perform similarly when exploring other burnout predictors. Our findings have implications for using and inter-

preting the self-defined and MBI burnout measures. The self-defined burnout measure appears to be an accept- able alternative to the MBI if primary aims are to track burnout trends within a single population or measure work environment factors that predict burnout. However, our results indicate it would be inappropriate to directly contrast high burnout estimates from the self-defined measures and MBI subscale measures. Researchers and health system leaders should use

caution when comparing burnout prevalence across dif- ferent populations or studies. For example, a recently published study using the self-defined measure conclud- ed that physician burnout may be decreasing in the USA.

7

Differences may actually be due to the measure- ment instrument used and the self-defined measure’s low sensitivity relative to the MBI. The National Academy of Medicine notes, B[Burnout]

terminology and measurement tools used vary substan- tially across studies…hampering efforts to quantitatively summarize outcomes (for example through meta-analy- ses), and slowing the rate of advancement in the field.^

1

Similar to other researchers who have identified need for greater consistency in defining and reporting burn- out,

31, 32 our study underlines these challenges to com- pare and pool findings across studies when different burnout measures are used. The self-defined burnout measure has several attrac-

tive qualities. It does not require a license fee, has low

response burden, and may have more face validity to healthcare workers than a multi-item scale score.

24

One drawback of the self-defined burnout measure is limited validity testing in contrast to MBI measures, which have been associated with outcomes such as clinical diagnosis of depression.

33

Our study also demonstrates the limita- tions due to the ordinal nature of the self-defined mea- sure. Forty to fifty percent of respondents in our sample selected the level 2 category, a skewed response that creates a large step-off effect. Consequently, the self- defined cut point cannot be smoothly titrated to achieve an optimal balance of sensitivity and specificity relative to the MBI (Fig. 2). This aspect of the self-defined measure may also reduce its predictive and discriminant utility when analyzing burnout gradients rather than yes/ no classifications. Our study has several limitations. First, we studied

clinicians and staff in three large health systems in a single region, which may limit generalizability. Yet burnout prevalence in our sample was similar to national samples of family physicians and general internists,

7, 15

suggesting that respondents’ work experience resembles that of other settings. Second, our sample had a high proport ion of women and part- t ime clinicians. Sensitivity/specificity sub-analyses stratified by gender and sessions per week did not meaningfully differ from the full sample. Third, as with any survey, response bias may influence the validity of the results. Our response rate of 74% is much higher than for most surveys of healthcare workers, mitigating potential non-response bi- as. Last, our study relied on survey measures of burnout

Table 3 Concurrent Validity: Associations with Team Culture and Workplace Atmosphere for MBI and Self-Defined Burnout Measures

Clinicians Staff

MBI burnout All clinicians (N = 444)

Burned out (N = 231)

Not burned out (N = 213)

Unadjusted odds ratio

All staff (N = 606)

Burned out (N = 277)

Not burned out (N = 329)

Unadjusted odds ratio

N (column %) OR (95% CI) N (column %) OR (95% CI) Team culture (score of 7 or greater on a 10-point scale)

277 (62.4%) 117 (50.7%)

160 (75.1%)

0.34 (0.23–0.51)

330 (54.5%)

105 (37.9%)

225 (68.4%)

0.28 (0.20–0.39)

Atmosphere in your primary work area (4 or 5 on a 5-point scale, 5 being Bhectic, chaotic^)

168 (37.8%) 119 (51.5%)

49 (23.0%)

3.56 (2.36–5.36)

301 (49.7%)

181 (65.3%)

120 (36.5%)

3.29 (2.35–4.59)

Self-defined burnout All clinicians (N = 432)

Burned out (N = 125)

Not burned out (N = 307)

Unadjusted odds ratio

All staff (N = 579)

Burned out (N = 181)

Not burned out (N = 398)

Unadjusted odds ratio

Team culture (score of 7 or greater on a 10-point scale)

271 (62.7%) 55 (44.0%) 216 (70.4%)

0.33 (0.22–0.51)

317 (54.8%)

62 (34.3%) 255 (64.1%)

0.29 (0.20–0.42)

Atmosphere in your primary work area (4 or 5 on a 5-point scale, 5 being Bhectic, chaotic^)

163 (37.7%) 71 (56.8%) 92 (30.0%)

3.07 (1.99–4.72)

288 (49.7%)

133 (73.5%)

155 (38.9%)

4.34 (2.95–6.39)

1349Knox et al.: Clinician and Staff Burnout MeasuresJGIM

and did not assess well-being with direct observation or qualitative methods. The self-defined burnout measure and MBI each have

advantages and disadvantages. We do not conclude from our study that there is necessarily a preferred burnout survey instrument. However, researchers and health sys- tem leaders addressing burnout must be aware of the measures’ different properties and lack of equivalency for assessing burnout prevalence. We recommend that organizations such as the National Academy of Medicine and Agency for Healthcare Research and Quality take the lead in

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.