Evaluate the Results and Discussion sections of the accompanying article and critique how the following were addressed. Do not c

Effect of an E-mental Health Approach to Workers’ Health Surveillance versus Control Group on Work Functioning of Hospital Employees: A Cluster-RCT Sarah M. Ketelaar1*, Karen Nieuwenhuijsen1, Fania R. Gärtner1, Linda Bolier2, Odile Smeets2,

Judith K. Sluiter1

1 Coronel Institute of Occupational Health, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands, 2 Innovation Center of Mental Health and

Technology (I.COM), Trimbos Institute, Netherlands Institute of Mental Health and Addiction, Utrecht, The Netherlands

Abstract

Objective: To evaluate an e-mental health (EMH) approach to workers’ health surveillance (WHS) targeting work functioning (WF) and mental health (MH) of healthcare professionals in a randomised controlled trial.

Methods: Nurses and allied health professionals (N = 1140) were cluster-randomised at ward level to the intervention (IG) or control group (CG). The intervention consisted of two parts: (a) online screening and personalised feedback on impaired WF and MH, followed by (b) a tailored offer of self-help EMH interventions. CG received none of these parts. Primary outcome was impaired WF (Nurses Work Functioning Questionnaire), assessed at baseline and after three and six months. Analyses were performed in the positively screened subgroup (i) and in all participants (ii).

Results: Participation rate at baseline was 32% (NIG = 178; NCG = 188). Eighty-two percent screened positive for at least mild impairments in WF and/or MH (NIG = 139; NCG = 161). All IG-participants (N = 178) received part (a) of the intervention, nine participants (all positively screened, 6%) followed an EMH intervention to at least some extent. Regarding the subgroup of positively screened participants (i), both IG and CG improved over time regarding WF (non-significant between-group difference). After six months, 36% of positively screened IG-participants (18/50) had a relevant WF improvement compared to baseline, versus 28% (32/115) of positively screened CG-participants (non-significant difference). In the complete sample (ii), IG and CG improved over time but IG further improved between three and six months while CG did not (significant interaction effect).

Conclusions: In our study with a full compliance rate of 6% and substantial drop-out leading to a small and underpowered sample, we could not demonstrate that an EMH-approach to WHS is more effective to improve WF and MH than a control group. The effect found in the complete sample of participants is not easily interpreted. Reported results may be useful for future meta-analytic work.

Trial Registration: Dutch Trial Register NTR2786 http://www.trialregister.nl

Citation: Ketelaar SM, Nieuwenhuijsen K, Gärtner FR, Bolier L, Smeets O, et al. (2013) Effect of an E-mental Health Approach to Workers’ Health Surveillance versus Control Group on Work Functioning of Hospital Employees: A Cluster-RCT. PLoS ONE 8(9): e72546. doi:10.1371/journal.pone.0072546

Editor: Jim van Os, Maastricht University Medical Centre, The Netherlands

Received January 11, 2013; Accepted July 11, 2013; Published September 12, 2013

Copyright: � 2013 Ketelaar et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: The Mental Vitality @ Work trial was co-financed by a grant from the Dutch Foundation Institute Gak (URL: http://www.instituutgak.nl, PrevBGZ/project D) and a grant from The Netherlands Organisation for Health Research and Development (ZonMW) (URL: http://www.zonmw.nl/en, grant number 208010001). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing Interests: The e-mental health interventions used in this study are stand-alone interventions which were developed by the Trimbos Institute at an earlier stage. However, neither the authors working at the Trimbos Institute (LB and OS) nor the institute itself derive financial income from the interventions. The other authors have declared that no competing interests exist.

* E-mail: [email protected]

Introduction

Nurses have a high risk of developing common mental health

complaints, such as distress, depression, and anxiety [1–3].

Impaired mental health of employees in healthcare occupations

can have serious adverse effects, endangering the health and safety

of not only themselves but also their patients. A study by Gärtner

and colleagues found that impaired mental health in nurses and

allied health professionals affects several aspects of their work

functioning, including cognitive aspects (e.g. staying alert) and

causing incidents at work [4]. Another study by Letvak and

colleagues showed that depression in nurses was associated with

presenteeism, which is in turn associated with patient falls,

medication errors, and lower self-reported quality of care [5].

Adding to this, increased levels of psychological distress, even in a

mild form, have been found to be associated with an increased

likelihood of obtaining a disability pension in later life [6]. To

sustain nurses’ and allied health professionals’ mental health and

to enable them to remain healthy and well-functioning in their

profession until retirement age, it is crucial to periodically screen

PLOS ONE | www.plosone.org 1 September 2013 | Volume 8 | Issue 9 | e72546

these employees and provide interventions to improve their mental

health and work functioning.

A potentially promising method for the early detection of

impaired mental health and subsequent treatment in nurses and

allied health professionals, is offering a mental module for workers’

health surveillance (WHS). Although attention has been paid to

the occupational hazards of healthcare employees [7], WHS

targeting work functioning and mental health of nurses and allied

health professionals has, to our knowledge, not been reported

before.

WHS is an important component of occupational healthcare

[8]. It is a means to implement preventive action by identifying

and treating health complaints relevant to work, and it should be

an essential component of programmes aimed at the protection of

employees [9]. In the Netherlands, it has three aims: 1) to prevent

the onset, recurrence, or worsening of work-related diseases, 2) to

monitor and promote work-related health, and 3) to monitor and

improve work functioning and employability [10]. It can be used

to periodically monitor employees’ health and work functioning to

detect impairments early and to bring timely interventions into

action to prevent further impairment. It is recommended to apply

a job-specific assessment, to allow for tailoring of interventions to

the specific detected work functioning impairments as fitting as

possible [11]. In this study, we detect early signs of impaired

mental health and impaired work functioning in nurses and allied

health professionals, and offer interventions using an e-mental

health approach.

E-mental health (EMH) is the use of information and

communication technology, and in particular the many tech-

nologies related to the Internet, to support and improve mental

health [12]. Applying EMH might be a useful and feasible

approach to perform a mental module for WHS. Online

screening is a practical and efficient method to screen for self-

reported impaired work functioning and impaired mental

health. Furthermore, EMH offers possibilities regarding the

subsequent interventions. Ritterband and colleagues defined

Internet interventions as typically focused on behavioral issues,

aiming to institute behavior change and subsequent symptom

improvement, usually self-paced, interactive, and tailored to the

user, and making use of the multimedia format offered by the

Internet [13]. EMH interventions exist which target a wide

variety of common mental disorders such as depression, anxiety,

panic, phobias, and various addictions. Unguided self-help

EMH interventions have been found to have positive outcomes

for a variety of mental health aspects (e.g. Warmerdam et al.

[14]; Farrer et al. [15]; Riper et al. [16]; Blankers et al. [17];

Billings et al. [18]), although to our knowledge their effects on

work functioning have not been studied in a specific working

population such as nurses and allied health professionals.

Moreover, EMH interventions have thus far only been offered

as stand-alone interventions for a specific mental health

complaint. In our study, we offer a choice of EMH interven-

tions, tailored to the specific complaints as indicated by the

individual’s screening results.

In this paper, we study the effect of an EMH-approach to

WHS targeting work functioning and mental health of hospital-

employed nurses and allied health professionals, on their work

functioning, distress, work-related fatigue, posttraumatic stress,

and work ability in a cluster-randomized controlled trial. We

hypothesized that WHS, consisting of online screening on

impaired work functioning and impaired mental health

followed by personalised feedback and a tailored offer of self-

help EMH interventions, will improve work functioning and

mental health.

Methods

The protocol for this trial and supporting CONSORT checklist

are available as supporting information; see Checklist S1 and

Protocol S1 (http://www.biomedcentral.com/1471-2458/11/

290).

Ethics statement The Medical Ethics Committee of the Academic Medical

Center Amsterdam approved this study (for approved protocol see

Protocol S2). All participants gave their written informed consent

before taking part.

Study design The study was designed as a cluster-randomised trial with block

randomisation carried out at ward level. In order to guarantee

allocation concealment, randomisation was performed by one

researcher (KN) who was not involved in the practical recruitment

of employees, using the computer software program Nquery

Advisor with a block size of three. The complete trial included two

intervention groups and one control group [19]. The present study

compared one of the intervention groups, the e-mental health

approach (EMH-approach) group, to the control group. The other

intervention group consisted of an invitation for a preventive

consultation with an occupational physician. A pre-randomisation

procedure with incomplete-double-consent design was applied

[20], meaning that individuals were only informed about their own

group.

Outcome measures were obtained from all participants at

baseline (March 2011) and follow-up measures were obtained

three and six months after baseline.

The design, conduct and reporting of this study adhere to the

Consolidated Standards of Reporting Trials guidelines [21,22].

Details of the study design are reported elsewhere [19]. The trial

registration number of the study is NTR2786 (Dutch Trial

Register: http://www.trialregister.nl).

Participants The study population of the complete trial was formed by all

nurses, including surgical nurses and anaesthetic nurses, and allied

health professionals (such as physiotherapists and radiotherapists)

employed at one academic hospital in the Netherlands (N = 1731).

Nurses and allied health professionals form two large groups of

hospital employees, and many of their work demands and work

conditions are similar. Since it regarded a preventive study,

participants were included if they were not, or were not expecting

to be on sick leave for more than two weeks at baseline.

All eligible employees were invited for participation in the study.

To detect a clinically significant effect (effect size$.33), while

conducting the tests with alpha = .05 (two-tailed) and power = .80,

and allowing for possible cluster effects and loss to-follow-up, the

minimum required sample size was 718 participants for the

complete trial [19]. After randomization at ward level (N = 86), 29

wards with 579 employees were assigned to the EMH-approach

group and 29 wards with 561 employees to the control group

(Figure 1).

Procedure In March 2011, potential participants received an invitation by

e-mail to fill out the online baseline questionnaire which could be

filled out at any time during six weeks. It was possible to

discontinue the questionnaire and complete it after logging in

again. Three reminders were sent, as well as an information letter

to their home address. Those who had completed the baseline

E-mental Health Approach to WHS

PLOS ONE | www.plosone.org 2 September 2013 | Volume 8 | Issue 9 | e72546

questionnaire were invited to fill out the follow-up questionnaires

three and six months after baseline.

Intervention E-mental health approach group. At baseline, participants

in the EMH-approach group were screened on the following

aspects (for details see Table 1 and Gärtner and colleagues [19]):

impaired work functioning, distress, work-related fatigue, risky

drinking behaviour, depression including suicide risk, anxiety,

panic disorder, and posttraumatic stress. Participants received

personalised feedback on their screening results immediately after

filling out the baseline questionnaire, both onscreen and in an e-

mail.

The personalised feedback was followed by an invitation for a

tailored offer of self-help EMH interventions, on the basis of an

algorithm based on the specific symptoms and the work-

relatedness of the symptoms (available as supporting information,

see Algorithm S1). Participants were mostly offered a choice of two

or three EMH interventions to leave room for personal

preferences. Participants who screened negative on all mental

health complaints were invited to follow an EMH intervention

aimed at enhancing and retaining their mental fitness.

The EMH interventions used in this study are self-help

interventions on the Internet aimed at reducing specific mental

health complaints or enhancing wellbeing. The interventions are

mainly based on the principles of cognitive behavioural therapy

and combine a variety of aspects, e.g. providing information and

advice, weekly assignments, the option of keeping a diary and a

forum to get in contact with others who have similar complaints.

The EMH interventions were developed as stand-alone interven-

tions by the Trimbos Institute (Netherlands Institute of Mental

Figure 1. Flow of participants through the trial. doi:10.1371/journal.pone.0072546.g001

E-mental Health Approach to WHS

PLOS ONE | www.plosone.org 3 September 2013 | Volume 8 | Issue 9 | e72546

Health and Addiction) at an earlier stage. The following EMH

interventions were used in the study:

– Psyfit [23]: aimed at enhancing mental fitness. Also applied for

healthy participants.

– Strong at work [24]: aimed at gaining insight into work stress and

learning skills to cope with it.

– Colour your Life [25]: aimed at tackling depressive symptoms.

– Don’t Panic Online [26]: aimed at reducing panic symptoms for

subclinical and mild cases of panic disorder.

– Drinking Less [16]: aimed at reducing risky drinking behaviour.

Psyfit was found to be effective in decreasing symptoms of

depression and anxiety and improving well-being and vitality [27].

Warmerdam and colleagues [14] showed that Colour your Life

resulted in significantly lower depression and anxiety scores

compared to a waiting-list control group and to significantly

higher quality of life scores. The number of participants showing

clinically relevant change regarding depression after 12 weeks was

significantly higher. Spek and colleagues also found a significantly

larger improvement in depressive symptoms compared to a

waiting-list control group [28,29]. Drinking Less resulted in more

participants who reduced their drinking successfully to within

guideline norms, and a significantly larger decrease in mean

weekly alcohol consumption than a control group [16].

In case of positive screening on impaired work functioning

(regardless of their mental health status), participants received an

onscreen educational leaflet on how to improve their work

functioning (available upon request).

Control group. Participants in the control group filled out

the same baseline questionnaire as the EMH-approach group, but

did not receive an intervention, and thus no screening results

either. However, they were informed that they would receive

personalised feedback and a tailored offer of self-help EMH

interventions after six months, following the six months follow-up

questionnaire.

Measures All outcomes were measured at baseline and at three and six

months follow-up.

Primary outcome. The primary outcome of this study was

impaired work functioning, measured with the total score of the

Nurses Work Functioning Questionnaire (NWFQ) [4]. This

questionnaire has been developed to assess impaired work

functioning in nurses and allied health professionals. In the

screening phase, all seven of the original subscales were used.

Participants scored either green, orange or red on each subscale.

A red score on one or more subscales and/or three or more

orange scores led to case identification of impaired work

functioning (i.e. scoring above cut-off point on impaired work

functioning) [19].

Only six of the seven original NWFQ subscales were used for

the outcome measure, in contrast to what was described in the

trial’s design study [19], because the reproducibility of the impaired

decision-making subscale was found to be poor [30]. The total score

on the NWFQ was calculated with the 47 items of the remaining

six subscales, with a total score range of 0–100, a higher score

indicating more severely impaired work functioning.

The difference between the EMH-approach group and the

control group regarding impaired work functioning was investi-

gated using the continuous outcome and the percentage of

individuals who had improved relevantly at follow-up [31].

Secondary outcomes. The secondary outcomes included

distress, work-related fatigue, posttraumatic stress, and work

ability.

Distress was measured with the distress subscale of the Four-

Dimensional Symptoms Questionnaire (4DSQ) [32,33]. The 16-

item questionnaire uses a 5-point response scale (0 = no, 4 = very

often) and has a total score range of 0–32, a higher score indicating

a higher level of distress (cut-off point $11 [34]).

Work-related fatigue after working time was measured using the

need for recovery subscale of the Dutch Questionnaire on the

Experience and Evaluation of Work (QEEW) [35]. The 11-item

questionnaire with dichotomous response categories (yes, no) has a

total score range of 0–11 and a standardized score range of 0–100,

a higher score indicating a higher level of work-related fatigue (cut-

off point $54.5 [36]).

Posttraumatic stress was measured with the Dutch version of the

Impact of Event Scale [37,38]. The 15 items can be answered on a

4-point response scale (0 = not at all, 3 = often). Total scores range

from 0–75, a higher score indicating a higher level of posttrau-

matic stress (cut-off point $26 [39]).

Work ability was assessed with the first item of the Work Ability

Index (WAI) [40]. This item concerns the evaluation of current

work ability compared to their lifetime best on an 11-point scale

Table 1. Screening instruments and cut-off points.

Aspect Instrument Cut-off point

Impaired work functioning Nurses Work Functioning Questionnaire (7 subscales) [4] Red score on $1 subscales and/or orange score on $3 subscales [19]

Distress Four-Dimensional Symptoms Questionnaire, distress subscale [32,33] Total score$11 [34]

Work-related fatigue Need for recovery subscale of the Dutch Questionnaire on the Experience and Evaluation of Work [35]

Standardised total score$54.5 [36]

Risky drinking behaviour AUDIT-C [49] Total score$5 for men, $4 for women [50]

Depression Brief Symptom Inventory, depression subscale [51] Mean score$0.42 [52]

(Suicide risk) (One item from Brief Symptom Inventory, depression subscale [51]) (Score$3 on 0–4 scale)

Anxiety Brief Symptom Inventory, anxiety subscale [51] Mean score$0.42 [52]

Panic disorder Patient Health Questionnaire [53], only assessed for participants identified as having anxiety complaints

2 answers affirmative on the first 4 items plus 4 symptoms affirmative on the following 11 items [54]

Posttraumatic stress Dutch translation of the Impact of Event Scale [37,38] Total score$26 [39]

doi:10.1371/journal.pone.0072546.t001

E-mental Health Approach to WHS

PLOS ONE | www.plosone.org 4 September 2013 | Volume 8 | Issue 9 | e72546

(0 = completely unable to work, 10 = work ability at its best), a higher score

indicating a higher level of work ability.

Statistical analyses All participants who completed the baseline questionnaire and

who screened positive on impaired work functioning and/or

impaired mental health (the targeted sample) were analysed, as the

work functioning and mental health of these participants could be

expected to change due to the intervention. However, since this

was not pre-specified in the trial registration, the analyses were also

performed with the total sample of participants (i.e. all partici-

pants, regardless of their screening results).

To describe participants, we used the following demographics:

sex, age, occupation, specialization (yes/no), years of working

experience, working hours per week, and type of contract.

Additionally, the number of participants scoring above cut-off

point for impaired work functioning and mental health complaints

were calculated.

The analyses were performed at the level of the individual

employee, according to the intention-to-treat principle. The

significance level was set at a = .05. All analyses were carried out using the statistical package IBM SPSS Statistics 19.

Drop-out analysis. A drop-out analysis was performed to

detect whether dropping out of the trial was related to the

primary outcome impaired work functioning, and to identify

potential predictive variables of drop-out. Dropping out of the

trial was defined as completing the baseline and three months

follow-up questionnaires, but not the six months follow-up

questionnaire; or completing the baseline questionnaire, but

none of the follow-up questionnaires. Differences between drop-

outs and non drop-outs in impaired work functioning over time

in both separate groups were explored in graphs. If different

patterns of the effect after three months were detected, a Mann-

Whitney U test was performed to test the significance of the

differences. In the event of statistically significant differences, a

multiple logistic regression analysis was performed with drop-

out as the dependent variable, to identify potential predictive

variables for drop-out. Screening positive on mental health

complaints at baseline (yes/no) and age were included as the

independent variables, as we expected that these two aspects

might be related to dropping out of the trial. If the multiple

logistic regression analysis showed one or both of these aspects

to have a statistically significant effect on drop-out, they were

included as a covariate in the effect analyses [41].

Effect analysis. To analyse the differences over time

between the EMH-approach group and the control group on

each outcome, Linear Mixed Models (LMM) were applied. If the

assumption of a normal distribution of residuals was not met, a

log-transformation was used for the LMM and the median and

range were used to describe the outcome. Otherwise, the mean

and standard deviation were used to describe the outcome.

For each outcome, the scores at three and six months follow-up

were included as dependent variables in the LMM, while the

baseline score was included as a covariate. The main effects of

group and time of measurement, and the interaction of group*time of

measurement were included as fixed effects in the model. Ward (the

cluster level) and subject (the individual level) were included as

random effects; however if the cluster level did not have a

statistically significant effect, it was considered negligible and was

therefore excluded from the model. The effects of interest were the

main effect of group (interpreted as the difference between the

groups from baseline to six months follow-up) and the interaction

effect of group*time of measurement (interpreted as the difference

between the groups from three to six months follow-up).

For all outcomes in the positively screened subgroup, we

calculated Cohen’s d [42] by determining the mean difference

between the baseline score and the score at follow-up, divided by

the pooled standard deviation. For Cohen’s d, a score of 0.2 to 0.5

can be considered a small effect, 0.5 to 0.8 a medium effect, and

greater than 0.8 a large effect [42].

Additionally, the relative change scores of individuals on

impaired work functioning after three and after six months of

follow-up compared to their baseline score were calculated.

Individuals with a relative improvement on their NWFQ total

score of 40% or more, which is the minimal important change

(MIC) value of the NWFQ total scale [31], were defined as

relevantly improved. The percentages of individuals who had

improved relevantly in each group were compared using a Fisher’s

exact test, for both three months and six months follow-up.

Results

Participant flow Figure 1 presents the flow of participants through the trial. From

March 15 th

until April 26 th

, 423 employees (37%) started on the

baseline questionnaire. Of those, 366 (32% of invited employees)

were eligible for participation, 178 (31%) in the EMH-approach

group and 188 (34%) in the control group. In the EMH-approach

group, 80 participants (45%) were lost to follow-up, compared to

33 participants (18%) in the control group. Reasons for withdrawal

were not assessed. Fifty-six participants (31%) in the EMH-

approach group and 126 participants (67%) in the control group

completed all three questionnaires.

Analyses were performed on the participants who screened

positive (primary outcome: EMH N = 75, 54%; control N = 131,

81%), and additionally on all participants (primary outcome:

EMH N = 98, 55%; control N = 155, 82%) who had participated

in at least one follow-up.

Twenty-two participants (17 positively screened) logged into

Psyfit, seven logged into Strong at work, four logged into Colour your

Life, and no-one logged into Don’t Panic Online or Drinking Less. Nine

participants (all positively screened) followed an intervention to at

least some extent (Psyfit: 6, Colour your Life: 3).

Study population at baseline As shown in Table 2, the study groups were quite similar

regarding demographic and occupational characteristics. The

majority of participants were female and employed as a nurse.

Participants in the EMH-approach group had a younger average

age of 37, compared to 42 in the control group. The participants

worked an average of 31 hours per week and most of them had a

permanent position in the hospital. Around 4/5 th

of participants

screened positive on work functioning impairments and/or

impaired mental health, more participants in the control group

(N = 161, 86%) than in the EMH-approach group (N = 139, 78%).

Drop-out analysis Graphs in which the scores of drop-outs and non drop-outs on

the primary outcome were compared, showed that in both groups

drop-outs had a worse score on impaired work functioning (EMH

baseline median = 13, 3 mn follow-up median = 12; C baseline

median = 14, 3 mn follow-up median = 11) than non drop-outs

(EMH baseline median = 9, 3 mn follow-up median = 8; C

baseline median = 12, 3 mn follow-up median = 8) at baseline

and three months follow-up. A Mann-Whitney U test identified

that these differences were statistically significant in the EMH-

approach group (baseline U = 4.688, p = .01; 3 mn follow-up

U = 970, p = .04) and in the entire group of participants (baseline

E-mental Health Approach to WHS

PLOS ONE | www.plosone.org 5 September 2013 | Volume 8 | Issue 9 | e72546

T a

b le

2 .

P a

rt ic

ip a

n t

c h

a ra

c te

ri st

ic s

a t

b a

se li

n e

fo r

th e

E M

H -a

p p

ro a

c h

g ro

u p

a n

d th

e c

o n

tr o

l g

ro u

p .

T o

ta l

s a

m p

le P

o s

it iv

e ly

s c

re e

n e

d s

a m

p le

E M

H -a

p p

ro a

c h

(N =

1 7

8 )

C o

n tr

o l

(N =

1 8

8 )

E M

H -a

p p

ro a

c h

(N =

1 3

9 )

C o

n tr

o l

(N =

1 6

1 )

V a

ri a

b le

n (%

) n

(% )

n (%

) n

(% )

S e

x

F e

m a

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.