March 7, 2022

Create graphic representations of four evaluation models. Use MS Word to create your graphic. Your graphic should include:

Descriptions of each overall model
Key components of each overall model
An important figure or figures in the development of each model
Significant ways each model has been used
Potential uses of each model in health informatics

These representations will be for your use in your upcoming course project, so the greater the detail, the more useful these representations will be to you.

Potential formats could include but are not limited to, tables, mind maps, Venn diagrams, or concept maps.

References attached.

HANDf~OOk:[ OF EVALUATION PII~TPlODS

11 Overview of Assessment Methods

In principle all aspects o f a system are candidates for assessment in all phases o f the system's development. In practice, some aspects are more prominent in some o f the phases than in others. During its life cycle, the assessment m a y change in nature from being prognostic (during planning), to screening and diagnosing (prior to switching over to daily operation), to treating (in the handling o f known error situations or shortcomings). Be aware, therefore, that even if a method is not listed under a specific phase, an information need may arise that requires inspiration from the methods listed under other phases.

Note that few o f the references given include a discussion o f the weaknesses, perils, and pitfalls o f the method described.

6.1 Overview of Assessment Methods" Explorative Phase

The methods included in this section are particularly relevant to the assessment o f issues raised during the establishment o f a User Re- quirements Specification, such as objectives, requirements, and expectations.

Method

Analysis of Work Procedures

Assessment of Bids

Balanced Scorecard

BIKVA

Areas of application

Elucidation of how things are actually carried out within an organization.

Comparative assessment of a number of offers from one or more bidders/vendors.

, . .

Ongoing optimization of the outcome of a development project by balancing focus areas by means of a set of indicators for a set of strategic objectives.

. .

Critical, subjective assessment of an existing practice.

Page no

Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science & Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691. Created from waldenu on 2022-03-04 00:20:40.

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

IdANDBOOI< OF: EVALUATION M{]TIdODS

Delphi

FieM Study

Focus Group Interview

Future Workshop

Grounded Theory

Heuristic Evaluation

Interview (no ns tandardiz ed)

KUBI

Logical Framework Approach

Organizational Readiness

�9 (Qualitative) assessment o f an e f f e c t – for instance, where the solution space is otherwise too big to handle

�9 Exploration o f development trends �9 Elucidation o f a problem a r e a – for instance,

prior to strategic planning.

Observation o f an organization to identify its practice and to clarify mechanisms controlling change.

This is in principle used for the same purposes as other interview methods. In practice, the method is most relevant during the early Explorative P h a s e – for instance, where attitudes or problems o f social groups need elucidation or when a model solution is being established.

Evaluation and analysis o f an (existing) situation in order to identify and focus on areas for change – that is, aiming at designing future practices.

Supportive analytical method for data acquisition methods that generate textual data, such as some open questionnaire methods and interviews (individual and group interviews).

This is used when no other realizable possibilities e x i s t – for instance, when:

�9 The organization does not have the necessary time or expertise

�9 There are no formalized methods �9 There is nothing tangible to assess yet.

This is particularly suited for elucidation o f individuals' opinions, attitudes, and perceptions regarding phenomena and observations.

Optimization of the outcome o f a long-term development project, based on a set o f user or customer/client-defined value norms and objectives.

Situation analysis to support the choice o f focus for a development but at the same time a simple technique for incorporation o f risk handling within project planning.

Assessment o f the readiness o f a healthcare organization for a clinical information system.

106

111

116

125

128

132

142

147

149

154

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

NANDi~OOK O~ EVALUATION I"']gTBtODS

Pardizipp

Questionnaire (no ns tandardiz ed)

Requirements Assessment

Risk Assessment

Social Network Analysis

Stakeholder Analysis

SWOT

Usability

Videorecording

WHO: Framework for Assessment of Strategies

Preparation of future scenarios.

Questionnaires are used to answer a wide range of questions, but its main area of application is (qualitative) investigations of subjective aspects requiting a high level of accuracy.

Within the European culture the User Requirements Specification is the basis for purchasing an IT-based solution or engaging in a development project. Consequently, the User Requirements Specification is a highly significant legal document that needs thorough assessment.

Identification and subsequent monitoring of risk factors, making it possible to take preemptive action.

Assessment of relations between elements within an organization (such as individuals, professions, departments or other organizations), which influence the acceptance and use of an IT-based solution.

Assessment of stakeholder features and their inner dynamics, aiming to identify participants for the completion of a given task, problem-solving activity, or project.

Situation analysis: establishment of a holistic view of a situation or a model solution.

Assessment of user friendliness in terms of ergonomic and cognitive aspects of the interaction (dialogue) between an IT system and its users. In this phase the concern is a planning or purchasing situation.

Monitoring and documenting as a means of analysis of what/how the work procedures or the users' activities are actually carried out or for investigation of complex patterns of interaction.

Assessment of different (development) strategies either individually or as a comparative analysis.

156

163

180

185

190

192

196

207

219

222

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

ldANDSOOI< O5 EVALUATION M~TIdODS

6.2 Overview of Assessment Methods" Technical Development Phase

The m e t h o d s listed in this section are particularly suited to user activities during the d e v e l o p m e n t and installation o f an IT-based solution and m a y be used to provide feed-back for the technical development.

A s s e s s m e n t in this phase is typically carried out under experimental conditions and not during real operation. The phase is usually completed with a technical verification to make certain that all necessary functions and features are present and work properly in compliance with the established agreement.

Method

Balanced Scorecard

Clinical~Diagnostic Performance

Cognitive Assessment

Cognitive Walkthrough

Areas of application

Ongoing optimization of the outcome of a development project by balancing focus areas by means of a set of indicators for a set of strategic objectives.

Measurement of diagnostic 'correctness' (for instance, measures of accuracy and precision) of IT-based expert systems and decision-support systems. '

Assessment of cognitive aspects of the interaction between an IT system and its u s e r s – for instance:

�9 Identification of where and why operational errors occur

�9 Identification of areas to be focused on for improvement in user friendliness.

Assessment of user 'friendliness' on the basis of system design, from specifications, muck-ups, or prototypes, aimed at judging how well the system complies with the users' way of thinking for instance:

�9 Identification of where and why operational errors occur

�9 Identification of causes behind problems with respect to user friendliness and consequently identification of areas for improvement.

Page no

102

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

I-IANID~OOI< O~ ~VALUATION IVIETIdODS

Heuristic Evaluation

Risk Assessment

SWOT

Technical Verification

Think Aloud

Usability

This is used when no other realizable possibilities e x i s t – for instance, when:

�9 The organization does not have the necessary time or expertise

�9 There are no formalized methods �9 There is not something tangible to assess yet.

Identification and subsequent monitoring of risk factors, making it possible to take preemptive action.

Situation analysis: establishment of a holistic view of a situation or a model solution.

Verification that the agreed functions are present, and work correctly and in compliance with the agreement. This may take place, for instance, in connection with delivery of an IT system or prior to daily operations and at any subsequent change of the IT system (releases, versions, and patches).

An instrument for gaining insight into the cognitive processes as feed-back to the implementation and adaptation of IT-based systems.

Assessment of user friendliness in terms of ergonomic and cognitive aspects of the interaction (dialogue) between an IT system and its users.

132

185

196

199

204

207

6.3 Overview of Assessment Methods" Adaptation Phase

In this phase, evaluation has the purpose o f providing support for the modification or refinement o f the IT-based solution, work procedures, and functions implemented within the IT system to make them work optimally as a whole during daily operations. This phase should be fairly short, provided that the implemented solution is functioning well from the beginning.

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

ldANDBOOI( OlZ EVALUATION IvlI~TI-IODS

N o w that real operational a s s e s s m e n t can take place, e r g o n o m i c , cognitive, and functionality a s s e s s m e n t will gain m u c h m o r e focus, as potential i n a d e q u a c i e s or s h o r t c o m i n g s will s h o w t h e m s e l v e s as operational errors, misuse, or the like.

Method

Analysis of Work Procedures

BIKVA

Clinical~Diagnostic Performance

Cognitive Assessment

Cognitive Walkthrough

Equity Implementation Model

Field Study

Focus Group Interview

Areas of appfication

Elucidation of how things are actually carried out, in comparison with the expected. This includes the actual use of the IT system in relation to its anticipated use.

Critical, subjective assessment of an existing practice.

Measurement of diagnostic ' correctness' (for instance, measures of accuracy and precision) in IT-based expert systems and decision-support systems.

Assessment of cognitive aspects of the interaction between an IT system and its u s e r s – for instance:

�9 Identification of where and why operational errors occur

�9 Identification of areas to be focused on for improvement in user friendliness.

�9 Identification of where and why operational errors occur

�9 Identification of causes behind problems with respect to user friendliness and consequently identification of areas for improvement.

Examine users' reaction to the implementation of a new system, focusing on the impact of the changes such a system brings about for the users.

Observation of an organization to identify its practices and to expose mechanisms that control change.

This is in principle used for the same purposes as other interview methods. In practice, the method is most relevant during the early Explorative P h a s e – for instance, where the attitudes or problems of social groups need elucidation or when a model solution is being established.

Page no

102

109

111

116

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

I-IANDBOOI< OF– ~VALUATION METHODS

Functionality Assessment

Grounded Theory

Heuristic Evaluation

Interview (nonstandardized)

Prospective Time Series

Questionnaire (nonstandardized)

RCT, Randomized Controlled Trial

Risk Assessment

Root Causes Analysis

1. Validation of fulfillment of objectives (realization of objectives) – that is, the degree of compliance between the desired effect and the actual solution

2. Impact Assessment (also called effect assessment) 3. Identification of problems in the relationship

between work procedures and the IT system's functional solution

The method will expose severe ergonomic and cognitive problems, but it is not dedicated to capture details of this type.

Supportive analytical method for data acquisition methods that generate textual data, such as some open questionnaire methods and interviews (individual and group interviews).

This is used when no other realizable possibilities e x i s t – for instance, when:

�9 The organization does not have the necessary time or expertise

�9 There are no formalized methods �9 There is not something tangible to assess yet.

Is in particular suited for the elucidation of individual opinions, attitudes, and perceptions regarding phenomena and observations.

Measurement of development trends, including the effect of an intervention.

Questionnaires are used to answer a wide range of questions, but its main area of application is (qualitative) investigations of subjective aspects requiting a high level of accuracy.

Verification of efficacy – that is, that the IT system – under ideal conditions – makes a difference to patient care. Particularly used in studies of decision-support systems and expert systems.

Identification and subsequent monitoring of risk factors, making it possible to take preemptive action.

Exploration of what, how, and why a given incident occurred to identify the root cause of undesirable events.

120

128

132

142

159

163

172

185

188

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

HANDBOOI( OiZ EVALUATION METHODS

Social Network Analysis

SWOT

Technical Verification

Think Aloud

Usability

User Acceptance and Satisfaction

Videorecording

Assessment of relations between elements within an organization (such as individuals, professions, departments, or other organizations), which influence the acceptance and use of an IT-based solution.

Situation analysis: establishing a holistic view of a situation or a model solution.

An instrument for gaining insight into the cognitive processes as feed-back to the implementation and adaptation of IT-based systems.

Assessment of user friendliness in terms of ergonomic and cognitive aspects of the interaction (dialogue) between an IT system and its users.

Assessment of user opinion, attitudes, and perception of an IT system during daily operation.

Monitoring and documenting as a means of analyzing how work procedures and user activities, respectively, are actually carried out or for investigation of complex patterns of interaction.

190

196

199

204

207

215

219

6.4 Overview of Assessment Methods" Evolution Phase

The starting point in time o f this phase is usually considered to be when the entire IT-based solution has reached a state o f sufficient stability with respect to bugs and corrections and w h e n evolutionary activities are started. Consequently, the shi~ be twe e n this and the previous phase m a y be fluid.

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

I-IANDE~OOI< (3[= EVALUATION METHODS

Method

Analysis of Work Procedures

Balanced Scorecard

BIKVA

Clin&al/Diagnostic Performance

Cognitive Assessment

Cognitive Walkthrough

Delphi

Equity Implementation Model

FieM Study

Areas of application

Elucidation of how things are actually carried out, in comparison with the expected. This includes its use in relation to measures of effect.

Ongoing optimization of the outcome of a development project by balancing focus areas by means of a set of indicators for a set of strategic objectives.

Critical, subjective assessment of an existing practice.

Measurement of diagnostic ' correctness' (for instance, measures of accuracy and precision) of IT-based expert systems and decision-support systems.

Assessment of the cognitive aspects of the interaction between an IT system and its u s e r s – for instance:

�9 Identification of where and why operational errors occur

�9 Identification of areas to be focused on for improvement in user friendliness.

Assessment of the user 'friendliness' on the basis of system design, from specifications, muck-ups, or prototypes of the system, aimed at judging how well the system complies with the users' way of t h i n k i n g – for instance:

�9 Identification of where and why operational errors occur

�9 Identification of causes behind problems with respect to user friendliness and consequently identification of areas for improvement.

1. (Qualitative) assessment of an e f f e c t – for instance, where the solution space is otherwise too big to handle

2. Exploration of development trends 3. Elucidation of a problem a r e a – for instance, prior

to strategic planning.

Examine users' reaction to the implementation of a new system, focusing on the impact of the changes such a system brings about for the users.

Observation of an organization to identify its practices and to expose mechanisms that control change.

Page no

102

106

109

111

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

HANDE~OOI< O.l::Z EVALUATION I"~I~TIdODS

Focus Group Interview

Functionality Assessment

Grounded Theory

Heuristic Evaluation

Impact Assessment

Interview (nonstandardized)

KUBI

Prospective Time Series

It is in principle used for the same purposes as other interview methods. In practice, the method is most relevant during the early analysis s t a g e – for instance, where attitudes or problems o f social groups need clarification or elucidation or when a model solution is being established.

1. Validation o f fulfillment o f objectives (realization o f objectives) – that is, the degree o f compliance between the desired effect and the actual solution

2. Impact Assessment (also called effect assessment) 3. Identification o f problems in the relationship

between work procedures and the IT system's functional solution

The method will expose severe ergonomic and cognitive problems, but it is not dedicated to capture details o f this type.

Supportive analytical method for data acquisition methods that generate textual data, such as some open questionnaire methods and interviews (individual and group interviews).

This is used when no other realizable possibilities e x i s t – for instance, when:

�9 The organization does not have the necessary time or expertise

�9 There are no formalized methods �9 There is not something tangible to assess yet.

Measurement o f the e f f e c t – that is, the consequence or impact in its broadest sense – o f an IT-based solution, with or without the original objective as a frame o f reference.

This is in particular suited for elucidation o f individual opinions, attitudes, and perceptions regarding phenomena and observations.

Optimization o f the outcome o f a long-term development project, based on a set o f user or customer/client defined value norms and objectives.

Measurement o f development trends, including the effect o f an intervention.

116

120

128

132

135

142

147

159

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

HANDE~OOk( O4Z ~VALUATION NI~TI4ODS

Questionnaire (nonstandardized)

RCT, Randomized Controlled Trial

Risk Assessment

Root Causes Analysis

Social Network Analysis

Stakeholder Analysis

SWOT

Technical Verification

Think Aloud

Usability

User Acceptance and Satisfaction

Questionnaires are used to answer a wide range of questions, but its main area of application is (qualitative) investigations of subjective aspects requiting a high level of accuracy.

Verification of efficacy – that is, that the IT s y s t e m – under ideal conditions – makes a difference to patient care. In particular used for studies of decision-support systems and expert systems.

Identification and subsequent monitoring of risk factors, making it possible to take preemptive action.

Exploration of what, how, and why a given incident occurred to identify the root cause of undesirable events.

Assessment of stakeholder features and their inner dynamics, aiming to identify participants for the completion of a given task, problem-solving activity, or project.

Situation analysis: establishment of a holistic view of a situation or a model solution.

Verification that the agreed functions are present, work correctly, and are in compliance with the agreement. This may take place, for instance, in connection with delivery of an IT system or prior to daily operations, and at any subsequent change of the IT system (releases, versions, and patches).

An instrument for gaining insight into the cognitive processes as feed-back to the implementation and adaptation of IT-based systems.

Assessment of user friendliness in terms of ergonomic and cognitive aspects of the interaction (dialogue) between an IT system and its users.

Assessment of users' opinion, attitudes, and perception of an IT system at daily operation.

163

172

185

188

190

192

196

199

204

207

215

C o p yr

ig h t ©

2 0 0 6 . E

ls e vi

e r

S ci

e n ce

& T

e ch

n o lo

g y.

A ll

ri g h ts

r e

se rv

e d .

ldANDBOOI< OF EVALUATION M~TIdODS

Videorecording

WHO: Framework f o r Assessment o f Strategies

Monitoring and documenting as a means of analyzing how work procedures and user activities, respectively, are actually carried out or for investigation of complex patterns of interaction.

Assessment of different (development) strategies either individually or as a comparative analysis.

219

222

6.5 Other Useful Information

T h e r e is certain information that c a n n o t be c a t e g o r i z e d u n d e r ' m e t h o d s ' but that s h o u l d be included n e v e r t h e l e s s b e c a u s e an u n d e r s t a n d i n g o f these issues is valuable. In general, the areas o f application outlined in the table b e l o w are valid for all p h a s e s within the life cycle.

Information

Documentation in an Accreditation Situation

Measures and Metrics

Standards

Areas of application

Planning of assessment activities in connection with the purchase of a 'standard' IT system when the user organization is or considers becoming certified or accredited.

Measures and metrics are used throughout evaluation, irrespective of whether it is constructive or summative. Planning of an assessment/evaluation study includes the conversion of an evaluation purpose to specific measures and subsequent establishing metrics for their measurement.

A number of de facto and de jure standards exists, which each defines a series of issues such as the contents of a User Requirements Specification, verification of an IT system, quality aspects of an IT system, as well as roles and relations between a user organization and a vendor in connection with assessment.

Page no

227

232

238

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.

Create graphic representations of four evaluation models. Use MS Word to create your graphic. Your graphic should include:

Related Posts

A 41-year-old male patient presents at the community walk-in clinic with complaints of severe elbow pain radiating into the forearm

Describe your clinical experience for this week

CHAMPIONING GENDER AND SEXUALITY ALLYSHIP CASE STUDIES