top of page

Assessment

Introduction  |  Research Design  |  Analytics & Results
Client
National Council of State Boards of Nursing internal project
​
Duration
9 months
​
Role on Project
Principal Investigator
​
Skills Demonstrated

Survey development

Content validation

Predictive validation

Inter-rater reliability

Analytics

Introduction

This project was an evaluation of methods for assessing the competency of licensed nurses. 

 

Four popular competency assessment methods were selected to be studied: exam, certification, continuing education, and self-assessment. These assessment methods were compared against supervisor-rated competency (.82 inter-rater reliability); which included the following 8 competencies:

  • Pharmacological & parenteral therapies

  • Reduction of risk potential

  • Physiological adaptation

  • Management of care

  • Safety & infection control

  • Health promotion & maintenance

  • Psychosocial integrity

  • Basic care & comfort

Introduction

01. Research Design

Research Question
Are there relationships between exam performance, self-assessed competency, certification, continuing education hours, and supervisor-rated competency?

PRIMARY STUDY VARIABLES

Exploratory Design
Exploration of demographic variables and their relationship with exam performance and supervisor-rated competency
Quasi-experimental Design
2 group design (pass vs fail exam) with the following outcome measures:
  • Supervisor-rated competency
  • Self-assessed competency
  • Professional certification (yes vs no)
Sample
  • A convenience sample of nurses and their supervisors from 4 hospitals in the Chicago area participated
  • Approximately 300 matched nurse—supervisor pairs were obtained for analyses
  • Nurses were given a $100 gift card when they signed up and another $100 gift card when they came in for their scheduled exam time
  • Supervisors were given $25 gift card for every competency rating provided
Research Design

02. Analytics & Results

Exploratory Analyses

Chi-square

Exam performance (pass vs. fail)

ANOVA

Supervisor-rated competency

(continuous measure)

  • Facility (A vs. B vs. C vs. D)

  • Race/ethnicity (3 categories)

  • Age (4 categories)

  • Gender (M vs. F)

  • Years of experience (6 categories)

  • Education (4 categories)

  • Hours worked per week (3 categories)

  • Shift (5 categories)

Exploratory Analyses
Primary Analyses
Hierarchical Categorical Regression
ANOVA
Correlation

Exam performance (pass vs. fail)

Self-assessed competency (continuous)

Professional certification (yes vs. no)

Number of continuing education hours (continuous)

Supervisor

Ratings

Continous

Supervisor

Ratings

Research Report

Click to view the de-identified results

Analytics & Results
bottom of page