Test for Outcome Evaluation in Nursing Education and Accommodations, Analyzing, Measurement Validity and Reliability

Accommodations for Test Taking, Analyzing Test Results, Concepts of Measurement, Validity of Test to be Conducted for Outcome Evaluation, Reliability of Test to be Conducted for Outcome Evaluation in Nursing Education.

Paper and Pencil Tests for Outcomes Evaluation in Nursing Education

When preparing for the test in paper and pencil format, the test should be easy to read and follow. Use standard font and type size; observe guidelines for page layout and use of white space. Space items evenly. Number items consecutively. Keep an item’s stem and options on the same page. Place introductory material (e.g., graph or chart) before an item (and make sure it reproduces clearly). Keep matching lists on the same page. Proofread the test after it is compiled but before it is duplicated. Print on one side only.

Maintaining an Appropriate Physical Test Taking Environment in Nursing Education

The physical environment should be conducive to the task. In a classroom, this includes adequate lighting, a comfortable temperature, sufficient workspace, and minimal interruptions. To reduce student anxiety, the faculty member should maintain a positive, nonthreatening attitude and avoid unnecessary conversation before and during the test.

Faculty should avoid giving unintentional hints to individual students who ask for clarification of questions during the test. When administering the test in an online environment, faculty must determine if the test will be administered in a proctored testing center in the school of nursing or on the campus or if the test will be administered to students on their own computer and during a specified period. The student should have a comfortable chair, desk, and computer, as well as sufficient space.

Maintaining a Secure Test Environment in Nursing Education

In recent years, maintaining a secure test environment and ensuring academic honesty has become a challenge (DiBartolo & Walsh, 2010; Klocko, 2014). Some suggestions for preventing cheating on tests administered in the classroom include the following:

1. Maintain test security (e.g., lock up copies of the test). Number tests and make sure all copies of tests are returned to the instructor before students leave the classroom.
2. Modify tests from one semester to another.
3. State the consequences of cheating early in the instructional process and inform students of academic honesty policies at the time of testing.
4. Have students sign an honor pledge that has been written on the test (or precedes an online test) that the work during the test is their own.
5. Require that book bags, cell phones, and items of clothing such as hats in which “cheat sheets” can be stored be brought to the front of the room.
6. Proctor students consistently throughout testing; Depending on class size, additional proctors may be necessary. Have a clear plan about how proctors are to manage cheating when it is observed or suspected.
7. Designate special seating arrangements (e.g., have an empty chair between students; have students sit in an assigned order).
8. Use alternative versions of the test.
9. Use alternative answer forms (e.g., listing responses down the page, listing responses across the page).

Accommodations for Test Taking in Nursing Education

Students with disabilities or circumstances that require special consideration may request accommodations for test taking.

Considerations may be given for the following reasons: allowing extra time for taking the test, needing additional or extra time for rest breaks, requiring adaptations for using a computer such as reading a computer screen (need for increased font size or color change) or for use of a trackball mouse, having hearing or other auditory disabilities, having visual or eyesight difficulties (such as requiring a Braille, large print), requiring a quiet space with limited distractions, having a reader to read or record test taker responses, needing a sign language interpreter, or having a medical device in the room.

At a college or university, the policies for accommodations for classroom tests are established by the faculty and made public. Students must have documented need for accommodations on file with the office that deals with Americans with Disabilities Act compliance and follow established procedures for implementing the request for accommodation for taking a test.

For other tests, such as licensing exams or certification exams the policies are set by the test administration service (state board of nursing in the case of the PN or RN licensing exam) and the test-taker must make the request in writing and have a letter from an appropriate health professional confirming the diagnosis or disability. Approval or disapproval of these requests may be determined based on the effect the disability has on the candidates’ ability to practice nursing in the state

Analyzing Test Results in Nursing Education

Once the test has been administered and scored, faculty should review the results using concepts of measurement and data analysis. Based on these findings, faculty assign grades. Faculty at most schools of nursing have access to test scoring services that calculate test statistics and provide item analysis.

Although there may be a fee associated with the service, the data provided by test scoring services is helpful, particularly for the first few times the test is used. Faculty should seek the assistance of these services and the consultation that can be obtained at testing centers.

Concepts of Measurement in Nursing Education

A variety of metrics are used to determine the effectiveness of a test. These include validity, reliability, and measures of central tendency.

Validity of Test to be Conducted for Outcome Evaluation in Nursing Education

The concept of validity refers to the appropriateness, meaningfulness, and usefulness of inferences made from test scores. Validity is the judgment made about a test’s ability to measure what it is intended to measure. This judgment is based on three categories of evidence: content-related, criterion-related, and construct-related.

Content related patterns of evidence should show that the test adequately samples relevant content. In nursing education, the relevant content is defined by nurse educators, the course, and the profession. Some examples of content-related evidence of validity correspondence of the test content with the following:

1. Specifications table
2. Professional judgments of peers
3. Core material as defined by professional organizations
4. Standards of care as defined by agencies and professional organizations

Criterion related patterns of evidence should show that the test adequately measures performance, either concurrently or predictively. The performance must be compared with some criterion variables. Nurse educators may use performance on the licensing examination (NCLEX-RN or NCLEX-PN) as the criterion variable (pass or fail).

Construct related patterns of evidence should show a relationship between test performance and some “quality” to be measured. This is a broad category of evidence that must include specifics about the test (from the content and criterion categories) in addition to a description of the quality or construct being measured.

Some of the factors that may adversely affect test validity are unclear directions, inconsistent or inadequate sampling from the table of specifications, poorly written test items, and subjective scoring (McDonald, 2013). Careful preparation of the test can improve test validity.

Reliability of Test to be Conducted for Outcome Evaluation in Nursing Education

Reliability refers to the ability of a test to provide dependable and consistent scores. A judgment about reliability can be made based on the extent to which two similar measures agree. Reliability is a necessary but not sufficient condition for validity. However, reliability may be high even with no validity. Nurse educators look for evidence to judge tests as both reliable and valid.

Among the factors that may adversely affect test reliability are insufficient length and insufficient group variability. For the purpose of increasing reliability, a minimum test length of 25 multiple choice questions with an item difficulty sufficient to ensure heterogeneous performance of the group is sufficient for classroom testing.

Reliability could be measured by giving the same test to the same group and noting the correspondence (test-retest method) or by giving “equivalent” tests to the same group. Both of these methods have major disadvantages for classroom testing and are not generally used by nurse educators.

Reliability can be measured with a single test administration using either the split half or internal consistency method. The split-half method separately scores answers to odd and even questions and then compares the “odd-question” score to the “even-question” score. The internal consistency of a test can be calculated using one of the Kuder Richardson formulas (McDonald, 2013).

Reliability is measured on a scale of 0 to 1.00. A reliability coefficient of 1.00 represents 100% correspondence between two tests or measures. Many standardized tests have reliability coefficients of 0.9 or higher. Good test reliability is greater than 0.80; acceptable test reliability is 0.70 to 0.80; poor test reliability is less than 0.70 (Tarrant & Ware, 2012).

Measures of test reliability are based on the assumption that all students had adequate time to answer all questions and that all test items are of about the same difficulty. Because the reliability coefficient works better when the variability of scores is maximized, tests administered to smaller groups of students (N) may have lower reliability coefficients.