Avoiding Potential Bias in Evaluation in Nursing Education and Hotspot (Rollover) Questions, Drag-and-Drop and Ordered Response, Graphic Items
Hotspot (Rollover) Questions, Drag-and-Drop and Ordered Response, Graphic Items, Avoiding Potential Bias in Test Items for Outcomes Evaluation in Nursing Education
Hotspot (Rollover) Questions for Outcomes Evaluation in Nursing Education
Hotspot questions ask the test-taker to locate a specific “spot” on a chart or figure. Hotspot questions use a scenario, a stem and directions to identify a particular “spot” on a diagram or illustration.
When the questions are developed to be administered in paper-and pencil formats the faculty can develop and number four options and place four “spots” on the figure (one correct, three incorrect spots) and ask the test taker to identify the number of the correct “spot.”
Hotspot questions are more effectively developed for computer-administered examinations in which the test-taker can use the mouse to roll over the correct spot on the chart or figure.
Advantages of Hotspot (Rollover) Questions
Hotspot questions provide an easy way to test understanding and application of anatomical knowledge. They offer an effective way to test understanding of physical assessment techniques. They can be used to test procedures and nursing skills with correct or incorrect positioning.
Disadvantages of Hotspot (Rollover) Questions
Hotspot questions are most effective in computer applications where a mouse can be used to roll over to identify the hotspot. Hotspot questions may be more difficult to develop because of the need to have an illustration as a reference point.
A client has not been voided for 10 hours. Identify the anatomical area where the nurse should assess for bladder distension.
Drag-and-Drop and Ordered Response Questions
Ordered-response questions require the test-taker to place information in a specified order. For example, questions can be developed to ask the test-taker to put steps of a procedure in order or, given a set of clients, to determine priorities for nursing care.
These items have a scenario and a stem, and the options, all of which are correct, must be placed in the correct order. On the licensing examination, candidates drag each response in the left column and drop it into the correct order in the right column.
Advantages of Drag-and-Drop and Ordered Response Questions
Ordered-response questions evaluate test-takers’ understanding of the order for steps of a procedure or how to set priorities for a client or groups of clients. They are relatively easy to develop. When writing for paper-and-pencil tests, the order can be “scrambled”, and the item writer can pose four or more possible sequences.
Disadvantages of Drag-and-Drop and Ordered Response Questions
Ordering all the steps or priorities may be confusing. Steps, sequences, and priorities may be controversial or context-specific; thus, the item needs to be structured such that experts will agree on the correct order, correct answer, and rationale.
A client begins to have a seizure. The nurse should do which actions in order from first to last?
- Note the time.
- Notify the physician.
- Protect the client from injury.
- Obtain a history of events prior to the seizure.
Graphic Items for Outcomes Evaluation in Nursing Education
Graphic items use photographs or illustrations in the question or in the answer options (graphic response). This type of question has a scenario, stem, and options. The test taker responds to details in the graphic to answer the question or chooses the answer by selecting the correct answer from four different graphics.
Advantages of Graphic Items for Evaluation
Graphic items test assessment and evaluation skills, and clinical decision making.
Disadvantages of Graphic Items for Evaluation
Graphic items require the use of art; expense could be involved to obtain images of incorrect responses. They may take time to develop.
The nurse is evaluating a client who has just been instructed on how to walk with crutches. The nurse should instruct the client (see figure) to:
- Lean forward 30 degrees.
- Pad the tops of the crutches.
- Hold the arms on the rungs of the crutches as shown.
- Lower your head to look for objects on the floor.
Audio and Video Items for Outcomes Evaluation in Nursing Education
In audio and video items, an audio or video clip is used as a stimulus in a part of the question or answer. Audio and video items use a scenario, stem, and answers. The audio or video clip is included in the scenario; the student clicks on the icon that brings up the audio file.
The National Council Licensure Examination–Registered Nurse (NCLEX-RN) exam currently uses audio item types and has considered the use of video item types (Wendt & Harmes, 2009a, 2009b).
Advantages
Audio and video items test students’ ability to identify sounds or respond to video information and make a clinical judgment based on information provided in the audio or video file. Test questions can be written at higher levels of the cognitive domain and require students to synthesize data from the scenario and the data provided in the audio or video file. These questions test competencies that may be difficult to assess with other test item formats.
Disadvantages
Audio and video items can only be administered in a computer managed environment that supports access to the audio and video files. Finding access to no-cost or low-cost files may be difficult; purchase of files may be necessary as copyright protections must be observed.
The nurse is assessing the breath sounds of a client who is hospitalized with bacterial pneumonia. Click here to listen to the breathing sounds. [The sounds are coarse crackles.] The nurse should do which first? [Note: The correct answer depends on the test-taker correctly identifying the breath sound and selecting the appropriate nursing action.]
- Have the client take a deep breath and cough to expectorate retained secretions.
- Encourage the client to drink one glass of fluids every hour.
- Check the blood levels of the antibiotic the client is receiving.
- Tell the client the breath sounds are clear and to continue the deep-breathing exercises.
Avoiding Potential Bias in Test Items for Outcomes Evaluation in Nursing Education
Students of equal ability should have an equal probability of correctly answering a test item. If systematic differences in responses to an item exist among members of groups, independent of total scores, then the item may be biased. Bosher (2003) classifies potential areas of bias in test items into four categories of bias: test wise flaws, irrelevant difficulty, linguistic bias, and cultural bias.
Test wise flaws are those errors in items that provide cues to the correct answer within the item or test itself. These flaws potentially provide an unfair advantage to students with more test-taking experience or training and to native English speakers who can more easily recognize grammatical and other cues. Items with irrelevant difficulty may be missed for reasons related more to format than to content.
This type of bias can occur from writing, unclear stems, providing superfluous information, or using negative phrasing. Linguistic complexity, grammatical errors, and inconsistent word use also may result in biased items. Linguistic modification involves writing short and clearly understood questions (Abedi, 2014).
Faculty can reduce the reading load of test items by eliminating irrelevant information, using simple words, and short sentences. Faculty also should avoid using idioms or “slang” that may only be understood by certain groups of students. Faculty can also reduce linguistic load in test questions by using the present tense and active voice.
Faculty must also guard against cultural bias when constructing test items (Hicks, 2011). These items depend on culturally specific knowledge and should not be used unless cultural practices per se are the domain of the question. The guidelines for writing articles are designed to avoid bias.