Influences on Validity In Assessment
A number of factors affect the validity of assessment results, including characteristics of the assessment itself, the administration and scoring procedures, and the test-takers. Teachers should be alert to these factors when constructing assessments or choosing published ones (Miller et al., 2009).
Characteristics of the Assessment
Many factors can prevent the assessment items or tasks from functioning as intended, thereby decreasing the validity of the interpretations from the assessment results. Such factors include unclear directions, ambiguous statements, inadequate time limits, oversampling of easy-to-assess aspects, too few assessment items, poor arrangement of assessment items, an obvious pattern of correct answers, and clerical errors in test construction (Miller et al. ., 2009). Ways to prevent test construction errors such as these will be addressed in the following chapters.
Assessment Administration and Scoring Factors
On teacher-made assessments, factors such as insufficient time, inconsistency in giving aid to students who ask questions during the assessment, cheating, and scoring errors may lower validity. On published assessments, an additional factor may be failure to follow the standard directions, including time limits (Miller et al., 2009).
Student Characteristics
Some invalid interpretations of assessment results are the result of personal factors that influence a student’s performance on the assessment. For example, a student may have had an emotionally upsetting event such as an auto accident or death in the family just prior to the assessment, test anxiety may prevent the student from performing according to true ability level, or the student may not be motivated to exert maximum effort on the assessment. These and similar factors may modify student responses on the assessment and distort the results, leading to lower validity (Miller et al., 2009).
Factors that impact or influence performance in a testing situation include client/patient/student factors, clinician factors, environmental factors, and those involving the actual testing process itself. This brief highlight these factors for both adult clients and children/students.
Factors that Influence Student and Institutional-Based Assessment
Challenges
While the assessments used in school settings have varying degrees of standardization and psychometric properties, most have more than acceptable levels of reliability and validity. So, when used appropriately with fidelity, assessments can be expected to provide useable information about students.
However, in addition to whether standardized testing is a good indicator of student performance, there is also debate on whether certain aspects in life—physical, mental, or situational—affect students’ assessment results. When working with educators, it is important that Clinicians and School Psychologists have a firm understanding of these factors in order to more accurately communicate the results obtained from assessment and other measures of student performance and behavior
Institutional Factors
- Does the student feel safe and/or connected at school?
- Does the student have a history of chronic absenteeism?
- Was there a fire drill or other unplanned emergency drill?
- Is the student markedly over- or under-age for the grade?
- Does the student have cultural support or disconnection?
- What is the student’s level of socialization and peer support?
- How well does the student relate to teachers and other school professionals?
- Was the child pulled from a class or school event he/she enjoys for the session?
- What is the testing environment (e.g., classroom or office lighting, noise level, temperature, number of interruptions)?
Influence of Rater/Teacher
- Cultural differences
> Language differences?
> Implicit basis?
- Rapport and management skills
- Previous experience with the student
> Amount and frequency
- Physical and mental state of the tester
- Fidelity to assessment qualifications and instructions
- Familiarity with the testing environment and test materials
- Interpretation experience/knowledge (e.g., discrepancy analysis, growth scale values, etc.)
Influence of Physical Factors
- Age level—The chronological age of a student is typically accounted for in test or assessment form selection
- Developmental level—Should be considered if a student is developmentally delayed or advanced when comparing to same-age cohorts
- Physical disabilities—Hearing loss, visual impairment, motor ability, etc.
- General health condition
> Had the student eaten that day?
> Did the student get enough sleep?
> Is there a history of substance abuse?
> What is their receptive and expressive language ability?
> Is the child on medication or off his/her normal medication?
> Is the student suffering from cold, flu, allergies, or other conditions?
Mental/behavioral factors
What are Student’s Factors
- Motivation level
- School learning ability level
- Special Ed, IEP, or 504 accommodations
- Activity/attention level (e.g., impulsive, distracted, etc.)
- Cognitive level (developmentally delayed or advanced)
- Specific mental health history (including comorbid or previous diagnoses, past trauma)
- Behavior (e.g., agitated, noncompliant, aggressive, shy, withdrawn, compulsive, anxious, frustrated, insecure, rejected, etc.)