Moss, P. A. (2005). Reconceptualizing Validity for Classroom Assessment. Educational Measurement: Issues and Practice, 22(4), 13–25. https://doi.org/10/bkxm42
This article explores the shortcomings of conventional validity theory for guiding classroom assessment practice and suggests additional theoretical resources from sociocultural theory and hermeneutics to complement and challenge conventional theory. To illuminate these concerns and possibilities in a concrete context, the author uses her own classroom experience in teaching a qualitative research methods course. The importance of examining cases of assessment practice in context for developing, teaching, and evaluating validity theory is discussed.
Keywords: classroom assessment, hermeneutics, sociocultural theory, validity
The question I consider in this article is to what extent does our understanding of validity in the measurement profession “assure that the relevant issues are addressed in classroom assessment and what role might other theoretical perspectives play in providing a more robust validity framework to guide thinking and action?
The following assumptions are considered:
Psychometric characterizations of learning infer learning from observed changes in individuals' performances over time.
Sociocultural perspective suggests that learning is perceived through changing relationships among the learner, other human participants, and the material and symbolic tools available.
Author's thinking about validity informed by interpretive social sciens, esp philosophical hermeneutics. Like psychometrics, hermeneutics interprets human products, expressions or actions by combining information across data sources and evidence [
assessment as research].
conventionally, validity is conceptualized as referring to an inference or interpretation, and a use or action based on a test score
The 1999 Standards defines validity as “the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests” (p. 9). the validity argument focuses on an interpretation or action based on an instrument. While this focus is sometimes relevant and useful, it is both too small and too large for most of the decisions I need to make.
authentic learner work makes the standardization infeasible
As Wenger suggests: One problem of the traditional classroom format is that it both too disconnected from the world and too uniform to support meaningful forms of identification. It offers unusually little texture to negotiate identities: a teacher sticking out and a flat group of students all learning the same thing at the same time. Competence, thus stripped of its social complexity, means pleasing the teacher, raising your hands first, getting good grades (p. 269)
consistent with a sociocultural prespective, the most appropriate unit of analysis is the social situation which entails the recursive relationship between person and context ... and claims about individuals must be grounded in interaction
As Mehan notes, By moving beyond the states and traits of individuals to social situations as the unit of analysis, it does not blame low achieving students’ school difficulties on their lack of motivation, diminished linguistic skills, or deficient cognitive styles. . , . [Students’performances can be] recast as collaboratively constructed and continuously embedded in face-to-face interaction in social environments. (pp. 251, 254)”
This idea moves the focus from an individual blog post or other artifact to the network of others' ideas that is demonstrated, critiqued, interpreted, and assimilated into an individual's way of being. This network can become visible in the hyperlinks between posts and ideas on a networked course, both within and outside any particular cohort.
Having multiple sources/pieces of evidence to inform a consequential interpretation/decision is a fundamental feature of the epistemology and ethics in any of the social science perspectives that I have encountered.
however, even the testing Standards hedge on this...
In educational settings, a decision or characterization that will have major impact on a student should not he made on the basis of a single test score. Other relevant information should he taken into account if it will enhance the overall validity of the decision. (p. 146)14
Whatever one’s definition of validity, with classroom assessment, understanding these effects is crucial to sound practice. I might go so far as to argue that validity in classroom assessment-where the focus is on enhancing students’ learning-is primarily about consequences. Assuming interpretations are intended to inform instructional decisions and that instructional decisions entail interpretations about students’ learning, it is on evidence of their (immediate, long-range, and cumulative) effects on which their validity primarily rests.