Developing Measurement Theory for Classroom Assessment Purposes and Uses

Brookhart, S. M. (2003). Developing Measurement Theory for Classroom Assessment Purposes and Uses. Educational Measurement: Issues and Practice, 22(4), 5–12. https://doi.org/10/dj7bxr

Abstract

In many fields of inquiry, the need for new theoretical developments is often best seen in areas of strain, and strain is apparent in several areas in which the conventions of measurement theory not quite "fit" classroom assessment. Three areas of do strain are analyzed in order to suggest how theoretical development might focus directly on information quality in the classroom assessment context. article describes the conThis text dependence of classroom assessment, its inextricable relationship instruction, and its simultaneous formative and with summative functions. Thus a case is made for new theoretical developments in the area of measurement in the classroom. (p. 5)

Keywords: classroom assessment, reliability, validity (p. 5)

Key Ideas

Classroom Assessment Environment

  • Classroom assessment occurs in the specific context of the local classroom environment and this context affects the assessment.
    • methods used
    • criteria for selecting particular assessments
    • quality of assessment
    • Ts use of feedback
    • Ts preparation and background in assessments
    • Ts perceptions of students
  • Ss contribute to assessment environment
    • may be involved in design
    • S perceptions are linked to the assessment experience
  • Large-scale assessment does not emanate from S daily learning experiences
    • does not come from a T with who a S has a relationship
    • Ss are not primary consumers of the results

The difference between decontextualized, one-time measurement done by outsiders and the ongoing, contextualized measurement done with a teacher changes teh nature of the measure.

Integration of Assessment and instruction

  • Good classroom assessments are not only measures of learning, but genuine episodes of learning themselves.
    • Ss receive different levels of help and guidance when assessment is part of instruction
    • measurement theory is not designed to apply to the assessment/instruction mix
    • that Ss may have help from Ts and peers on a project implies that the grade applies not only to the outcome, but also the process of 'studenting'
    • in large-scale assessment, outside help is cheating and a threat to validity and reliability
    • in constructivist classroom assessment/instruction, this help puts Ss in the ZPD
  • measurement theorists remove assessment from instruction
    • a unit test is an 'after-learning' checkup that is separate from the instruction that preceded it; this is not consistent with classroom realities
  • integration makes the test a part of the fabric of student learning

This is a fundamental difference between classroom and large-scale assessment where the major function of the assessment is to provide someone besides the student with interpretable information.

Formative and Summative Purposes

Formative - using evaluation in the development or improvement of some educational process
Summative - using evaluation in decision-making about the end result of an educational process

formative
  • cyclical
    • Ss task is to compare 'ideal' and 'actual' performance
    • act to close the gap
    • compare 'ideal' and 'improved' performance
  • 3 things need to happen before Ss can improve
    • develop a concept of the learning goal
    • originally the teacher's, but gradually internalized
    • S ability to monitor their own work and compare actual and desired performance
    • S ability to act in a way to close the gap between actual and ideal performance
    • involves setting their own goals
  • Ts responsible for providing feedback that Ss need to do these three things...this is called 'formative assessment'

Student is always at the centre of the process b/c only the student can act to improve performance. The teacher cannot learn for the student.

  • formative assessment is part of learning because Ss come to understand the standard of quality work to which theirs is compared.
  • summative assessment - the assessor is the only one who needs to understand the standard
    • changes the relationship between measurers and the measured

Summative assessment requires that teachers (or other assessors) become members of a community of practice, while formative assessment requires that the learners become members of the same community of practice (Wiliam, 1998)
Wiliam, D. (1998, September) Enculturating learners into communities of practice: Raising achievement through classroom assessment. Paper presented at the European Conference for Educational Research, University of Ljubljana, Slovenia.

Measurement Theory for classroom assessment

Table 1. (P. 9)

Concepts in Large-Scale Assessment Concepts in Classroom Assessment
Validity: The measure is external to the inferences made and actions taken.
- Students are 'subjects' upon whom observations are made.
- The validity goal is a meaningful inference about student performance and/or effective use of that information fro a specified purpose.
Validity: Inferences made and actions taken are internal to the measurement process.
- Students are observers jointly with teachers; 'those measured' make the inferences and take the actions in the formative assessment process.
Students' awareness of and benefit from assessment information are part of the 'information' itself.
- The validity goal is an understanding of how students' work compares to 'ideal' work (as defined in the learning objectives) and/or effective use of that information for further learning.
Validity: The measurement context is construct-irrelevant
- Content specifications describe a domain.
- Administration can be standardized.
Scores can be equated or linked across contexts and forms of assessment.
Validity: The measurement context is construct-relevant.
Assessment is part of instruction. A good assessment is an 'episode of genuine learning'
- Content specifications reflect both the domain (learning objectives) and instructions (modes, activities)
- Teacher beliefs, teacher instructional practices, and teacher understanding of both the subject matter and students (including cultural and linguistic differences) are relevant validity concerns
Reliability is consistency over irrelevant factors
- Occassions, time, items, and/or tasks are facets of error variance.
- The reliability goal is stable ranking of students on a score scale (NRT) or stable categorization of students on an achievement continuum (CRT)
Reliability is sufficiency of information.
- The reliability goal is stable information about the gap between students' work and 'ideal' work as defined in students' and teachers' learning objectives