Executive Summary

  • 3 broad purposes of assessment
    • assist learning
    • measure individual achievement
    • evaluate programs
  • the more purposes a single assessment aims to serve, the more each purpose will be compromised

Assessment is always a process of reasoning from evidence.

  • assessment is always imprecise and is an estimate of what a person knows or can do.

3 pillars of Assessment

  • a model of how learners represent knowledge and develop competence in the domain (cognition)
  • tasks or situations that allow one to observe learners' performance (observation)
  • interpretation method for drawing inferences from the performance evidence obtained (interpretation)

All three foundations - the assessment triangle - must be explicitly connected and designed as a whole.

  • A model of learning and cognition should underlie the assessment process and should be based on the best available understanding of how learners represent knowledge and develop competence in a domain
  • educational assessment does not exist in isolation; it must be aligned with curriculum and instruction to support learning
  • aspects of learning assessed in the classroom should align with learning targeted by large-scale assessment, but these two are often out of alignment.

Science of thinking and learning

  • need to focus beyond discrete bits of knowledge to encompass more complex aspects of achievement
  • cognitive architecture includes
    • working or short-term memory (highly limited)
    • long-term memory (virtually limitless)
  • salient point is how to access and evoke long-term memories, so emphasis should be placed on how people organize LTM
  • assessments should evaluate schemas an individual has and under what circumstances those schemas are considered relevant
  • assessments should attempt to determine whether an individual has good metacognitive skills
  • learning entails transformation from naïve to more complete and accurate comprehension, so assessments should focus on making thinking visible to both teachers and learners so that instructional strategies can be selected to support future learning
  • timely and informative feedback is critical
  • because learning happens through discourse and interaction and in a particular context, assessments should encompass how learners engage in the discourse

Contributions of Measurement and Statistical Modeling to Assessment

  • certain kinds of assessments require the capabilities of formal statistical models for the interpretation element of the assessment triangle
    • high-stakes assessments
    • assessment of remote users (interpreters without day-to-day interaction with students)
    • complex models of learning
    • large volumes of data
  • it is now possible to characterize learner achievement in terms of multiple aspects of proficiency, rather than a single score
    • chart progress over time
    • offer multiple paths or alternative methods of valued performance
    • model, monitor, and improve judgements on the basis of informed evaluations
    • model performance at student, group, class, school, and state levels.
  • Many newer models are not used b/c they are not understood or accessible to those w/o a strong technical background
    • technology can help overcome this

Implications of the New Foundations for Assessment Design

  • design of high quality assessments is iterative and interdependent, not linear and sequential
  • model of cognition and learning is central
    • based on empirical studies of learners in the domain
    • provides a developmental perspective
    • shows typical ways in which learners progress toward competence
  • interpretation model must fit the cognition and learning model
    • interpretation techniques in the absence of a rich learning model will fail to produce meaningful interpretations
    • rich learning models are impoverished in the absence of interpretive techniques or data
  • well-delineated descriptors of learning in the domain are key to being able to communicate effectively about the nature of learner performance

Assessment in Practice

  • the power of classroom assessment is in its close connection between instructions and the teachers' knowledge of the learners' instructional history
  • large-scale assessments can communicate across time and space, but, because of the lack of local context, have limited utility in the classroom
  • classroom assessment and large-scale assessment contrast because of their different purposes and contexts
  • teachers need theoretical and practical training to implement effective formative assessment practices

Information Technologies

  • improved assessment practices must link the three pillars of cognition, observation, and interpretations
  • technology can extend the nature of the problems presented to learners
  • allows greater data collection

Recommendations

  • Recommendation 1: Accumulated knowledge and ongoing advances from the merger of the cognitive and measurement sciences should be synthesized and made available in usable forms to multiple educational constituencies.
  • Recommendation 2: Funding should be provided for a major program of research, guided by a synthesis of cognitive and measurement principles, focused on the design of assessments that yield more valid and fair inferences about student achievement.
  • Recommendation 3: Research should be conducted to explore how new forms of assessment can be made practical for use in classroom and large-scale contexts and how various new forms of assessment affect student learning, teacher practice, and educational decision making.
  • Recommendation 4: Funding should be provided for in-depth analyses of the critical elements (cognition, observation, and interpretation) underlying the design of existing assessments that have attempted to integrate cognitive and measurement principles
  • Recommendation 5: Federal agencies and private-sector organizations concerned with issues of assessment should support the establishment of multidisciplinary discourse communities.
  • Recommendation 9: Instruction in how students learn and how learning can be assessed should be a major component of teacher preservice and professional development programs.
  • Recommendation 10: Policy makers are urged to recognize the limitations of current assessments, and to support the development of new systems of multiple assessments that would improve their ability to make decisions about education programs and the allocation of resources.
  • Recommendation 11: The balance of mandates and resources should be shifted from an emphasis on external forms of assessment to an increased emphasis on classroom formative assessment designed to assist learning.