Lord Mansfield is reputed to have said “Decide promptly, but never give any reasons. Your decisions may be right, but your reasons are sure to be wrong.” I hope his admonition is correct because I get a sinking feeling each time I see another list of what form the Commonwealth’s new assessment should take.
The reasons for the sinking feeling are many. Here I will deal with just three.
The first has to do with the technical aspects of whatever form the new assessment takes. I believe the substantive ideas behind the assessment, not its technical requirements, should drive it. Yet, good technical advice early can save time, money and embarrassment. In the past the Commonwealth has been blessed with unusually good technical advice from panels that were composed of assessment persons who were in the mainstream of the assessment world. They gave dispassionate advice that vendors or persons not in the mainstream could not be expected to give. There should be technical assistance early and often as plans for a new system develop.
There are a number of proposals that make me wonder about the basis for them. To take just one: that formative assessment instruments could or should be provided by vendors. I have taught test and measurement classes for over 30 years and I know that I can teach teachers to make better tests than are presently being used (apart from CATS) in the Commonwealth.
Some very important things go on when one constructs a test. One needs to think seriously about standards, instructional materials, and the learner as the items are written and tests developed. Having gotten the results one has to think seriously about whether students have not learned because they have not been taught well or because the items are no good. Each of these activities is related to effective instruction. None of them come with already made assessments.
I would like to hear the discussion of whether difficulties with the writing portfolio are unique to that form of assessment or whether having assessments in any accountability context corrupts them regardless of their form. Are some kinds of assessments less corruptible than others? Are there practices that are less likely to be gamed by those who seek higher scores at the expense of better learning? Are there ways to gather and report learning outcomes that depend less on massive formal testing procedures? I believe the answer to each of these questions is “yes.” And I think one could give good reasons for answering these questions in the affirmative.
And so, I will enter the fray. Below I am presumptuous enough to come up with a set of principles that should guide the design of the new Commonwealth assessment. I made the mistake of including things we have learned as my reasons for the recommendations. I wonder if there is a Lord Mansfield redux.
New Assessment Principles
After almost two decades of high-stakes assessment in the Commonwealth it is time to step back and decide how what we learned can help us think about the new directions we should take.
1) We learned that assessments can be very time consuming and very costly. We learned that they may not be as cost effective as we hoped.
A new statewide assessment should be:
a. Clear about each of its purposes
b. Less time consuming
c. Less costly
2) We learned that teachers believe they spend far too much time testing and preparing for testing.
A new statewide assessment should
a. Carefully delineate what should be assessed statewide and what should be assessed locally and be controlled by teachers
b. Be restricted to a small but crucial part of the curriculum
3) We learned that parents wish to know how their children’s scores compare with other children’s scores nationwide. We know that off-the-shelf tests have user norms not nationally representative ones. We know that the National Assessment of Educational Progress (NAEP) is the only national test with a nationally representative sample.
A new statewide assessment should
a. Use wisely the NAEP state assessments
b. Link Kentucky assessments to NAEP
c. Provide students with scores that can be compared to national ones
4) We learned that measuring so many content areas at so many grades in all schools is an inefficient way to assess Kentucky’s educational progress.
A new statewide assessment should:
a. Sample schools to track progress over time
b. Accept NAEP scores in mathematics, reading and science as progress indicators
5) We learned that testing requirements of No Child Left Behind (NCLB) have a profound effect on what and how a state assesses.
A new statewide assessment should be
a. Cognizant of new requirements of NCLB
b. Prepared to seek exemptions from some requirements of NCLB
6) We learned that others admired Kentucky’s assessment when it was bold and pioneering
A new statewide assessment should
a. Lead other states and nations in producing useable information about students and schools
b. Emphasize formative and instructionally embedded assessments over summative ones
c. Place students and teachers in the center of schools and assessment
I would like to hear the discussion of whether difficulties with the writing portfolio are unique to that form of assessment or whether having assessments in any accountability context corrupts them regardless of their form. Are some kinds of assessments less corruptible than others? Are there practices that are less likely to be gamed by those who seek higher scores at the expense of better learning? Are there ways to gather and report learning outcomes that depend less on massive formal testing procedures? I believe the answer to each of these questions is “yes.” And I think one could give good reasons for answering these questions in the affirmative.
And so, I will enter the fray. Below I am presumptuous enough to come up with a set of principles that should guide the design of the new Commonwealth assessment. I made the mistake of including things we have learned as my reasons for the recommendations. I wonder if there is a Lord Mansfield redux.
New Assessment Principles
After almost two decades of high-stakes assessment in the Commonwealth it is time to step back and decide how what we learned can help us think about the new directions we should take.
1) We learned that assessments can be very time consuming and very costly. We learned that they may not be as cost effective as we hoped.
A new statewide assessment should be:
a. Clear about each of its purposes
b. Less time consuming
c. Less costly
2) We learned that teachers believe they spend far too much time testing and preparing for testing.
A new statewide assessment should
a. Carefully delineate what should be assessed statewide and what should be assessed locally and be controlled by teachers
b. Be restricted to a small but crucial part of the curriculum
3) We learned that parents wish to know how their children’s scores compare with other children’s scores nationwide. We know that off-the-shelf tests have user norms not nationally representative ones. We know that the National Assessment of Educational Progress (NAEP) is the only national test with a nationally representative sample.
A new statewide assessment should
a. Use wisely the NAEP state assessments
b. Link Kentucky assessments to NAEP
c. Provide students with scores that can be compared to national ones
4) We learned that measuring so many content areas at so many grades in all schools is an inefficient way to assess Kentucky’s educational progress.
A new statewide assessment should:
a. Sample schools to track progress over time
b. Accept NAEP scores in mathematics, reading and science as progress indicators
5) We learned that testing requirements of No Child Left Behind (NCLB) have a profound effect on what and how a state assesses.
A new statewide assessment should be
a. Cognizant of new requirements of NCLB
b. Prepared to seek exemptions from some requirements of NCLB
6) We learned that others admired Kentucky’s assessment when it was bold and pioneering
A new statewide assessment should
a. Lead other states and nations in producing useable information about students and schools
b. Emphasize formative and instructionally embedded assessments over summative ones
c. Place students and teachers in the center of schools and assessment
No comments:
Post a Comment