Kentucky’s Math And Reading Scores Show Improvement

This from WFPL:

Kentucky’s math and reading test scores show continued growth and, in some instances, exceed the national average, according to test scores from the National Assessment of Educational Progress (NAEP). The scores released this week measure 4th and 8th graders nationwide and Kentucky shows promise when compared to its peers.

That may be attributed to the education reforms made in 1990, said University of Louisville professor Sam Stringfield. They were likely the most successful state reforms of the last 20 years, he said. “And that the state has gone from being below the national average on all these measures, to virtually at or sometimes slightly above the national average, when you consider that we are still economically below the national average is really quite an accomplishment,” said Stringfield.

Earlier this year the Kentucky Department of Education (KDE) reported around two-thirds of students are proficient in reading. NAEP says that number is closer to one third. But Stringfield says this is normal. “And what is consistent across all 50 states, is that a higher percentage of students score at the proficient level on the state tests than on the NAEP tests,” said Stringfield.

Kentucky will be changing its state testing this year. KDE’s Lisa Gross said she expects scores in reading and math to drop slightly this year, while schools adjust to the changes. The largest improvements in Kentucky over the past decade have been in math scores, while the state’s reading scores have increased slowly, mirroring a national trend that doesn’t meet the approval of educators.

Others critics observe the disparity between average scores and scores for minority and low-income students, which remains consistently lower. But Kentucky should be proud with the progress it has made, said Stringfield. Where Kentucky was once behind, now it’s ahead, he said.

The Nation’s Report Card: Civics 2010

The National Assessment of Educational Progress (NAEP) releases The Nation’s Report Card: Civics 2010.

This report presents results of the National Assessment of Educational Progress (NAEP) 2010 civics assessment. National results for representative samples of students at grades 4, 8, and 12 are reported as average scale scores and as a percentage of students performing at or above three achievement levels: Basic, Proficient, and Advanced. Scores are also reported at selected percentiles, showing changes in the performance of lower-, middle-, and higher-performing students. Results for student demographic groups defined by various background characteristics (e.g., race/ethnicity, gender, and students’ eligibility for free or reduced-price school lunch) are included, as well as sample assessment questions with examples of student responses.

Results from the 2010 assessment are compared to those from two previous assessments in 1998 and 2006. The Technical Notes and appendix tables provide information on NAEP samples, school and student participation rates, and the exclusion and accommodation of students with disabilities and English language learners.

In comparison to the last assessment in 2006, average scores in 2010 were higher at grade 4, not significantly different at grade 8, and lower at grade 12. Gains for Hispanic students from 1998 to 2010 contributed to a narrowing of the White–Hispanic score gaps at all three grades. The percentage of students performing at or above the Proficient level in 2010 was 27 percent at grade four, 22 percent at grade eight, and 24 percent at grade twelve.

Major findings from the 2010 report include:

• The average score for fourth-graders was higher than in previous assessments in 2006 and 1998. The average score for eighth-graders in 2010 was not significantly different from either 2006 or 1998.

• At grade 12, the average score in 2010 was lower than in the 2006 assessment, but not significantly different from the average score in 1998.

• Average scores for Hispanic students in grade 8 were higher in 2010 than in 2006; average scores for Hispanic students at all three grade levels were higher than in 1998.

• Average scores for students eligible for either free or reduced price school lunch were higher in 2010 than in 2006 at both grades 4 and 8; additionally, a higher percentage of fourth-graders eligible for free lunch performed at or above Basic than in 2006.

• The average score for female fourth-graders increased since 2006; there were no significant changes in score for male fourth-graders since 2006.

• The percentage of twelfth-grade English language learners performing at or above Basic was lower in 2010 than in 2006, but not significantly different from the percentages in 1998.

Kentucky NAEP Science Scores Outpace National Averages

The results of the 2009 National Assessment of Educational Progress (NAEP) in science show that Kentucky's 4th and 8th graders scored higher than the national averages, the Kentucky Department of Education announced today.

Scale scores for 4th and 8th graders in 2009 are significantly above the national average. The NAEP science grading scale ranges from 0 to 300.

KENTUCKY AND NATIONAL SCALE SCORES FOR NAEP SCIENCE

NOTE: The NAEP 2009 science results are based on new science frameworks, meaning that direct comparisons to prior years’ overall scale scores are not recommended.

Disaggregated scale scores also show that, in all but one category, Kentucky students outperformed the nation. The only Kentucky group that did not have higher scores than the nation was 8th-grade white students, whose average scale score was 159, compared to the national average of 161 for the same group.

NAEP SCIENCE SCALE SCORES BY FREE OR REDUCED-PRICE MEAL ELIGIBILITY

NAEP SCIENCE SCALE SCORES BY GENDERNAEP SCIENCE SCALE SCORES BY RACE
2003 marked the first year NAEP was administered in all states, as mandated in the federal No Child Left Behind Act. States applying for Title I funds must indicate that they plan to participate in NAEP.

KEY FINDINGS FOR 2009 NAEP SCIENCE

Grade 4:
In 2009, the average science score for 4th-grade students in Kentucky was 161. This was higher than that of the nation's public schools (149).

In 2009, the percentage of students in Kentucky who performed at or above Proficient was 45 percent. This was greater than that for the nation's public schools (32 percent).

In 2009, the percentage of students in Kentucky who performed at or above Basic was 83 percent. This was greater than that for the nation's public schools (71 percent).

Grade 8:
In 2009, the average science score for 8th-grade students in Kentucky was 156. This was higher than that of the nation's public schools (149).

In 2009, the percentage of students in Kentucky who performed at or above Proficient was 34 percent. This was greater than that for the nation's public schools (29 percent).

In 2009, the percentage of students in Kentucky who performed at or above Basic was 71 percent. This was greater than that for the nation's public schools (62 percent).

NAEP is a project of the National Center for Education Statistics (NCES), reporting on the academic achievement of elementary and secondary students in the United States. For more than 30 years, NAEP has been the country's only nationally representative and continuing survey of students' educational achievement. Because the national NAEP samples were not designed to support the reporting of state-level results, state NAEP samples were introduced in 1990 to provide participating states reliable data concerning the achievement of their students. The assessment is authorized by Congress, directed by the National Center for Education Statistics (NCES) and developed by Educational Testing Service of Princeton, New Jersey. Westat, Inc. of Rockville, Maryland, conducts sample selection and data collection.

NAEP protects the confidentiality of students, teachers and schools that participate by not reporting individual student, teacher, school or district data. NAEP provides results for major demographic groups, and states that meet NAEP reporting criteria are able to compare their results with both national results and the results of other states.

SOURCE: KDE Press release

Ravitch gives a Shout Out to Jefferson County

This from Diane Ravitch at Bridging Differences:

This year the federal testing program (the National Assessment of Educational Progress, or NAEP) expanded to include 18 urban districts. ...The new participants in 2009 testing include Baltimore; Detroit; Fresno, Calif.; Jefferson County, Ky.; Miami-Dade (County); Milwaukee; and Philadelphia.

NAEP is recognized as the gold standard of achievement assessment: its tests are excellent, combining multiple-choice questions with short-answer questions and constructed-response questions so that students actually have to explain what they know or how they would solve a problem or write a short essay. NAEP is a no-stakes exam; no teacher or student gets a reward or a punishment based on its results. And no one can prepare for NAEP because no one knows who will take it. For all these reasons, NAEP tends to be more trustworthy than state exams, which typically produce inflated scores.

In 4th grade, the highest-performing district is Charlotte, followed closely by Austin. The districts that have made the biggest gains since 2002 are Atlanta and Washington. The new additions to TUDA include some very low-performing districts, such as Baltimore, Detroit, Fresno, Milwaukee, and Philadelphia. The districts in which 50 percent or more of 4th grade students are "below basic" are Atlanta, D.C., Chicago, Baltimore, Fresno, Los Angeles, Milwaukee, Philadelphia, Cleveland, and Detroit. In Detroit, a depressing 73 percent of students in 4th grade are "below basic." Nationally, 34 percent of students in this grade are "below basic."

A demographer would probably see a close correlation between NAEP scores and poverty.

Eighth grade reading scores have been stagnant nationally. The overall reading scores for the nation in this grade were precisely the same in 2009 as they were in 1998. Among the TUDA districts, two saw significant improvements in this grade: Atlanta and Los Angeles.

Four urban districts are at the national average for this grade:

Austin, Charlotte, Jefferson County, and Miami-Dade. This is very impressive. These four districts seem to be doing something very right: Their profile for 8th grade students looks very similar to the national profile and is significantly higher than the large-city average. What is especially impressive about Miami-Dade is the strong performance of its Hispanic students in 8th grade; the white-Hispanic gap in Miami is only 12 points, compared with 28 points in New York City, 30 points in Houston, 31 points in Austin, and 32 points in Los Angeles....

Beshear to Appoint New Board Members to Implement Senate Bill 1

Over at Dr H's Blog, Education Commissioner Terry Holliday reminds us that Governor Steve Beshear will soon replace as many as seven members of the Kentucky Board of Education.

Those whose terms expired on the 14th of this month (and who typically serve until replaced) are: Chairman Joe Brothers, Vice Chair C.B. Akins and members Kaye Baird, Jeanne Huber Ferguson, Austin W. Moss, Judy H. Gibbons and Doug Hubbard.

That leaves Vice Chair Dorie Combs, Billy Harper, Brigitte B. Ramsey and David K. Karem whose terms expire 4/14/2012.

The new board will have the task of implementing Senate Bill 1 which calls for a new accountability system, internationally benchmarked standards, formative and diagnostic assessments, and thankfully, student growth measures.

Vision for Education Reform in Kentucky

The Commish says,

I have started the process of engaging all advisory councils in the dialogue concerning a new accountability system and strategic plan for the Kentucky Board of Education. The board will hold a strategic retreat prior to its June meeting... The key to our success in transforming education in Kentucky will reside in our ability to focus on a few goals with a few strategies that are done with precision and fidelity. These few goals will focus on the vision of every child proficient and prepared for success.

This vision will be measured by indicators of proficiency, growth and closing gaps among student groups.

Proficiency will be measured by the cohort graduation rate and our comparative position among states.

Growth will be measured by the increases in our annual percentage of high school graduates who are prepared for college and career as compared against other states.

Closing gaps will be measured by the decreases in gaps for the graduation and readiness rates among student groups in Kentucky as compared against other states.

Holliday sees the National Assessment of Educational Progress (NAEP) along with the Educational Planning and Assessment System (EPAS) system as the "ultimate measures" of Proficiency, Growth and Gap rates.

The NAEP is the only nationally representative longitudinal assessment of what America's students know and can do in various subject areas. Assessments are conducted periodically in mathematics, reading, science, writing, the arts, civics, economics, geography, and U.S. history.
Since NAEP assessments are administered uniformly using the same sets of test booklets across the nation, NAEP results serve as a common metric for all states and selected urban districts. The NAEP has been used to show how some states lowered their own standards under the pressure of NCLB's current accountability model; a model Education Secretary Arne Duncan says he wants to change.

The EPAS system is required in Kentucky but does not enjoy NAEP's nearly universal acceptance due to its contrived set of benchmarks. Testing experts, like Skip Kifer, say the Hierarchical Linear Modeling EPAS uses is statistically indefensible and creates flawed benckmarks because it ignores institutions. Students are part of institutions and should be modeled that way.

Holliday's strategic priorities are based on federal guidance provided by Race to the Top, State Fiscal Stabilization Funds, the proposed plans for the reauthorization of Elementary and Secondary Education Act (NCLB), SB 1, SB 168, SB 130 and other state statutes.

The new lingo is:

• Next Generation Learners
• Next Generation Professionals
• Next Generation Support Systems
• Next Generation Schools and Districts

In addition, Governor Steve Beshear has appointed a Transforming Education in Kentucky (TEK) Task Force which is scheduled to make recommendations for the 2011 legislative session.

Holliday has called for comments and suggestions from the public and the alphabet groups. The TEK Task Force will conduct community forums and focus groups in August to gather input.

Kentucky tops nation in reading progress

This from Toni Konz at C-J:

Kentucky is getting a gold star from the U.S. secretary of education for its reading progress.

The state’s fourth- and eighth-graders made greater gains in reading than any other state on the 2009 National Assessment of Educational Progress, according to results released Wednesday.

The assessment, commonly referred to as the “nation’s report card,” shows that Kentucky’s eighth-graders scored an average of 267 on a 500-point scale on the test, compared with the national average of 262. The state’s fourth- graders scored an average of 226, compared with the national average of 220.

Kentucky was the only state that made significant increases at both grade levels in 2009. While Indiana students also registered increases, they were not considered to be significant.

Kentucky fourth-graders ranked seventh among states for average reading scores, while eighth-graders ranked 14th.

“This is very good news for Kentucky,” said Terry Holliday, the state’s education commissioner.

“We’ve been focusing so heavily on literacy over the past few years by working directly with teachers to improve instruction and to give them the training they need. I think that focus is starting to pay off.”

A handful of states showed slight improvements at one grade level, but not both. And 38 states or jurisdictions showed no significant change at either grade level.

U.S. Secretary of Education Arne Duncan told The Courier-Journal in a telephone interview Wednesday that Kentucky should be proud of its results.



“Kentucky has done lots of work to help lead the way nationally in terms of setting standards, measuring progress and holding schools accountable,” he said. “All that work is paying off and you are seeing real increases in student achievement.

...In Kentucky, 7,489 fourth- and eighth-graders from 316 schools were randomly selected to take the test.

Of that number, approximately 2,800 students in both grade levels from Jefferson County Public Schools were tested, said Bob Rodosky, director of testing and accountability for the district.

“I am really encouraged by the state’s results. and I would like to believe that Jefferson County had something to do with the rise of those scores,” he said.

...Last fall, Kentucky’s fourth-graders also made gains on the math portion of NAEP, scoring 239 points, up from 235 in 2007. The national average was 239. Eighth-graders scored 279, the same as in 2007, but three points below the national average of 282.

Kentucky NAEP Scores Up Significantly

The results of the 2009 National Assessment of Educational Progress (NAEP) in reading show that Kentucky's 4th and 8th graders made significant gains when compared to the state's performance in previous NAEP assessments.

According to NAEP, Kentucky is one of three states that had a statistically significant increase in 4th-grade reading scores from 2007 to 2009. Kentucky is also one of nine states that had a statistically significant increase in 8th-grade reading scores from 2007 to 2009. Kentucky is the only state in the nation to report increases in both 4th- and 8th-grade reading scores.

Scale scores for Kentucky 4th and 8th graders in 2009 surpassed the national average. The NAEP grading scale ranges from 0 to 500.

Students' performance on NAEP fits into one of four categories: Below Basic, Basic, Proficient or Advanced. In Kentucky, in reading, 72 percent of 4th graders and 79 percent of 8th graders scored at the Basic level or above. Since 1998, Kentucky's results have remained steady, with minor gains and losses, but the 2009 results in Kentucky have significantly increased above the national average. Since 1998, Kentucky’s 4th graders have gained 8 points on the NAEP reading assessment, and 8th graders have gained 5 points. 2003 marked the first year NAEP was administered under the No Child Left Behind (NCLB) Act of 2001. Under NCLB, states applying for Title I funds must indicate that they plan to participate in NAEP. In 2009, all 50 states, along with the District of Columbia and Department of Defense schools, participated in the NAEP reading and mathematics assessments.

NAEP reports data based on whether accommodations were allowed in the testing process, meaning that students with disabilities and those considered to have limited English proficiency (LEP) can have special assistance in order to complete the test. Certain students may be excluded from the test based on teacher recommendations using students’ Individual Education Plans (IEPs). NAEP does not allow all the accommodations Kentucky students need and are allowed on the state's tests. Kentucky also offers an alternate assessment program for those students with the most severe disabilities, and NAEP does not offer an alternate assessment.

For more than 40 years, NAEP has been the country’s only nationally representative and continuing survey of students’ educational achievement. The assessment is authorized by Congress, directed by NCES and developed by Educational Testing Service of Princeton, New Jersey. Westat, Inc. of Rockville, Maryland, conducts sample selection and data collection.

NAEP protects the confidentiality of students, teachers and schools that participate by not reporting individual student, teacher or school data. NAEP provides results for major demographic groups, and states that meet NAEP reporting criteria are able to compare their results with both national results and the results of other states.

Kentucky's Lowered CATS Standards Remained Relatively High

Over at Prichard, Susan Weston has been looking into the lowering of academic standards in Kentucky circa 2007.

You will recall, that is the time Kentucky was switching from the old CATS to the new (and not improved) CATS - thanks largely to NCLB's strictures screwing up a fairly good (yet, far from perfect) accountability system.

A retrospective report from the National Center for Education Statistics allows us to more clearly see the degree of lowering that actually went on. Turns out, Kentucky's reading and mathematics Kentucky Core Content Test (KCCT) standards were indeed lowered, but Kentucky still had higher proficiency standards than many other states.

The NCES report allows each state to compare its criteria for proficiency with that of other states and determine whether the rigor of its proficiency standards as represented by NAEP scale equivalents changed from 2005 to 2007. The analysis allows NAEP to corroborate state-reported changes in student achievement and provide states with an indicator of the construct validity and generalizability of their test results.

Weston reports here and here,

In fourth grade reading, the study shows:

• 206 as the NAEP score identified as equivalent to our "old CATS" proficiency standard.
• 205 as the NAEP score equivalent to our "new CATS proficiency standard.

In eight grade mathematics, the study shows:

• 285 as the NAEP score equivalent to "old CATS" proficiency.
• 279 as the NAEP score equivalent to "new CATS" proficiency.

In effect, the report gives us a formal demonstration that we lowered the proficiency bar in 2007. The reading change was quite small, but the math change more substantial.

The report also shows Kentucky's 2005 standards to have been quite high in national perspective, with only 21 percent of the included states having higher standards in either subject.

KSN&C argued against lowering academic standards at the time.

Moving up in math!

This from Susan Weston at the Prichard Committee:

2009 NAEP scores released today show Kentucky fourth graders performing at the national average in mathematics. Given our history of being behind in math, it is a sweet, sweet thing to see. Two snapshots of the data are below... We've got a long road to travel, my friends, but this result is a milestone worth some mighty celebration.

Kentucky is one of eight states that had a statistically significant increase in 4th-grade mathematics scores from 2007 to 2009. Mathematics scores for Kentucky 8th graders were not significantly different from the national average.
Scale scores for Kentucky 4th and 8th graders in 2009 are at or near the national average. The NAEP grading scale ranges from 0 to 500.

Older students less successful on math NAEP

This from Ed Week by way of KSBA:

The proportion of 13-year-olds taking algebra has grown steadily for at least two decades, but the increase in the number of students taking harder mathematics classes is not translating into higher average math scores by 17-year-olds on the National Assessment of Educational Progress, according to long-term trend data released today.

Average scores for 9- and 13-year-olds in math on NAEP have risen since 2004, but scores in that subject for 17-year-olds have not budged significantly since then.

In fact, flat scores in math for the older students have persisted since the early 1970s. That’s despite the fact that the proportion of 13-year-olds taking algebra has more than doubled—from 13 percent to 30 percent—from 1986 to 2008.

Meanwhile, reading scores increased for 17-year-olds since 2004.

“If you ask me, what is the single most perplexing problem since I’ve gotten involved in education, that’s it,” Darvin M. Winick, the chairman of the National Assessment Governing Board, which oversees NAEP, said in a telephone interview. “The data is not only flat, but it is flat while the kids are taking more math.” ...

The story is slightly different for reading.

Reading scores increased 3 points, to 286, for 17-year-olds from 2004 to 2008, which is considered statistically significant. That differs from the math trend for 17-year-olds, in which the change in scores during that period is not considered significant. But, as is true with math, the reading scores for 17-year-olds in 2008 were not considered significantly higher than those in the early 1970s.

In both reading and math, average scores climbed a significant amount (by at least 2 points) for 9-year-olds and 13-year-olds. They increased 4 points, to 220, for 9-year-olds in reading and 4 points, to 243, in math. For 13-year-olds, scores increased 3 points, to 260, in reading and 3 points, to 281, in math...

The New Commonwealth Assessment

By Skip Kifer

Lord Mansfield is reputed to have said “Decide promptly, but never give any reasons. Your decisions may be right, but your reasons are sure to be wrong.” I hope his admonition is correct because I get a sinking feeling each time I see another list of what form the Commonwealth’s new assessment should take.

The reasons for the sinking feeling are many. Here I will deal with just three.

The first has to do with the technical aspects of whatever form the new assessment takes. I believe the substantive ideas behind the assessment, not its technical requirements, should drive it. Yet, good technical advice early can save time, money and embarrassment. In the past the Commonwealth has been blessed with unusually good technical advice from panels that were composed of assessment persons who were in the mainstream of the assessment world. They gave dispassionate advice that vendors or persons not in the mainstream could not be expected to give. There should be technical assistance early and often as plans for a new system develop.

There are a number of proposals that make me wonder about the basis for them. To take just one: that formative assessment instruments could or should be provided by vendors. I have taught test and measurement classes for over 30 years and I know that I can teach teachers to make better tests than are presently being used (apart from CATS) in the Commonwealth.

Some very important things go on when one constructs a test. One needs to think seriously about standards, instructional materials, and the learner as the items are written and tests developed. Having gotten the results one has to think seriously about whether students have not learned because they have not been taught well or because the items are no good. Each of these activities is related to effective instruction. None of them come with already made assessments.

I would like to hear the discussion of whether difficulties with the writing portfolio are unique to that form of assessment or whether having assessments in any accountability context corrupts them regardless of their form. Are some kinds of assessments less corruptible than others? Are there practices that are less likely to be gamed by those who seek higher scores at the expense of better learning? Are there ways to gather and report learning outcomes that depend less on massive formal testing procedures? I believe the answer to each of these questions is “yes.” And I think one could give good reasons for answering these questions in the affirmative.

And so, I will enter the fray. Below I am presumptuous enough to come up with a set of principles that should guide the design of the new Commonwealth assessment. I made the mistake of including things we have learned as my reasons for the recommendations. I wonder if there is a Lord Mansfield redux.

New Assessment Principles

After almost two decades of high-stakes assessment in the Commonwealth it is time to step back and decide how what we learned can help us think about the new directions we should take.

1) We learned that assessments can be very time consuming and very costly. We learned that they may not be as cost effective as we hoped.

A new statewide assessment should be:
a. Clear about each of its purposes
b. Less time consuming
c. Less costly

2) We learned that teachers believe they spend far too much time testing and preparing for testing.

A new statewide assessment should
a. Carefully delineate what should be assessed statewide and what should be assessed locally and be controlled by teachers
b. Be restricted to a small but crucial part of the curriculum

3) We learned that parents wish to know how their children’s scores compare with other children’s scores nationwide. We know that off-the-shelf tests have user norms not nationally representative ones. We know that the National Assessment of Educational Progress (NAEP) is the only national test with a nationally representative sample.

A new statewide assessment should
a. Use wisely the NAEP state assessments
b. Link Kentucky assessments to NAEP
c. Provide students with scores that can be compared to national ones

4) We learned that measuring so many content areas at so many grades in all schools is an inefficient way to assess Kentucky’s educational progress.

A new statewide assessment should:
a. Sample schools to track progress over time
b. Accept NAEP scores in mathematics, reading and science as progress indicators

5) We learned that testing requirements of No Child Left Behind (NCLB) have a profound effect on what and how a state assesses.

A new statewide assessment should be
a. Cognizant of new requirements of NCLB
b. Prepared to seek exemptions from some requirements of NCLB

6) We learned that others admired Kentucky’s assessment when it was bold and pioneering

A new statewide assessment should

a. Lead other states and nations in producing useable information about students and schools
b. Emphasize formative and instructionally embedded assessments over summative ones
c. Place students and teachers in the center of schools and assessment

Recalculating The 8th-Grade Algebra Rush

This from Jay Mathews at the Washington Post:

Nobody writing about schools has been a bigger supporter of getting more students into eighth-grade algebra than I have been. I wrote a two-part series for the front page six years ago that pointed out how important it is to be able to handle algebra's abstractions and unknown quantities before starting high school. I have argued that we should rate middle schools by the percentage of students who complete Algebra I by eighth grade.

Now, because of a startling study being released today, I am having second thoughts.
Tom Loveless, director of the Brown Center on Education Policy at the Brookings Institution, has looked at the worst math students, those scoring in the bottom 10th on the National Assessment of Educational Progress eighth-grade test. He discovered that 28.6 percent of them -- let me make that clear: nearly three out of every 10 -- were enrolled in first-year algebra, geometry or second-year algebra.

Almost all were grossly misplaced, probably because of the push to get kids into algebra sooner...

Jack Jennings on the new CEP Study

KSN&C recently posted a story on the new report from the Center on Education Policy, "Has Student Achievement Increased Since 2002?"

CEP presents what is at least the fourth study in the past two years to confirm the significant progress being made in Kentucky schools - one of only four states to show moderate-to-large gains in both reading and math.

Richard Innes, a Bluegrass Institute researcher, doubted the credibility of the study.

Last year, a Kentucky Long-Term Policy Research Center study revealed that Kentucky made across-the-board improvements in its rank among the states; growing from roughly from 43rd to the 34th.

The Bluegrass Institute doubted the credibility of that study, too.

The KLTPRC report mirrored two previous studies done outside of Kentucky. Kentucky was 34th in Education Week's Quality Counts 2007 Achievement Index and was 31st in the Morgan Quinto 2006-2007 Smartest State Index."

I assume BGI didn't think much of those studies either.

When Richard questioned the quality of this study, I contacted Jack Jennings, President of CEP for a comment.

Here's his response:

Mr. Day, you sent a message on Wednesday saying that a commentator had a question about the report we released on Tuesday about student achievement. In
particular, he claimed our report would have credibility problems because Kentucky had a decline in 8th grade reading on NAEP that had not been reflected in the results on the state’s test. I asked the authors of the report to look at this issue and to respond. This is what they say:

First, this is consistent with the CEP report. Table 4-F (in the national summary) shows a moderate-to-large gain on the state assessment but a moderate-to-large
decrease on NAEP for grade 8 reading.

In the state profile, the first bullet says: "From 2002 to 2006, student achievement increased in both reading and math at all grade levels analyzed, according to percentages proficient and effect sizes." This is true for the state assessment, and presumably this is what the e-mail author is referring to.

Kentucky changed its assessment in 2007, so it's possible (although admittedly quite a stretch) that performance dropped in 2007... which showed up on NAEP but we couldn't see on state testing.

The report provides several reasons why NAEP and state tests might differ.

After receiving your e-mail, I got a call from a person from an organization in Kentucky who was asking the same question about our report. I asked him if he had read the report and he said that he had not, and that he was probably the source of your question. I urged him to read the report before he started raising concerns about it.

By the way, the report—Has Student Achievement Increased Since 2002?—and the fifty state profiles, as well as all the test score data from the states is on our web site, CEP-DC.org.

Thanks.

Jack Jennings, President, Center on Education Policy

Now, Innes's suspicions were not irrational. Knowing that there was a decline in 8th grade reading on the NAEP seems inconsistent with the study's conclusions and knowing that the researchers accounted for the NAEP data is important.

But taken as a whole, BGI tends only to report the bad news about public schools...and I suspect he's was looking to discount the good news about Kentucky schools any time he can. It's probably in his job description.

It would seem any study that produces what BGI believes to be "the wrong answer" must necessarily be a bad study. This is a problem whenever one is hired to substantiate their boss's predetermine perception of reality.

NAEP Gap Continuing for Charters

This from Education Week:

Sector's Scores Lag in Three Out of Four Main Categories

Nearly four years after a front-page story in The New York Times sparked a fierce debate by suggesting that charter school students nationally were lagging academically behind their peers in regular public schools, the national testing program that informed the controversy has generated far more data for researchers and advocates to scrutinize...

...The picture that emerges from the growing data set appears mixed for charter schools. While many analysts urge caution in using NAEP to judge the 4,300-school charter sector, the latest data do not bolster the early hopes of charter advocates that the sector as a whole would significantly outperform regular public schools. The overall scores of charter students tested in 2007 in the nationally representative assessment program were lower than for students in regular public schools in 4th grade reading and mathematics, and in 8th grade math, all by statistically significant margins...

The new NAEP data are released. That means it's time to spin the news!

The release this week of national test scores in reading and math has generated a fresh round of conversation about how Kentucky's students are performing.

We have this conversation at least three times a year when the various "yardsticks" (NAEP; CATS; NCLB) trot out their measurement data. Then there's SAT, ACT... EIEIO....

Wouldn't it be great if such data were integrated into a comprehensive value-added system? But I digress.

NAEP scores nationally, and in many individual states, showed modest gains from 2005 to 2007.

As Diane Ravitch explains in today's New York Post,

The federally sponsored National Assessment of Educational Progress (NAEP) is known in the education world as the gold standard of testing. In 2002, Congress authorized NAEP testing in every state to serve as a check of the states' own claims about their progress. (Congress rightly worried that individual states would dumb-down tests that they themselves develop and administer.)

And, there is at least reason to be suspicious of Kentucky's "new and improved" test. It appears Kentucky may have joined a number of other states in a race to the bottom by the redefining of proficiency.

Whenever test score data are released the spinning begins. The Kentucky Department of Education has an interest (some might say a duty) in pointing out the progress made by the schools. So they publicly shine a light on the best numbers, and privately express concern for the worst.

It's a little thing called spin. Everybody seems to delight in the practice these days.

In a Tuesday press release, the Kentucky Department of Education said,

"The results of the 2007 National Assessment of Educational Progress (NAEP) in reading and mathematics show that Kentucky's 4th and 8th-graders made gains when compared to the state's performance in previous NAEP assessments..."

True. Gains were made. Kentucky's student achievement, as measured by the NAEP, has trended steadily upward overall. (See charts below.)

So that's KDE's headline; Progress over time.

On the other side of the argument, assessment watchdogs are sniffing out specific areas of concern. Writing for the Bluegrass Institute this week, Richard Innes took issue with KDE's discounting of declines in 8th grade reading.

KDE claims of eighth grade reading since 1998, “Kentucky’s 8th-graders’ scores have remained steady, with minor gains and losses.”

Is that a fair description? Let’s examine the facts.

In the new ... NAEP assessments ... Kentucky had a reading proficiency rate of 30 percent in 1998. That rose to 32 percent in 2002 and went up again to 34 percent in 2003.

Then, things came unglued.

Eighth grade reading proficiency decayed to 31 percent in 2005, and in 2007 it slid again to just 28 percent. The 2007 proficiency rate is statistically significantly lower than both the 2002 and 2003 scores and is clearly six points lower than the 2003 performance. That six point difference isn’t just statistically significant – it’s just plain SIGNIFICANT.

No other state lost more ground in this time frame.

What’s more, during the same time period, the CATS Kentucky Core Content Test reading proficiency rates for eighth graders continuously rose. Do you believe that?
CATS up 10 points while NAEP declined six?

I'll take Dick's word for it that the declines are statistically significant and that Kentucky's decline is the nation's worst. The decline certainly looks significant to me.

We don't generally consider six point drops minor, and more to the point, downward trends are antithetical to progress. If it were me...I think I'd have left the word "minor" out of KDE's statement.

A BETTER ATTACK?

Innes has discovered some bad news, but arguably, not the worst news.

While Kentucky has progressed steadily, so have other states. Growth is a vital factor to consider, but so is excellence. Kentucky's relative standing among the states frequently leaves the state in all too familiar territory.

For example, who do Kentucky students outperform in 4th grade math? (See map below)

New Mexico, Louisiania, Mississippi and Alabama. All other states are roughly equal to (9 states), or exceed (36 states) Kentucky's progress. You're not going to hear that in a KDE headline.

It's a little better at 8th grade. Add California, Nevada, Oklahoma, Tenessee, Hawaii and West Virginia to the list.

Kentucky only outscores eight states in 8th grade reading.

But clearly the best news for Kentucky is in 4th grade reading where Kentucky joins the national leaders and is only outscored by seven states. What happens between 4th grade and 8th grade in reading ought to be of concern.

MARK YOUR CALENDARS

We're less than a week away from KDE's next big announcement of progress. I predict the new CATS assessment will show average performance gains of 7% or so across-the-board and in some places, jumps will be huge based at least partly upon changes...
a) to the test itself
b) to the "cut scores" used to define proficiency

The new test data can not be compared to the previous tests - but it will be. It's the data school folks have.

We discussed the NCLB data situation last night at UK. Without advance comment, I asked a group of graduate students (and future principals) to analyze the NCLB proficiency rates in Kentucky. The general reaction to the sharp increase was "Wow!" One of the students shared her experience working with the assessment company to establish the new cut scores. We discussed changes to the system that might account for the dramatic increases, and how school leaders could "present" the data. That's when one of the students came up with the best spin ever. (Pay attention Lisa. Here's your angle.) The new assessment is a truer reflection of the content actually taught in Kentucky's schools, and therefore the 7-point spike in proficiency levels is a fairer measure of the actual progress Kentucky students have made than under the old test.

Terrific.

Now, if we can only get the NAEP data to bear that out....

We have a fundamental problem in our current accountability system. It's initial purpose was political (to garner the support of the business community for KERA's big price tag). It not focused so much on student achievement and curriculum. The focus was school accountability.

Better, would have been a assessment system that began with content and then folded the data into a value-added system, such as the one used in Tennessee. If CATS had been designed to improve instruction for individual students, it would have looked very different.

To their credit, and after the fact, many educators began to look at interim assessment systems that would help teachers identify learning problems early and intervene quickly. There has been a lot of good work done in the trenches, but the state system has become a hodgepoge under NCLB.

Interpreting test data to the public is a national problem, and "interested" parties will always spin the data to suit their own purposes. What we really need is a "disinterested" assessment/accountability reporting source.

As Ravitch understands, we need...

an independent, nonpartisan, professional audit agency to administer tests and report results to the public.

Such an agency should be staffed by testing professionals without a vested interest in whether the scores go up or down. Right now, when scores go down, the public is told that the test was harder this year - but when scores rise, state officials never speculate that the test might have been easier. Instead, they high-five one another and congratulate the state Board...for their wise policies and programs.

What the public needs are the facts. No spin, no creative explanations, no cherry-picking of data for nuggets of good news.

Just the facts.

I may even know the right folks for the job. I understand Ben Oldham at Georgetown College has recruited Skip Kifer and Tom Guskey to form an assessment center at Gtown. I studied under Ben and Skip and am familiar (along with most of the national academic community) with Tom's work. I can't think of a more valuable state resource to guide a more effective and fair assessment program that the guys at Georgetown.

KENTUCKY SCHOOL DATA from the 2007 NAEP Exam

Student Characteristics
Number enrolled: 679,878
Percent in Title I schools: 60.6%
With Individualized Education Programs (IEP): 16.0%
Percent in limited-English proficiency programs: 1.5%
Percent eligible for free/reduced lunch: 52.4%

School/District Characteristics
Number of school districts: 176
Number of schools: 1,426
Number of charter schools: N/A
Per-pupil expenditures: $7,2541
Pupil/teacher ratio: 16.0
Number of FTE teachers: 42,413

Racial/Ethnic Background
White: 86.3%1
Black: 10.6%1
Hispanic: 2.1%1
Asian/Pacific Islander: 0.9%
American Indian/Alaskan Native: 0.2%

Scale Scores for Mathematics

Kentucky vs. National Public

For more specific data check out NAEP's Data Explorer.

Cross State Comparison

Percent At or Above Proficient

4th Grade Mathematics

Where:
Blue = Kentucky
Green = States above Ky
Yellow = States about the same as Ky
Red = States below Ky

For more specific data check out NAEP's Data Explorer.

"Yo Mama's so wimpy, she's satisfied with Mississippi's cut scores."

This from University of South Florida Education Professor, Sherman Dorn ,who looks at National Standards as Policy Machismo. He agrees. He disagrees.

Alexander Russo and I agree on National (Yawn) Standards (Again) (his title), regarding last week's CEP report on state proficiency percentage trends and the NCES comparison of state proficiency cut-scores and NAEP cut-scores and also the double-report week's politics. In a different way, I also agree with U.S. Secretary of Education Margaret Spellings in her dissing of national standards. Same (in a yet third way) with the Education Sector's Danny Rosenthal.

Alexander Russo is right on the politics of national standards: dead for now.

Spellings is right that the federal government does a better job of collecting data than telling the states what to do.
She's wrong that the federal government does a better job of telling the states what to do when it's labeled NCLB.

Rosenthal is correct that there is a difference between setting curriculum standards and setting cut scores.
He's wrong in asserting that the cut scores are what is important.

The cut-score debate would be a silly one except for the stakes involved in states and the way that cut scores frame the education policy debate inside the Washington, D.C., beltway.

As anyone who has taken elementary statistics should know, the division of an interval scale into several tiers creates an ordinal scale. Whether one labels the tiers Expert, Proficient, Basic, and Below Basic; Red, Orange, Yellow, Green, and Blue; or Venti, Grande, and Tall, tying values to ordinal tiers doesn't tell us anything about the tiers themselves other than that someone wanted to label them.

Confusing cut scores with rigor is an act of policy machismo, not common sense.

Most Nations Fall Short of NAEP Proficiency, Analysis Finds

Education Week reports:
Outside of a handful of Asian nations, the typical 8th grader in many foreign countries would not meet “proficient” levels on U.S. tests of mathematics and science, according to a reanalysis of international achievement data being published today. Then again, the study also shows, neither do most American students.

The American Institutes for Research, a Washington-based research organization, posted a press release and the new analysis comes from AIR’s chief scientist, Gary W. Phillips.

Mr. Phillips’ idea was to statistically “link” scores from two well-known testing programs: the Trends in International Mathematics and Science Study, or TIMSS, which is given every few years to students in more than three dozen countries, and the National Assessment of Educational Progress, a congressionally mandated program known as the “nation’s report card.”

Skills Gap on State, Federal Tests Grows, Study Finds

Far greater shares of students are proficient on state reading and mathematics tests than on the National Assessment of Educational Progress, and those gaps have grown to unprecedented levels since the No Child Left Behind Act became law in 2002, concludes a new study.

The study by Policy Analysis for California Education, a nonprofit research group based at the University of California, Berkeley, was released during the annual meeting of the American Educational Research Association. The researchers compiled state and federal testing results for the period 1992 to 2006 from 12 states: including Kentucky.

In all but two states the disparity between the share of students proficient on state reading tests and on NAEP, a congressionally mandated program that tests a representative sample of students in every state, grew or remained the same from 2002 to 2006. A similar widening occurred between state and federal gauges of math performance in eight of 12 states.

Those findings call into question whether the state-reported gains are real or illusory, according to the researchers.

This from Lynn Olsen at Education Week.

Home life affects students in school

Published on Mon, Nov. 27, 2006

By Richard Day

(At Issue: Nov. 14, 2006, Race disparities persist in U.S. by Stephen Ohlemacher of the Associated Press, in the Lexington Herald-Leader)


The U.S. Census Bureau recently reported that decades after the civil rights movement, racial disparities in income, education and home ownership persist and may be growing again. Median income for black households had remained about 60 percent of the income for white households since 1980. In dollar terms, the gap has grown from $18,123 to $19,683. Last year, data showed average incomes in white households to be two-thirds higher than those for blacks. Despite claims that "trickle down" economic policies create more jobs and help poor families, the evidence provides reason for doubt.

Closing educational achievement gaps is probably the nation's best avenue to racial equality, but gaps persist after more than a decade of intense effort and despite the highest levels of school accountability in our nation's history.

This trend can't be too surprising to anyone with a sense of American history. The collective effects of slavery, Jim Crow and persistent social attitudes designed to keep down African-Americans have played powerful roles. In that sense, the achievement gap was deliberately created -- over many decades -- by majority rule.

It is instructive to remember that achievement gaps were closing in the wake of President Lyndon B. Johnson's Great Society social programs. As Harvard social-policy professor Christopher Jencks reported in his exhaustive book, The Black-White/Test Score Gap, the greatest gains made by black students came between 1968 and 1980 and were explained by the civil rights movement, the War on Poverty and school-based improvements.

What happens to a student outside school -- such as access to health care and a safe home environment -- matters when it comes to achievement in school. Direct and indirect benefits to black student achievement contributed to 0.6 standard deviation growth for blacks on the National Assessment of Educational Progress, with a corresponding 0.1 standard deviation growth for whites.

This demonstrates the very definition of gap closing: finding ways to accelerate the learning of disadvantaged students while continuing growth among all students.

Unfortunately, President Ronald Reagan's economic policies reversed that, and the gap began to grow again. Reagan's decisions were informed by the prevalent social science view at that time: that school resources had little impact on student achievement.

Jencks, for example, argued in his influential 1972 book, Inequality, that reducing cognitive inequality would not do much to reduce economic inequality. He has since reversed his position, now stating that changes in education and earnings would reduce racial differences in crime, health and family structure.

New data prompted him to conclude that reducing the black-white test score gap is probably necessary and sufficient for reducing racial inequality in educational attainment and earnings. President Bush's economic policies echo those of Reagan. The best feature of Bush's education legislation, No Child Left Behind, may well be its title. It expresses the goal that schools will educate every student and will be evaluated by the performance of all subgroups of students, not just schoolwide averages. But NCLB is underfunded by billions of dollars, which, when combined with conservative economic policies, casts serious doubts about its success.

As Brandeis University law and social-policy professor Thomas Shapiro understood, "the wealth gap is not just a story of merit and achievement; it's also a story of the historical legacy of race in the United States." Changing that legacy will require more than strong leadership that motivates teachers to work harder and more effectively. It will require a fully funded and efficient system that provides the resources students need to attain high standards; greater access to pre-school and kindergarten programs, more help for English-language learners and special needs students, and additional school days.

There are a growing number of schools in low-income neighborhoods that are defying the odds and raising the performance of their students to top levels. It is being achieved with a combination of culture building, high expectations and hard work that focuses on each student's individual needs. But if one looks broadly at the performance of our schools, it is clear that much remains to be done. The census data demonstrate the cost to our nation if we fail to close achievement gaps across the board.

© 2006 Lexington Herald-Leader and wire service sources.
All Rights Reserved.http://www.kentucky.com