tag:blogger.com,1999:blog-5776587.post79776445529820242..comments2023-11-03T04:00:24.785-04:00Comments on Kentucky School News and Commentary: Report details Kentucky's progress on education, future challenges Richard Dayhttp://www.blogger.com/profile/14586435007687942849noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-5776587.post-23717624550908680562016-06-17T20:40:23.495-04:002016-06-17T20:40:23.495-04:00The Prichard/Chamber report falls for a considerab...The Prichard/Chamber report falls for a considerable amount of poorly done research.<br /><br />For example, the Center for Business and Economic Analysis paper cited in the Prichard/Chamber report ranks all states for their ACT Composite Scores. With the ACT still optional at student discretion in 37 states as of 2015, that is pure nonsense. Do you really think it is acceptable to rank Kentucky, where every graduate takes the ACT, to Maine, where only one in ten grads does? Even ACT, Inc. says they don’t recommend this. I asked.<br /><br />By the way, 13 states did test all of their 2015 high school grads with the ACT in 2015. Properly done comparisons between those states are more valid. When I ranked white students’ ACT Composite Scores for those 13 states, Kentucky came in dead last. How does that square with what CBER and Prichard/Chamber are trying to sell us?<br /><br />Another bogus ranking from CBER compares the high school graduation rates for the 50 states. Sadly, there are absolutely no uniform standards across the states for what it takes to get a diploma, so this is just more unsuitable analysis. <br /><br />However, do keep in mind that while Kentucky’s published high school graduation requirements specifically include the course content of Algebra II, over the past four years no more than 40 percent of the students who took the Algebra II End-of-Course Exam in each of those years scored proficient or more. How do you reconcile that with the state’s claimed 2015 high school graduation rate of 88 percent? <br /><br />Maybe, after you sift out the social promotion diplomas, Kentucky’s real high school graduation rate should only be about 40 percent. Of course, who knows what kinds of similar manipulations are going on in other states? <br /><br />CBER delights in doing lots of comparisons of overall “all student” average scores for the NAEP when even the NAEP’s own reports say you need to consider differing student demographics across the states when you use this federal test to compare states (The NAEP 2009 Science Report Card has a special section on this – guess which state is the prime eighth grade example of why you need to consider disaggregated data!). What CBER’s rankings essentially do is to match up a lot of white, English speaking kids in Kentucky against non-white, sometimes non-English-as-a-first-language students in other states. This is clearly bogus; but, it sure does make Kentucky seem to be good.<br /><br />How about the Chamber/Prichard report citing Education Week’s Quality Counts rankings? What a hoot. It seems like everyone involved with the Prichard/Chamber report developed convenient amnesia about Quality Counts 2013, which somehow ranked Kentucky’s education 10th best in the nation. Now we rank much lower than that in Quality Counts 2016. Did we somehow experience a disastrous decline? Or, is this just more evidence that these ranking schemes are highly unstable? <br /><br />Now, here is a major takeaway: It is really hard to develop valid state-to-state education rankings. The CBER certainly doesn’t understand the data well enough to do this right, and neither does the special unit at Education Week that does Quality Counts (regular reporters at EdWeek do not develop this “stuff”). <br /><br />If you want to push this stuff without comment as valid, you only undermine your own credibility. I think you want to be better than that.<br /><br />Now, here’s a thought – How about a symposium at Eastern about how to do this sort of ranking better. I’ll come. It could be a nationwide attention grabber!Richard Inneshttps://www.blogger.com/profile/14892409111799693983noreply@blogger.com