KDE wants a third year of data before changing student “growth” factors in school, district assessments

This from KSBA with a little graphics help from KSN&C:

Officials of the Kentucky Department of Education say they’ve heard “loud and clear” the concerns local leaders have about the impact of the student growth percentile on whether schools and/or districts met their overall Unbridled Learning accountability goals this year.

But, during Wednesday’s monthly KDE webinar for superintendents, Associate Commissioner for Assessment and Accountability Ken Draut said the agency wants another year of testing data before proposing any changes in how student and school progress is measured in Kentucky.

Other topics drawing attention by Education Commissioner Terry Holliday and KDE staff during the webinar including questions about the just-released first program review measurements and the ongoing development of the new process for school board evaluations of superintendents.

Growth anxieties

Draut said the No. 1 topic of discussion following September’s release of the 2013 Unbridled Learning test results was the student growth measurement, which he described as tracking “How did our kids perform compared to their peers statewide?”

“In the growth model, you are looking at individual kids who were placed in a state band (grouping based on previous year’s scores) all across the state,” Draut said. “We’ve had calls asking, ‘How can I be moving up in achievement (another measurement in the state index) but moving down in growth?’“

According to Draut, while achievement scores are based on how all students in a school or district performed compared to that school’s previous year’s scores, growth measures whether an individual student kept pace with those in her or his statewide peer band. “You might have only one student in a school in that specific state band,” he said.

Example above from Massachusetts Dept of Ed.

• Points are awarded for percentages of students showing typical or high academic growth in reading and mathematics in grades 3 through 8.
• For high schools, growth will be indicated by comparing PLAN and ACT composite math and reading scores.
• The growth calculation compares an individual student’s score to the student’s academic peers, using a Student Growth Percentile.

“The model works well. It does what it’s supposed to do. It’s a different way of measuring achievement. Every student has a growth score,” Draut said. “You can have an increase in achievement and then have growth that went up or decreased.”

Draut acknowledged three common themes on questions about the growth measurement from this past spring’s test:

• Growth bounces around – up one year, down another.

• Specific targets on growth can’t be set because it depends on what others around the state do.

• The weighting of growth in the accountability index is too high, ranging from 40 percent of elementary schools’ scores to 28 percent at the middle school level.

While those concerns had led KDE staff to “dig deeper” into the issue, Draut said now is too soon to propose changes in the accountability calculations.

“We want three years of data, so you don’t have one year that was an anomaly,” he said, adding that after the 2014 test results are in, “We’ll take the issue to the (state board) next fall for adjustments.”

Program review audits

Another testing-related topic that arose during Wednesday’s webinar was that of individual school performances in the just-released program reviews involving arts and humanities, writing and practical living.

Apparently, some schools’ program review scores raised eyebrows in Frankfort.

“We do have concerns about the quality control on program reviews. Several schools, not many, but a few schools, had high program review scores and very low writing scores,” Holliday said. “Those two things on face value are inconsistent.

“Districts have the responsibility to sign off on schools’ program reviews, so we’ll be helping you with district auditing procedures and training,” the commissioner said.

“This is the first year we’ve put program reviews in for accountability purposes, so we’re still in a growing mode. But remember this is a Senate Bill 1 requirement and it is part of accountability,” he said. “We do have to address validity and reliability issues.”

Holliday said he would discuss the issue in more detail in his weekly Internet blog , posted on Friday.

Superintendent review specifics

During an update on the department’s work to create a new model superintendent evaluation process, Holliday discussed a series of 19 questions that relate to the three new elements of evaluations: test scores, district finances, and facilities and resource needs as measured in the biennial TELL Kentucky survey. While at first saying superintendents “need to go over” the 19 questions with the board, a pair of email inquiries from webinar viewers drew a quick clarification from the commissioner.

“We’ve left this up to your discretion,” Holliday said. “If you feel like you’ve addressed all of the components of delivery targets, TELL Kentucky survey and the budget with your board and your minutes so reflect, you’ve met the requirements.

“If you’ve had a meeting with your board to discuss TELL results, then you’re in good shape. If you’ve had a meeting with your board to go over your school report card, you’re in good shape. If you’ve passed an operating budget that has the 2 percent contingency and the other legal requirements, you’re in great shape. Don’t panic. Just make sure you’ve got the board minutes to reflect it,” he said.

“This is the first year. We’re going to be very open to minutes that might not reflect these questions 100 percent. Don’t panic. Go over school report card, TELL results and your normal budget requirements, then you just answer those questions in ASSIST (KDE’s online information sharing system), upload it by Dec. 20, put a copy of the minutes as an attachment. And you’ll be fine. We’ll work with you,” he said.

Under a state directive, all school boards and superintendents must have a public discussion of the budget, academic and resource issues and submit board meeting minutes confirmations to KDE via the ASSIST system by Dec. 20. Then next spring and summer when each board conducts its annual evaluation of its superintendent, those three elements must again be incorporated into the review in a public session and so reported to KDE.