Concerns about PGES results partly behind delay in factoring data into 2015-16 scores

This from Brad Hughes at KSBA:

Kentucky will delay incorporating job performance ratings of teachers and principals into the Unbridled Learning accountability model for at least a year following a decision Thursday by the state board of education.

Specifically, a scheduled inclusion of up to 10 points in school and district scores, based on how principals evaluated teacher growth and superintendents calculated the same for principals, won’t be factored into the annual progress scores for the 2015-16 school year. The 10 points will remain with the existing assessment areas on student academics (achievement, gap, growth, college/career readiness and graduation rate) and the program reviews (covering writing, arts and humanities, practical living, K-3 studies in elementary schools and world languages in high schools for 2016).

In presentations to the KBE, Education Commissioner Terry Holliday and Associate Commissioner for Assessment and Accountability Rhonda Sims said the one-year wait would be the best of several options, and addresses concerns raised about the PGES (Professional Growth and Effectiveness System) ratings for educators.

“Taking the one year will give more time to build capacity,” Holliday told the state board, which voted unanimously for the delay but also indicated it would review the decision next year. 

In an interview with eNews, Sims agreed that in terms of inclusion in the overall ratings for schools and districts, PGES “is not fully ready yet.

“When you put something in accountability, you want it ready for that kind of reporting because it carries consequences,” she said. While there would have been no immediate use of PGES evaluations in personnel decisions in 2015-16, eventually that option is part of the Senate Bill 1 law that required development of the Unbridled Learning system.

According to a Kentucky Department of Education staff report to the KBE (highlights included below), another point of concern dealt with the possibility of PGES ratings being inflated at the local level. At Thursday’s KBE meeting, it was discussed that more than 90 percent of educators who have had their 2014-15 evaluations reported to the state are rated as being “exemplary” or “accomplished,” the two highest possible ratings.

Another unknown that could be resolved before 2016 is the outcome of Congressional negotiations on reauthorization of the Elementary and Secondary Education Act (formerly referred to as No Child Left Behind). A key point remaining to be settled between the House and Senate is future federal requirements for use of educator evaluations, either as part of state accountability systems or as data outside the ratings but reported separately to the public.

KDE Staff report for KBE Aug. 6 meeting

Next Generation Professionals Accountability

Senate Bill 1 (2009) required Kentucky to begin a new assessment and accountability system in the 2011-12 school year. The assessment and accountability model is a balanced approach that incorporates all aspects of school and district work and is organized around the Kentucky Board of Education’s four strategic priorities: nextgeneration learners, next-generation professionals, next-generation support systems and next-generation schools/districts. Kentucky's Unbridled Learning Accountability Model is made up of three components.

Next-Generation Learners (70%) was the first component to be enacted in the 2011-12 school year. This component measures performance in the areas of achievement, gap, growth, college/career readiness and graduation rate.

Next-Generation Instructional Programs and Support (20%), the second component, officially began in the 2013-14 school year following a pilot of the program reviews. The program reviews are expanding from the original Arts and Humanities, Practical Living/Career Studies and Writing to include K-3 and World Languages. The K-3 program review was included in the 2013-14 calculation following a pilot process. World Languages will have a similar track and begin as a pilot for high schools in 2014-15 with elementary and middle school levels included at a future date.

Next-Generation Professionals (NGP) (10%) is the final component slated to be added to the model in the 2015-16 school year.

Educator effectiveness has always been a state priority and is reflected in state and national (Race to the Top (RTT) and Elementary and Secondary Education Act (ESEA) Flexibility Waiver) expectations. As the discussion began concerning Next Generation Professionals Accountability, the School Curriculum, Assessment and Accountability Council (SCAAC) recommended that performance category percentages be used as the measure for the ten points in the accountability model for Next Generation Professionals.

However, concerns have been expressed that Professional Growth and Effectiveness System (PGES) ratings may be inflated. Therefore, at the July SCAAC meeting, KDE discussed two options that could be considered for calculating the ten points.

Option 1 is an all or nothing model. In this model, the Kentucky Department of Education (KDE) would set a delivery target for each school based on reducing novice performance to two percent or less by 2020. If a school meets the target, then the school gets all ten points in the teacher/principal accountability component. If the school does not meet the target, then it gets zero points. If the school eliminates novice and maintains zero percent novice, then it gets the ten points every year.

Option 2 is a 50/50 model. In this model, KDE would use the novice reduction model in option one but assign only five points. The other five points would go toward the percentage of teachers meeting student growth goals. The district accountability model would be the average of the points achieved by the schools.

Groups Consulted and Brief Summary of Responses:

School Curriculum, Assessment, and Accountability Council (SCAAC)

Teacher Effectiveness Steering Committee (TESC)

Kentucky Education Association (KEA)

Feedback from the listed groups did not support either option that was proposed above.
SCAAC did not support Option 1 nor Option 2. The committee shared that educator effectiveness is measured through the student outcomes of students throughout all components of the accountability model. Although the advisory council did not have a quorum to make a formal recommendation, they did discuss it and found consensus on two additional recommendations as follows:

Solution 1: Distribute the educator effectiveness points into the Achievement, Gap, and Growth components. pasting. (At the KBE meeting, this option was corrected to distribute points across the whole accountability model with Next-Generation Learners remaining at 77 percent and Program Reviews at 23 percent.)

Solution 2: If the educator effectiveness component could not be redistributed, they would like to recommend the original recommendation of 70% Overall Teacher Effectiveness and 30% Overall Principal Effectiveness.  

The TESC was polled after the June KBE meeting to receive input on support for the two options presented to the board. Thirteen members responded with the majority suggesting other options including distribution of points across each component, removing the measure, and using the Professional Practice rating.

The KEA expressed concern over the two options proposed to the KBE and suggested the ten points be distributed across Achievement, Gap, and Growth.

Staff Recommendation and Rationale:

Staff recommends that the KBE approve the suggestion of distributing the educator effectiveness points across the entire accountability model. It was most recently suggested by SCAAC but was one of the ideas voiced by the other constituency groups.

The recommendation is reflective of feedback from constituents and aligns with the ESEA waiver requirements.