EXPLORE and PLAN Results Show Improvement

Kentucky’s 8th- and 10th-grade public school students participated in a statewide administration of the EXPLORE and PLAN assessments in the fall of 2009, and overall school scores moved up slightly in most subject areas tested.

Unlike most other states, all of Kentucky’s 8th- and 10th-grade public school students participate in EXPLORE and PLAN assessments. As is often the case with these examinations, when the pool of test-takers is large, overall scores tend to be lower. For the 2009 administration, Kentucky’s overall scores increased in nearly every subject area.

In the 2009 administration, 48,347 8th-grade Kentucky students in 323 public schools took the EXPLORE assessment. The scoring scale for the assessment is from 1 to 25.

In the 2009 administration, 49,589 10th-grade Kentucky students in 228 public schools took the PLAN assessment. The scoring scale for the assessment is from 1 to 32.

The national normative data for EXPLORE and PLAN are based on students who took all four academic tests within standard time limits as part of a national study conducted in Fall 2005.
ACT, Inc. developed College Readiness Benchmarks in English, mathematics, science and reading and applied those to the EXPLORE and PLAN scores. The benchmarks indicate the degree of college readiness of 8th and 10th graders.

The benchmark scores for EXPLORE are:
§ 13 or higher on the English Test
§ 17 or higher on the Mathematics Test
§ 15 or higher on the Reading Test
§ 20 or higher on the Science Test

The benchmark scores for PLAN are:
§ 15 or higher on the English Test
§ 19 or higher on the Mathematics Test
§ 17 or higher on the Reading Test
§ 21 or higher on the Science Test

Scores by ethnicity and gender followed trends similar to those found in other assessments.

Administration of the EXPLORE, PLAN and ACT assessments, which are provided by ACT, Inc., was mandated by Senate Bill 130 (codified in KRS 158.6453) in the 2006 session of the Kentucky General Assembly. The assessments will help schools focus on meeting academic standards across the entire secondary school program. Scores from the assessments will be helpful in measuring student achievement, gauging their readiness for transition and evaluating school programs.

EXPLORE is a high school readiness examination designed to help 8th graders explore a broad range of options for their future. The exam assesses four subjects (English, mathematics, reading and science) and provides needs assessments and other components to help students plan for high school and beyond.

PLAN helps 10th graders build a solid foundation for future academic and career success and provides information needed to address school districts' high-priority issues. The exam assesses four subjects (English, mathematics, reading and science) and is a predictor of success on the ACT.

Both assessments help schools pinpoint areas of weakness for individual students and schoolwide curriculum and make changes to improve learning. Schools will analyze their individual results to inform decision-making.

State law (KRS 158.6459) mandates that the Individual Learning Plans of students whose scores on EXPLORE and PLAN indicate that they need additional assistance in particular subject areas will incorporate strategies to help them improve their performance.

See complete details on district and school EXPLORE and PLAN data by clicking here.

SOURCE: KDE press release.

Attention Bluegrass Institute: This is How it's Done

The Kentucky Department of Eucation today issued a retraction for some misleading information contained in charts showing overall average scores for PLAN and EXPLORE tests given to Kentucky students.

The errors were called to their attention by Richard Innes, an analyst with the Bluegrass Institute, here, here and here.

KSN&C agrees with Innes's analysis.

This from KDE:

In News Release 09-017 (EXPLORE and PLAN Results Released), please note that the tables showing overall average scores for those assessments may be misleading.

The tables show Kentucky's average English, mathematics, reading, science and composite scores for 2006, 2007 and 2008, along with average scores nationwide for those years.

The tables should have indicated that the national scores are based on the averages of students who took those tests in 2005, when EXPLORE and PLAN were normed as part of a national study. The 2005 nationally normed score also...used the national average for 2006, 2007 and 2008 and will continue to be the "score of record" for national purposes until the tests are re-normed.

The corrected tables were attached to the press release and show the 2005 normed score only.

Now, if we can just the Bluegrass Institute to retract their exaggeration of of a recent UK study, we'll have two groups who have demonstrated some integrity this week.

Explore and Plan Results Released

(FRANKFORT, Ky.) – Kentucky’s 8th- and 10th-grade public school students participated in a statewide administration of the EXPLORE and PLAN assessments in the fall of 2008, and overall school scores moved up slightly in most subject areas tested.

Unlike most other states, all of Kentucky’s 8th- and 10th-grade public school students participate in EXPLORE and PLAN assessments. As is often the case with these examinations, when the pool of test-takers is large, overall scores tend to be lower. For the 2008 administration, scores nationwide remained flat, while Kentucky’s overall scores increased in nearly every subject area.

In the 2008 administration, 48,653 8th-grade Kentucky students in 324 public schools took the EXPLORE assessment. The scoring scale for the assessment is from 1 to 25.

In the 2008 administration, 50,531 10th-grade Kentucky students in 229 public schools took the PLAN assessment. The scoring scale for the assessment is from 1 to 32.

The national normative data for EXPLORE and PLAN are based on students who took all four academic tests within standard time limits as part of a national study conducted in Fall 2005.
Scores by ethnicity and gender followed trends similar to those found in other assessments.

Administration of the EXPLORE, PLAN and ACT assessments, which are provided by ACT, Inc., was mandated by Senate Bill 130 in the 2006 session of the Kentucky General Assembly. The assessments will help schools focus on meeting academic standards across the entire secondary school program. Scores from the assessments will be helpful in measuring student achievement, gauging their readiness for transition and evaluating school programs.

The ACT Index is included in accountability calculations for middle and high schools. PLAN and ACT were included in the high school ACT Index beginning with the 2007-08 school year. EXPLORE scores will be included in the 2008-09 middle school ACT Index.

The EXPLORE program is a high school readiness examination designed to help 8th graders explore a broad range of options for their future. The exam assesses four subjects (English, mathematics, reading and science) and provides needs assessments and other components to help students plan for high school and beyond.

The PLAN program helps 10th graders build a solid foundation for future academic and career success and provides information needed to address school districts' high-priority issues. The exam assesses four subjects (English, mathematics, reading and science) and is a predictor of success on the ACT.

Both assessments help schools pinpoint areas of weakness for individual students and schoolwide curriculum and make changes to improve learning. Schools will analyze their individual results to inform decision-making.

State law (KRS 158.6459) mandates that the Individual Learning Plans of students whose scores on EXPLORE and PLAN indicate that they need additional assistance in particular subject areas will incorporate strategies to help them improve their performance.

See complete details on district and school EXPLORE and PLAN data by visiting the Kentucky Department of Education's Web site at http://www.education.ky.gov and clicking on the Testing and Reporting link or HERE.

SOURCE: KDE Press release

Dale Brown on CATS Task Force

This from the Daily News by way of KSBA:

Brown will help review CATS test

A task force of educators from around the state, including Warren County Schools Superintendent Dale Brown, will get the opportunity to shape Kentucky’s education accountability model.

Brown will join other educators and members of the General Assembly to review the current design of the Commonwealth Accountability Testing System. Brown will be representing the Partnership for Successful Schools - a Lexington-based nonpartisan education advocacy organization - on the task force. “With any assessment, we should always be open to review,” Brown said.

The task force will seek input from teachers, administrators, parents, business people, elected officials, education advocacy groups and others. The group will be looking at each component of the accountability model - to determine the effectiveness in meeting the needs of students - and make recommendations toward the future assessment design, Brown said.The group will have its first meeting in July.

The task force is a result of years of legislators looking at ways to overhaul how tudent growth is measured, said Lisa Gross, director of communications for the Kentucky Department of Education. The testing system is key part of the Kentucky
Education Reform Act.“There are a lot of people that just don’t like CATS,” she said. “There has been criticism of the system, and there are legislators that are just not fond of it.”

After Senate Bill 1, which would have replaced CATS with other standardized tests, failed to pass the legislature this year, the Kentucky Department of Education said a statewide task force would be put together by Education Commissioner Jon E. Draud to review the student-testing system after the General Assembly ended.

CATS includes the Kentucky Core Content Test, in which students are tested in seven subject areas; nonacademic data; writing portfolios; alternate assessments for students with severe disabilities; and the ACT college entrance exam and its precursors, PLAN and EXPLORE.

While many legislators question the effectiveness of the state’s accountability model, Brown said it has worked for Warren County Schools.“We are making progress toward proficiency and some schools have already met proficiency,” he said. “But we must always be open to study and make recommendations to improve the current system.”

Kifer says EXPLORE and ACT Overstate Claims

Skip Kifer of the Center for Advanced Study of Assessment at Georgetown College released a report this month that calls into question claims made by the ACT program regarding the efficacy of its EPAS system.

Kifer has a history with the ACT... and its a good one. In fact, he has to have been considered a fan. In Mental Measures Yearbook (Kifer, 1985) he lavishly praised test-maker ETS and the technical construction of the ACT. In fact, he found their practices to be worthy of admiration and emulation.

But times change. Since states have begun requiring exams that are specific to state curriculum, ETS has entered the state market and made claims about the efficacy of the EPAS system. It is these claims Kifer examines.

The study was motivated by the introduction of Senate Bill 1 during the 2008 legislative session. The bill included additional reliance on off-the-shelf standardized tests. Since the state assessment already contains such measurements under the EPAS system, a relatively new ACT program, Kifer set out to determine whether the program lived up to its claims.

Kifer looked at the characteristics of EXPLORE and ACT, two components of EPAS taken by 8th, 10th and 11th or 12th grade students, to determine whether or not the claims made for it and them are valid. He also compared ACT predictions to those obtained from other components of the Commonwealth’s assessment.

Kifer concludes that claims for the efficacy of the EPAS are stronger than the findings would suggest.

For example, EXPLORE lacks a representative national comparison group and therefore lacks true national norms. EXPLORE was constructed on the basis of national information, and therefore does not represent the Kentucky curriculum. Kifer also says,

"Although the technical manual suggests that the tests measure 'higher order thinking skills,' there is no dimension for those attributes in the test construction descriptions and no indications of how many questions of that type are on a test. EXPLORE is alleged to be diagnostic but is diagnostic in the limited sense that scores on EXPLORE predict scores on the ACT. Finally, there are validity studies of Kentucky’s assessments and ACT scores that suggest that the Commonwealth’s assessments are comparably effective predictors of performance in higher education."

Kifer concludes that EXPLORE and ACT do not live up to their claims. It also shows that other measures in the Commonwealth’s assessment do about as well and sometimes better than the ACT tests in predicting grade point average in college.

What this means, says Kifer is that,

If a student were certain that he or she was going to attend a public university in Kentucky, it would not be necessary to take the ACT. That student’s CATS score would be sufficiently predictive to take the place of the ACT. A consequence of using CATS scores in a college admissions process might motivate high school students to do their best on the statewide assessments. Finally, not requiring the ACT could save the Commonwealth a substantial amount of money.

EXPLORE and PLAN Results Released

(FRANKFORT, Ky.) – Kentucky’s 8th- and 10th-grade public school students participated in a statewide administration of the EXPLORE and PLAN assessments last fall, and overall school scores are mid-range or higher.

Unlike most other states, Kentucky tests all of its 8th- and 10th-grade public school students. As is often the case with ACT examinations, when the pool of test-takers is large, overall scores tend to be lower.

In fall 2007, 48,450 8th-grade Kentucky students in 320 public schools took the EXPLORE assessment. The scoring scale for the assessment is from 1 to 25.

In fall 2007, 50,097 10th-grade Kentucky students in 230 public schools took the PLAN assessment. The scoring scale for the assessment is from 1 to 32.

*National normative data are based on students who took all four academic tests within standard time limits as part of a national study conducted in Fall 2005.

Administration of the EXPLORE and PLAN assessments, which are provided by ACT, Inc., was mandated by Senate Bill 130 in the 2006 session of the Kentucky General Assembly. The two assessments and the ACT, which will be administered statewide for the first time in March, will help schools focus on meeting academic standards across the entire secondary school program. Scores from the assessments will be helpful in measuring student achievement, gauging their readiness for transition and evaluating school programs.

As part of the ACT Index, PLAN scores factored into accountability results for schools and districts for the 2006-07 school year; the inclusion of EXPLORE scores in accountability will be reviewed by the Kentucky Board of Education at its meeting February 6 and 7. Beginning with the 2007-08 school year, scores from both PLAN and the ACT will be included in the ACT Index.

The EXPLORE program is a high school readiness examination designed to help 8th graders explore a broad range of options for their future. The exam assesses four subjects (English, mathematics, reading and science) and provides needs assessments and other components to help students plan for high school and beyond.

The PLAN program helps 10th graders build a solid foundation for future academic and career success and provides information needed to address school districts' high-priority issues. The exam assesses four subjects (English, mathematics, reading and science) and is a predictor of success on the ACT.

Both assessments help schools pinpoint areas of weakness for individual students and schoolwide curriculum and make changes to improve learning. Schools will analyze their individual results to inform decision-making.

State law (KRS 158.6459) mandates that the Individual Learning Plans of students whose scores on EXPLORE and PLAN indicate that they need additional assistance in particular subject areas will incorporate strategies to help them improve their performance.

See complete details on district and school EXPLORE and PLAN data by visiting the Kentucky Department of Education's Web site at http://www.education.ky.gov/ and clicking on the Testing and Reporting link.

SOURCE: KDE press release

PLAN/EXPLORE Results to be Released

(FRANKFORT, Ky.) – Results from the 2007 administration of the PLAN and EXPLORE assessments will be released on Wednesday, January 30.

A news release will provide state-level data. School- and district-level data will be available on the Kentucky Department of Education’s Web site at the Testing and Reporting link at 2 p.m. EST.

The EXPLORE program is a high school readiness examination designed to help 8th graders explore a broad range of options for their future. The exam assesses four subjects (English, mathematics, reading and science) and provides needs assessments and other components to help students plan for high school and beyond.

The PLAN program helps 10th graders build a solid foundation for future academic and career success and provides information needed to address school districts' high-priority issues. The exam assesses four subjects (English, mathematics, reading and science) and is a predictor of success on the ACT.

SOURCE: KDE press release

Student readiness "near average?!" What does that mean?

PREPAREDNESS FOR HIGH SCHOOL, ACT TESTED

Senate Bill 130, requires all Kentucky high school juniors to take the ACT exam starting in 2008. One of the tests required by the bill, called EXPLORE, measures whether eighth-graders are prepared for high school. The other, called PLAN, shows whether 10th-graders are ready for the ACT, a college entrance exam.

Yesterday, the Kentucky Department of Education released data from these new readiness tests that were given to middle and high school students for the first time in October and eventually will factor into overall state achievement scores under the Commonwealth Accountability Testing System.

How'd we do?

Statewide - below average, again.

Kentucky eighth-graders were nearly as prepared for high school as their peers nationwide, while the state's 10th-graders were not as ready for the ACT college entrance exams as similar students nationwide.

Kentucky received a 14.5 composite score on the EXPLORE tests, compared with a national score of 14.9. The national PLAN composite score was a 17.5, and Kentucky students scored 16.4.

Fayette County's EXPLORE composite was 15.1; the district's PLAN composite was 17.8.

"This is the first time that we have this data," (my colleague and friend) Principal Jock Gum told the Herald-Leader. "It helps us to understand where deficiencies are as far as preparation for high school." Gum said that, unlike CATS, there was little pressure put on students because this is thought of as a diagnostic test.

The press release from KDE said, "...overall school scores are mid-range or higher." That comment prompted a retort from the Bluegrass Institute saying, "...the Kentucky Department of Education has lost no time in trying to spin below average results into “scores that are mid-range or higher.”

This is a definitional argument and a pretty good example of how data can be made to sing a certain song depending on one's orientation.

So in this case, who's correct? Well, they both are - to a point.

Every statistician on the planet understands that the mid-range of human performance is generally somewhere between the 40th and 60th percentiles. Some will describe mid-range as including all scores that are between one standard deviation below, and one standard deviation above the mean. This is a fair description of the EXPLORE and PLAN results, and KDE is not inaccurate in their description. But did they describe the data with a positive spin? You bet.

On the other hand, the scores are, in fact, below average (with the sole exception of 8th grade reading, which is exactly average) and the high school scores trail those of the middle school. Is it fair to say Kentucky's scores are generally below average? Yes, they are.

But let's not reinterpret Webster to say that "mid-range" is the same as "average." Mid-range is a set of scores that surround the mean. An average (the mean) is a single score.

Here are the data, courtesy of the Herald-Leader.