Ky. orders review after questions raised about use of calculator software on statewide tests

ACT forbids calculators with built-in algebra systems, 

but not those that have had software like Zoom Math added to them!?

This from the Herald-Leader:

The Kentucky Department of Education has ordered a review of how students use advanced calculators on a statewide college-readiness test, citing concerns raised by experts that the devices could be artificially inflating scores.

At issue is the math portion of ACT's COMPASS test, which is used as a placement test for high school seniors who have not met college-readiness benchmarks on the ACT test they take as juniors.

According to ACT rules, which are followed by the state, students are allowed to use certain calculators on the test, including ones that have been loaded with an algebra software program called Zoom Math. ACT forbids calculators with built-in algebra systems, but not those that have had such software added to them.

Stephen Newman, a math professor at Northern Kentucky University, said he is alarmed by the expanding use of Zoom Math, which allows students to plug in algebraic equations and get the correct answer without understanding how to do the math.

In late March, he and some colleagues conducted an experiment to measure the potential impact of Zoom Math on COMPASS test scores. They took the test 10 times, using calculators with Zoom Math to answer all equation problems. On any word problems, they simply chose the multiple choice "A" every time. In all 10 cases, they scored well above the cutoff.

"Based on our experiment, about 55 percent of COMPASS problems can be done without thought using Zoom Math," he said. "This has national implications and it could become a disaster if it continues."

COMPASS assessments are taken by more than 2.2 million students across the U.S. annually in high schools and colleges, which use them for class placement. About 18,000 Kentucky high school students took the COMPASS test in the 2012-13 school year, according to the state Education Department.

ACT is standing firm by its corporate decision to allow use of the add-on software.

"Through ongoing research, ACT ensures that our test administration policies remain fair and consistently enforceable," according to a statement from the Iowa-based company. "ACT math specialists continually review new calculator models, as well as algebraic systems and software, to determine their potential impact on test questions. Our current policies, which prohibit calculators with built-in algebra systems, remain in effect. We are confident that math test scores achieved under our current policies are valid and representative of student achievement."

Kentucky education officials aren't as confident. Ken Draut, director of assessment for the Education Department, has ordered a review to determine if Kentucky needs more stringent rules than those set by ACT.

"We'll be getting together with ACT to discuss our findings no matter what they are," Draut said. "At this point, I'm not worried because I don't think it's widespread, but I want to nip this in the bud for the future. I don't think it has had a big effect yet, but I am worried about the future if this is a program that provides a score we can't consider valid."

According to an Internet search, at least nine Kentucky school districts have purchased Zoom Math. Fayette County Schools do not use the program, but it is used by students on tests in Jefferson County, although the school district would not say by how many schools.

Newman is part of a group of math educators who developed a similar statewide assessment for the Kentucky Department of Education called KYOTE; it is forbidden to use Zoom Math on that test. (Newman said he has no financial interest in the KYOTE test.)

Zoom Math started as a tool to help special education students, said Tammy Herrada, the software company's CEO. Although its software has been around since 2006, Herrada said she heard of its use on ACT tests only in the past year.

The company has never advocated the software as a testing aid, she said.

"It allows students to go over the things they've learned in the past," she said. "We're not selling it to pass any ACT tests. We're selling it to teach and benefit."

Some school officials said its use appears to be expanding around Kentucky.

Cindy Beals, the district assessment coordinator for Warren County Schools, said the district's high school students have access to Zoom Math, but she does not know to what extent they use it on the COMPASS test.

"As long as they are allowed, then we allow our kids to use it," she said.

Tamela Porter, a guidance counselor at Bath County High School, said students use Zoom Math on the ACT and COMPASS tests.

"We were always told that it was fine since KDE had approved it," Porter said. "We use it in daily instruction ... We felt like if we are using it in instruction every day, we don't want to change that once they go take the ACT."

Newman said he doesn't think Zoom Math would have as much benefit on the regular ACT test because there are more analytical word problems.

Still, he would like to see other experts try his experiment and is angry that ACT hasn't moved to ban the software.

"They're wrong," he said. "We cannot continue this fraud. People are suspicious of gains in college readiness. It's our college readiness program that is being threatened."

"Career and college readiness" benchmarks are part of a 2009 overhaul of the state testing system in which ACT products were adopted at nearly every grade level. In 2013, the percentage of students deemed ready for college or a career jumped to 54.1 percent, up from 34 percent in 2010.

However, it's not clear what role, if any, Zoom Math might have had in the 20-point jump, since that score includes results from COMPASS, KYOTE, ACT and other career tests, such as ACT WorkKeys.

At least one other state has already banned use of Zoom Math on statewide tests. Mississippi took that action in 2011 after its education department concluded that Zoom Math "provided students with an unfair advantage and compromised our ability to make specific claims about student learning based on our assessment," said spokeswoman Patrice Guilfoyle.

Bob Schaeffer, a spokesman for the National Center for Fair and Open Testing, said new technology is constantly providing opportunities to give some kids a leg up.

"As high stakes testing moves into the computer area, teachers need to find ways to ensure that some students don't have tools that give them an advantage on the exam," he said.

Donna Caldwell, the district assessment coordinator for Madison County schools, said she heard of Zoom Math only a few weeks ago and doesn't know if individual schools might be using the software on tests.

But she said ACT and the Kentucky Department of Education need to address the issue as the availability of similar technologies expands.

"There's got to be some effect," she said. "Does it make the scores invalid? I don't know that you can say that. Do I think there's a huge impact? No, but a policy has to be developed, and it has to start with ACT."