Wednesday, January 14, 2009

Safe Schools, Information Systems Testing: Targets of Senator's Questions

This morning, Interim Education Commissioner Elaine Farris fielded questions from members of the Senate A & R committee for a couple of hours. She will do the same thing this afternoon in the House.

Senators queried Farris on a range of budgetary issues including the Kentucky Instructional Data System (KIDS).

The timing and nature of this questioning was interesting, given yesterday's report from state Auditor Crit Luallen. Luallen accurately expressed problems with the current state of data-collection at KDE and related agencies like CPE and EPSB. But it is not clear that senators see fixing these long standing problems as a priority. One referred to a data repository as a suppository.

The auditor's report underscores a persistent need to shore up data collection so that policy-makers have better data sets from which to make informed decisions. And there's a lot of different types of data. Some legislators seem to be looking for cuts and are much less open to making needed improvements.

KDE spokeswoman Lisa Gross told KSN&C that KIDS is designed to:
  • add longitudinal student tracking with both enrollment and assessment data
  • enable interoperability of data systems across district and state databases
  • create a data warehouse that combines demographic, assessment and financial data
  • create a foundation that will allow other sources of data to be added and searched/queried, such as data from CPE and EPSB
  • be the foundation for a more robust Knowledge Management Portal that will serve up a wealth of targeted instructional resources, including standards-based units of study, lesson plans, curriculum maps, assessments and other educational resources; the portal will offer a collaborative workspace that teachers can use to share best practices, develop test items and expand their professional skills
"KIDS will corral a lot of data from a school or district into one place and enable authorized users to view special reports. For instance, school administrators could generate reports that show student test scores, poverty level, attendance and more. Teachers could use KIDS to "personalize" their lesson plans for individual students,"
Gross said. And the KIDS system would be open to the general pulbic as well.

"It's really a data warehouse. The average person could use KIDS to generate spreadsheets showing school/district financial information (state, local, federal funding), enrollment, test scores, etc. And, it promises longitudinal data from test scores, which is a big plus."
The development of KIDS was funded through a federal grant from the NCES Institute for Education Science. Kentucky was one of 14 states to receive this funding. But in today's presentation Farris is telling legislators that among the current service reductions at KDE, KIDS is not being funded. $1.5 million per year is needed to operate the system annually.

Infinite Campus is in the final stages of implementation following a 2006 outlay of $10 million and another $4 million in 2008. Infinite Campus, like STI before it, collects the student attendance data that calculates average daily attendance for the SEEK formula. KDE really needs about $7 million to properly train districts to use the software. However required reductions have caused KDE to reduce the funding request to the bare minimum - $5.5 million in contract costs only. Given the headaches experienced by local school already trying to implement the system, look for an especially rocky implementation as new school districts are added in March.

One legislator hinted at sticking with STI for now to save money, but that would require KDE to renegotiate a new service agreement with STI. I wonder how those negotiations might have gone. KDE's Greg Rush said he didn't think that would save the state money.

3 comments:

Richard Innes said...

RE: Lisa Gross said, “The average person could use KIDS to generate spreadsheets showing school/district financial information (state, local, federal funding), enrollment, test scores, etc. And, it promises longitudinal data from test scores, which is a big plus."

Well, district financials are available now in the KDE Web site. Search for “Revenue and Expenditures” to bring up many years of data.

Likewise, enrollment data for districts is available for many years under the “Growth Factor Reports.”

Test Scores – Plenty of that in the Kentucky Performance Reports and annual CATS reports and PLAN, EXPLORE and ACT reports.

No one has a higher interest in longitudinal data than I do, but we won’t have anything valid until CATS gets fixed, first. You can’t do decent longitudinal work with a matrixed test. Sure, you can generate numbers, but what validity will they have given the fact that one year a kid gets tested on curriculum subset A, and then next year he gets subset B, and then maybe C the next year? I asked William Sanders about that once (he’s the guru of value-added testing) and he summarily dismissed the idea.

There is a lot of other “stuff” in the KDE Web site, as well. If KDE just adds school level membership (KDE already has that – I requested it and got it last year), and school-level financials, we will pretty much cover everything Lisa mentioned at a lot lower cost than KIDS. Besides, KIDS is stealing scarce programmer time from fixing MUNIS, a much more important need, IMHO.

KIDS mostly looks like a duplication of what we already have, or we should have once Infinite Campus is up and running (which is also straining those scarce KDE programmers). The one exception is the database interconnectivity to other state agency databases, but I’m not sure that is such a great idea. How many teachers want their KDE data freely available to the Finance Cabinet, for example? And, who is going to be happy if someone hacks into the fish and wildlife computers and then penetrates KDE’s databases through that hole? That is a potential threat with interconnectivity.

Richard Day said...

OK...I'll just remain confused. Maybe this is more about what you're against than what you're for.

I would think centralized data collections would be right in BIPPS's wheelhouse. You all talk about government transparency all the time...but now that KDE goes that route, y'all get cold feet.?!

15-stop-shopping is no route to transparency. In fact, if someone wanted to hide things from the public, a pretty good method would be to break the information up into as many pieces as possible and increase the effort needed to extract it.

...and my tech buddies tell me MUNIS is not the problem. (I hated MUNIS because I found it unfriendly operationally, but it generally gave me the information I needed to manage a school's accounts.) If you accurately tell it how to account for a set of expenditures - it does it as good as the next program. It's the chart of accounts that needs fixin'.

A brand new accounting package wouldn't fix the problem either if it's based on a faulty chart of accounts.

I am assuming that hacking and other systemic threats are always a problem. But that hasn't deterred BIPPS's call for increased transparency. Why not? It seems to me that concerns over security for employeers social security numbers and other sensitive data is a constant in any system.

Richard Innes said...

The Principal Writes,

“I would think centralized data collections would be right in BIPPS's wheelhouse. You all talk about government transparency all the time...but now that KDE goes that route, y'all get cold feet.?!”

You are confusing centralized data collections that solely benefit of the bureaucracy with a different issue of making useful data available to the public. Such useful data does not include privacy-invading material, which I understand could be swapped rather extensively from agency to agency using the KIDS system. That won’t help transparency a bit. In fact, the costs involved will probably just interfere with getting transparency data out to the public.

The Principal also writes, “15-stop-shopping is no route to transparency. In fact, if someone wanted to hide things from the public, a pretty good method would be to break the information up into as many pieces as possible and increase the effort needed to extract it.”

I agree with that, if we are talking about data that is suitable for the public. However, again, my understanding is that much of the money to be spent on KIDS won’t improve public access to anything. I think the major costs are to create an inter-agency data swap portal.

The Principal continues, “...and my tech buddies tell me MUNIS is not the problem. (I hated MUNIS because I found it unfriendly operationally, but it generally gave me the information I needed to manage a school's accounts.) If you accurately tell it how to account for a set of expenditures - it does it as good as the next program. It's the chart of accounts that needs fixin'.”

I guess we are mostly talking semantics here. The Chart of Accounts is an integral part of MUNIS – MUNIS is worthless without it. The Chart of Accounts is indeed a mess right now.

However, since you brought it up, MUNIS may have some serious technical issues that go beyond the Chart of Accounts. Awkward user interfacing, as you mention, is one. Furthermore, the system apparently has some security holes, as recent convictions of Johnson County and Louisville education folks indicate. In trials of the Johnson County crowd it turned out one of the conspirators was reprogramming MUNIS to hide the crimes. How did that happen without any flags showing up?

Finally, the Principal Writes, “I am assuming that hacking and other systemic threats are always a problem. But that hasn't deterred BIPPS's call for increased transparency. Why not? It seems to me that concerns over security for employeers (sic) social security numbers and other sensitive data is a constant in any system.”

Security is always an issue with computer systems, but what BIPPS envisions is a well-separated system with good firewalls and which only contains de-identified, aggregated reports. We are not interested in individual student’s social security numbers, learning disabled status, or eligibility for free and reduced cost lunches. However, other state agencies probably would like access to all of that data.

The sort of transparency system we envision would be much easier to control than something like KIDS, which could, and probably would need to, swap all sorts of individual and sensitive data with other agencies. It’s a risk-benefit software design issue that can drive up costs dramatically. So, while we certainly want more government transparency, that doesn’t extend to our accessing your social security number or last year’s individual tax return, or desiring a system that might be vulnerable to penetration by hackers that would want to steal those sorts of data bits.

So far, it sounds like the few features in KIDS that we would support can be implemented elsewhere at lower costs. For the majority of the KIDS mission, we don’t see much priority in the austere financial climate we currently face. Let’s fix MUNIS’ problems, first.