HB 151: School Grading Systems and School Improvement

On Wednesday, April 3, 2013 the House Education Committee spent two more hours on HB 151, and the more testimony I hear on the bill, the more it sounds like something that is certainly well-intentioned, but which confuses the issue of school grading with school improvement and would be unworkable for Alaska.  Rep. Paul Seaton, Rep. Peggy Wilson, and Rep. Dan Saddler all sounded pretty skeptical of the efficacy of HB 151, and they all made good points in their discussion of the bill.  Staff from the University of Oklahoma and Oklahoma State University just published a report in January, 2013 on the Oklahoma school grading system: http://www.ossba.org/Websites/ossba/images/A-Fpdf_Reportresearchstudy.pdf .  The executive summary of the report is helpful in understanding the system and understanding why it may not be the best measure of school performance, particularly if the ultimate goal is school improvement.  If nothing else, I recommend reading the executive summary, which states in part:

 “There is an undisputed need to assess public schools to determine their effectiveness. Oklahoma responded to this need by legislating an assessment system intended to be comprehensive, while at the same time, understandable and transparent, using A-F grades as the reporting outcome.  Strengths of the assessment system are its inclusion of student achievement in a breadth of content areas, a measure of growth for low achieving students, and consideration of multiple artifacts in whole school performance.  However, the effort of the state to report school quality using the familiar letter grade, while laudable, falls short of providing a clear and credible picture of individual school performance for a variety of reasons outlined in this report.

It is our goal to support the good intentions of Oklahoma policymakers in their school improvement efforts by identifying methods of the grading system that may be potentially misleading (Baker & Linn, 2002).  Some problems with the A-F Report Card are unique to methods used by the Oklahoma State Department of Education (OSDE) to calculate student achievement, student growth, and school performance indices.  Other problems are longstanding conceptual and methodological constraints associated with aggregated test scores as measures of school performance.  Although achievement data are obviously important for assessing schools, an accountability grade based almost exclusively on test scores does not account for numerous critical factors that contribute to school performance.

            Performance measurement and accountability systems in many sectors take a more balanced approach to assessment.  In other enterprises, executives and managers are not expected to make strategic decisions based on outcome data alone. Healthcare, for example, has made great strides adopting scientific process and outcome measures for evaluation.   Manufacturing forecasts future profitability using measures of customer satisfaction, demand, internal processes, and innovation and growth, relying on much more than past financial reports.  It is now standard accounting practice to evaluate companies as much by their intangible resources as by their physical capital.  Sole reliance on outcome indicators produces biased assessment and does not depict fairly or accurately how school leaders and teachers respond to the dynamic needs of students.

Accountability systems are only useful if their measures are credible and clear.  Despite good intentions, the features of the Oklahoma A-F grading system produce school letter grades that are neither clear, nor comparable; their lack of clarity makes unjustified decisions about schools. Further, A-F grades are not productive for school improvement because they do not explain the how or why of low performance.  Building on what has already been done, Oklahoma can and should move toward a more trustworthy and fair assessment system for holding schools accountable and embracing continuous, incremental improvement.”