The role of learning gains in new school grades decisions
1/5/2016
As we’ve been covering closely over the past several weeks, and considering over the past few years, some major changes to the state’s testing and accountability system are expected to be finally decided at tomorrow’s State Board of Education meeting in Tallahassee.
Issues scheduled to come before the board for a vote tomorrow include approving the performance-level “cut scores” for evaluating student performance on the Florida Standards Assessments, as well as a likely decision on the revised school accountability grades thresholds to determine what qualifies a school to be rated an “A”, “B”, “C”, D”, or “F” school.
Within and around each of these rule considerations, there are multiple moving parts that can make it difficult to clearly determine the short and long-term impacts of any particular decision at hand, which is why we’ve tried to help inform the conversation leading up to these votes by pulling apart some of the different major set of influences to better understand the potential impacts of each.
One more critical aspect we want to be sure to highlight before the vote is what the lack of any learning gains points components in baseline new school grades calculations could mean, both this year and in ramifications for years beyond.
As a reminder and preface to this, our longstanding recommendation has been and remains that the most appropriate course of action – if at all possible – would be to not issue school grades at all for 2014-2015 due largely to the problems raised by not being able to calculate accurate learning gains between two tests, one of which is brand new (or worse, forcing an inaccurate calculation that doesn’t exist).
However if that is not a viable option, whether due to legal requirements or other considerations, it’s critical to be aware of the potential impacts to be weighed in whether school grading thresholds are adjusted to offset the lack of learning gains components in this year’s calculations or not.
To take a simple look at what learning gains-based performance components mean to Florida schools in the school grading process, we went back to the 2014 school grades calculation data and grouped schools into two groups:
1 – Proficiency Majority Points schools: Schools that earned more points toward their 2014 school grade in proficiency-based component categories than in learning gains-based component categories.
2 – Learning Gains Majority Points schools: Schools that earned more points toward their 2014 school grade in learning gains-based component categories than in proficiency-based component categories.
The size of the groups alone raised a potential early caution against producing baseline grades and establishing long-term grading thresholds based on the impact data of those calculations: in 2014 there were 657 schools statewide that earned more points in proficiency categories than learning gains, and 2,560 schools that earned more points in learning gains categories than proficiency.
We then looked at those schools again using the simulated 2015 grades data to see whether Learning Gains Majority Points schools were differently impacted by any of the proposed school grading rule scenarios than Proficiency Gains Majority Points schools.
Table 1 shows what the year-to-year changes in school grades would look like (based on FLDOE’s simulated 2015 school grades) if proficiency-only grades were calculated under each of the four currently proposed possible grading thresholds for these two groups of schools.
Table 1:
For example we see in the top two bars that if proficiency-only grades were calculated using the Commissioner’s proposed school grading thresholds, about 3.8% of Proficiency Majority Points schools from 2014 would see their grades drop and 21.1% would improve whereas 25.1% of Learning Gains Majority points schools would see their grades drop and 21.1% would see their grades improve.
As you look down at each of the pairs below that, you can see how each of the other recently proposed alternative school grading thresholds would differently impact schools with higher annual proficiency rates and schools with higher annual learning gains achievement statewide.
While Table 1 compares the groups based on percentages, Table 2 looks at the same comparisons based on raw numbers to give a better perspective on the magnitude of the number of Learning Gains Majority Points schools statewide that would be potentially disparately impacted by the calculation of proficiency-points only scores.
Table 2:
Beyond these impacts we can estimate, there are also just general questions to consider from a long-term planning perspective as to whether it makes sense to set school grading thresholds this year based on impact data calculated using proficiency points only, knowing that the thresholds that will be established are intended to be used against fully calculated scores (proficiency + learning gains) going forward.
Will the impact of these thresholds in future years be anything at all resembling the impact of the proficiency-only baseline scores they were structured around, or will the thresholds need to be tweaked again before 2015-16 grades are released in June?
What else do you see going on in this data or any of the other impacts we’ve looked into recently that should be considered going into tomorrow’s vote? Leave your comments or questions below, and be sure to watch tomorrow’s State Board of Education meeting tomorrow beginning at 9:00am on The Florida Channel.