OSDE changes led to a big jump in student test scores, but did performance really improve?

A change to Oklahoma’s method for scoring state tests will cause student proficiency rates to jump by as much as 33% even while internal state records indicate actual academic gains have been stagnant.

School district leaders say the state Department of Education has not disclosed the upswing is due in large part to a new scoring system rather than improvements in public school performance.

Oklahoma schools and families received preliminary test scores this month, showing dramatic increases statewide. But public documents and internal data show the state lowered the amount of knowledge students had to demonstrate in order to score at a proficient level this year.

Records show math and English language arts scores would have been similar or a tick lower than 2023 results across all grade levels had the state not changed its scoring system. Students only in grades 3-8 take state tests in English and math.

More: Former OSDE employees describe a department in disarray under Ryan Walters after federal report

In the past, the state Education Department disclosed when new scoring methods caused major statistical differences in test results, but Oklahoma school districts say that hasn’t happened this time.

The state Department of Education declined to comment before publication of this story.

This summer, the agency established new performance expectations for math and English language arts tests. It then applied the new grade-level expectations, called cut scores, to the tests Oklahoma students took in the 2023-24 school year.

Oklahoma Voice received the new cut scores and a description of the revision process through an open records request to the Office of Educational Quality and Accountability, whose commissioners approved the new scoring system last month.

As a result of the changes, 51% of Oklahoma third graders have been labeled as proficient or advanced in English language arts, according to the cut scores, which also reflect test results the Education Department posted on its website.

Only 28% of third graders would have been considered proficient or higher under the previous cut scores, according to the Education Department’s internal data analysis, which Oklahoma Voice obtained from a source involved in the state assessment process.

That rate is similar to third-grade achievement levels from the previous school year.

Fifth-grade results in English language arts had the greatest boost from the new scoring method. About 59% of fifth graders will be considered proficient this year, an increase of 33%.

Without the recent scoring changes, their proficiency rate would drop to 26%. In 2023, the proficiency rate for fifth-grade English was 28%.

Math results also are expected to jump by 3% to 12%, depending on the grade level, with the new cut scores.

States often reset their cut scores after updating academic standards or when a new administration takes over the public school system — two factors that both apply to Oklahoma, said Marianne Perie, an assessment expert who has worked with Oklahoma and other states on past scoring revisions.

More: Federal agency joins criticism of Oklahoma education department's financial controls

What isn’t common is to distribute state test scores without informing schools or the public that a major statistical change has taken place, Perie said. Significant updates to a state’s cut scores mean the new test results should not be compared to data from the year before.

“I think what happens is it causes confusion in how to interpret the score,” Perie said.

The state Education Department has not yet released A-F report cards for public schools, which factor in state tests and other accountability measures.

However, the agency already posted state test results on its website, showing statewide averages much higher than last year. Districts have received their preliminary scores, and families have been provided their children’s results.

The state Education Department informed school districts through memos and newsletters that the agency reset grade-level expectations for state tests, a process called “standard setting.”

The standard setting process took place June 17-20 in Tulsa with 66 math and English teachers from across the state. The teachers developed recommendations for new cut scores, with help from state officials and a vendor that administers the state tests.

But, school leaders in multiple districts say the state agency hasn’t informed them that the standard setting process resulted in material differences to proficiency rates.

Multiple districts with schools that ranked in the bottom 5% of the state said the Education Department has not told them if the jump in scores will impact whether they will continue receiving federal school improvement funds, which support the lowest-performing schools.

The U.S. Department of Education did not return a request for comment on whether the state agency has notified the federal government of the scoring changes.

Even without an explanation from state officials, it’s obvious something must have changed, district leaders said.

Midwest City-Del City Public Schools Superintendent Rick Cobb said he’s never seen results grow or decline this significantly except in years when the state changed its scoring.

“We’re probably seeing some growth, probably hitting that point past the pandemic where statewide we’re reversing the learning loss, but we didn’t reverse 20% of it in one year,” Cobb said. “Nobody did.”

In Watonga Public Schools, Superintendent Kyle Hilterbran said he expected his students to show improvement, but the leap in statewide averages told him the state must have reset its scores. That makes it difficult to gauge exactly how much his students have grown academically.

“It’s really hard to gain knowledge from last year’s test because they changed the score so drastically,” Hilterbran said.

Hilterbran said he’s not opposed to changing the scoring system. He said the previous method set too high a bar to reach proficiency, and the new version might give a more accurate view of student performance in Oklahoma.

The last time Oklahoma test results changed so dramatically was in 2017 when the state updated its scoring methods to align with national standards.

The 2017 revision caused proficiency levels to drop by as much as 43% in a single year.

Back then, schools and the public were informed before test results were released that there would be significant differences. The state Education Department held media briefings before distributing the results to warn Oklahomans of the impending performance drop .

The state also overhauled the way it presented results to make it clear that the 2017 scores could not be compared to the previous year, said Perie, who helped oversee the process in Oklahoma seven years ago as an expert consultant. It replaced its old 400-990 scoring scale with a new range of 200-399, with 300 being the benchmark for proficiency.

This, along with advance public notice of major cut score changes, is considered best practice to ensure schools and parents don’t mistakenly conflate two wildly different data sets, Perie said.

Test results posted on the state Education Department’s website indicate the agency has kept the same 200-399 scale, despite significantly revising its scoring methods this year.

“It really hurts the ability to understand what’s going on with testing when we change things underneath but don’t change things on the surface,” Perie said.

Oklahoma Voice is part of States Newsroom, a nonprofit news network supported by grants and a coalition of donors as a 501c(3) public charity. Oklahoma Voice maintains editorial independence. Contact Editor Janelle Stecklein for questions: info@oklahomavoice.com . Follow Oklahoma Voice on Facebook and X .

This article originally appeared on Oklahoman: OSDE changes led to a big jump in student test scores, but did performance really improve?