On June 17, District 65 School Board members discussed an evaluation of an algebra pilot program that was established at the start of the 2012-13 school year at King Lab and Bessie Rhodes magnet schools. The pilot eliminates tracking and collapses Algebra 1(high school Algebra 1) with Algebra 8 (eighth-grade algebra).
Algebra 1 has historically been for students at or above the 80th percentile rank. Algebra 8 has been offered to all other students. Algebra is the only course that is tracked at District 65. A recent report issued by the Brown Center on Education Policy found that in 2011, 76% of the schools in the nation track algebra classes in eighth grade.
The pilot had a contentious beginning. Board members first learned about the pilot about a week before school began last Fall. Members of the Board’s Policy Committee felt the pilot violated a directive that any proposed changes in the math program be brought to the full Board before being implemented. The Board asked for an evaluation of the pilot after the school year and directed that it not be expanded without Board approval.
At the June 17 meeting, administrators said the algebra pilot is achieving its goal of increasing the number of students placed in geometry as their next math course. Board members, though, raised questions about the data presented and the conclusions that could be drawn from the data. They also noted potential detrimental effects on low-performing students.
The Board, by consensus, decided to continue the pilot next year, but not to expand it beyond the magnet schools to the middle schools in the coming school year. The Board will also use next year to set goals for the pilot and assess it, using additional data.
Suzanne Farrand, mathematics director, said the pilot eliminates tracking. Students are not assigned to an algebra class based on their achievement level or preparedness.
"The algebra pilot is a multi-level algebra course which means it has all kinds of kids in it," said Ms. Farrand. "It has kids from the zero percentile to the 99th percentile." Most of the students in the Algebra Pilot were eighth-graders, 21% were seventh-graders.
The pilot uses Connected Math Project 8 (CMP 8) as the textbook, which Ms. Farrand said is the text book used in the Algebra 8 class. She said it is a rigorous textbook, that it covers all the topics covered by the textbook used by the Algebra 1 class, and is the textbook favored by math teachers.
Ms. Farrand said the pilot was established to answer two questions: 1) Can CMP 8 be used successfully as an algebra text for all students regardless of achievement level? and 2) Would a mixed-level class result in more students earning placement in a geometry course in 2013-14, in comparison to the current Algebra 1 and Algebra 8 courses?
She later defined the issue: "The real thing that we’re testing here is what happens to kids who aren’t necessarily slated to go to geometry. Do those kids benefit from [having] higher-achieving kids in their classroom?"
Ms. Farrand stated three findings:
• Acceleration: "What we found is that, in general, kids move up from quartile to quartile regardless of the course and regardless of the subgroup. So far the pilot is the same as the other two courses," she said.
• Growth: "A significant majority of students demonstrated above typical growth, regardless of the course and regardless of the subgroup. Again, what we’re seeing is that the pilot doesn’t have any particular edge over the other two configurations."
• Next-Year Placement: "The big difference is in the subsequent course placement. The majority of students completing Algebra 1 – and those are our top students – and the Algebra Pilot will take geometry in the 2013-14 school year. The majority of Algebra 8 students will take algebra."
The analysis did not compare changes in the percentile ranks of students or shifts in stanine levels, which provide a much more nuanced analysis of any impact for both high-achieving and low-achieving students. In addition, the analysis did not present student results on the year-end placement exams.
Superintendent Hardy Murphy said, "We think the results are encouraging, intriguing at the very least, and suggest to us that the pilot itself may be tapping into some potential for student excellence and academic achievement that we hadn’t heretofore tapped into."
Placement in Geometry Courses
The data focused on by administrators is that 78% of the students in the pilot will be placed in a geometry course next year. Data showing tentative placements suggests that about 46% will be placed in geometry honors, 27% in geometry and 4% in geometry with supports.
"What was really encouraging to us is that the Algebra pilot had almost 80% of the kids moving to geometry, primarily as freshmen," said Ms. Farrand. "That’s about 80% of the black students, and about 90% of the multi-racial students [in the pilot]."
Ms. Farrand said, "The big difference is the kids who are lower achieving are getting opportunities they might not otherwise have."
Board member Claudia Garrison said, "I think that’s a very good thing." She explored expanding the pilot on a limited basis to one of the middle schools next year.
Board member Eileen Budde said, "That’s very hopeful, but I think the question will remain for a couple of years to see how well they do when they get there. It’s really good they passed the placement tests to get [placed in geometry], because that means their algebra skills are solid. But I’m very curious to see what happens to those kids over the next two years."
Ms. Budde also raised three points that relate to whether the pilot’s results may be fairly compared to the results of students taking Algebra 1 and Algebra 8. First, she said the results of students taking Algebra 1 and Algebra 8 should be "blended" to make a fair comparison with the pilot. She said 44% of all students who took Algebra 1 and Algebra 8 were placed in geometry honors for 2013-14, which is roughly the same as the 46% rate for students in the pilot. As far as placements in geometry honors, there is not much difference.
Board President Tracy Quattrocki said placement in geometry honors was a key measure of success, because students placed in geometry honors have an increased chance of succeeding in an Advanced Placement class at Evanston Township High School by senior year.
Second, Ms. Budde said that students in the magnet schools may have started out with a higher achievement profile than students in the middle schools, which may skew the results.
Data presented by Ms. Farrand shows that 74% of the students in the Algebra pilot scored at or above the 50th percentile on the Fall 2012 MAP test, while only 65% of the students who took Algebra 1 and Algebra 8 scored at or about the 50th percentile on that test.
Ms. Budde also pointed out that 13% of the students in the pilot were in the bottom quartile, while about 19% of the students in Algebra 1 and Algebra 8 were in the bottom quartile.
If students in the pilot started out with a higher achievement profile, higher year-end results would be expected.
Third, Ms. Budde raised a question about class size. Data presented by Ms. Farrand showed that the average class size for the pilot was 16.4 students, compared to 20.2 for students in Algebra 8 and 27.6 for students in Algebra 1.
"The pilot had a low class size. I think we should expect significantly better results with that kind of personalized attention that kids can get with only 16 kids in a class," said Ms. Budde. "I think that’s a huge advantage."
Ms. Farrand presented data showing that a class of 25 students in the pilot had placement results comparable to the results of the pilot as a whole.
Dr. Murphy said the only way to tell how the mixed-level algebra class would fare with the large class levels at the middle schools was to pilot the program at one of the middle schools.
Ms. Budde pointed to data that showed there were over 120 eighth-graders who scored in the bottom quartile in math on the Fall 2012 MAP test.
The data indicated that 31 students in the bottom quartile who took Algebra 8 classes, or 36% of those students, moved up to a higher quartile during the year.
By contrast there was no progress in the pilot. There were 11 students in the bottom quartile in the beginning of the year, and 11 in the bottom quartile at the end of the year.
Ms. Budde said, "the lowest performing kids are being served well" in Algebra 8, and it "worries me" there is not comparable progress in the pilot.
Board member Richard Rykhus said he was concerned about this data. "One thing we’ve talked about over the last 15 months is to try to go back to a ‘root-cause analysis,’ because the problem isn’t happening in the sixth or seventh grade. It’s happening somewhere else along the line. And I don’t know where it is and I don’t know if any of us know exactly where it is." He urged, though, that the District do more to prepare students early on so they will be better prepared by eighth grade.
Ms. Quattrocki said, "As a Board, if we can see no one’s moving out of that bottom quartile in the pilot, we have to look at that and say that’s concerning to us."
One recent meta-analysis found that some low-performing students may benefit by taking algebra earlier rather than later, but it says, "under-prepared students who are admitted to algebra do not fare well." The study also adds a caveat that "achievement gains occurred in settings where policies were accompanied by strong supports for struggling students, particularly more time for algebra instruction. We do not have strong evidence that universal algebra policies lead to achievement gains minus those strong supports." Algebra: A Challenge at the Crossroads of Policy and Practice, Review of Educational Research 81, no. 4 (2011) 453-492.
Goals for the Pilot And an Assessment Plan
Board member Candance Chow suggested that administrators use a "control group" to assess the pilot. She added that the Board needed to develop criteria by which to assess the pilot. "We need to talk about significant improvement in terms of the numbers of kids going to geometry. But we also have to talk about no detrimental effect to other students overall. … I do see a potential detrimental effect to kids who are struggling and we want to make sure this program is a win/win overall."
Mr. Rykhus said, "I think Candance nailed it." He said the Board needs to determine how to evaluate the program "to move kids forward at the high end and not have a detrimental effect on other kids."
Ms. Quattrocki recapped that the pilot would continue next year, but there would be no expansion. She said the Board would determine goals for the pilot and how to assess the pilot, administrators would analyze the data in light of comments made by Ms. Budde and Ms. Chow, and take a closer look at how low-performing students were doing.