D65 and 202: The Quagmire of Different Tests and Results

By Larry Gavin and Jennie Berkson

School District 65 administrators said at a School Board meeting on Nov. 5 that District 65 has made substantial gains on the Illinois Standard Achievement Test (ISAT) and that those gains erode at Evanston Township High School by the time eleventh-graders take the Prairie State Achievement Examination (PSAE).

Paul Brinson, director of information services for District 65 said, "One of the things the high school says is if we sent them better students, they could do better. Well, we're sending them pretty good students."

Mr. Brinson's remarks come on the heels of District 202's being placed in restructuring status for failing to meet Adequate Yearly Progress under the No Child Left Behind Act for black, Hispanic and other student subgroups for the fifth year in a row.

"We know that there is a lot of good work with students going on at District 65," Eric Witherspoon, Superintendent of District 202 told the RoundTable. "Students leave eighth grade and make the transition to high school at lots of different levels and proficiencies. Some are highly accelerated and other students come struggling greatly."

Judith Levinson, director of research, evaluation and assessment at ETHS, says the ISATs and PSAEs are not aligned and that an eighth-grader's ISAT results cannot be used to predict how he or she will do as an eleventh grader on the PSAEs.

During the same time-frame in which District 65 has made substantial gains on the ISATs, the District's eighth-grade students' performance on another test, the EXPLORE test, has been flat. The EXPLORE test is "designed to measure students' curriculum-related knowledge and the complex cognitive skills important for future education and careers," according to the technical manual for the test.

The various tests raise questions concerning whether achievement levels are eroding at the high school and/or whether a substantial number of students are being graduated from District 65 without the knowledge and cognitive skills to succeed in high school and later in college.

The ISAT vs. PSAE debate
Mr. Brinson presented slides at the Nov. 5 District 65 School Board meeting that showed District 65's gains on the ISATs during the last four years. Black and Hispanic eighth-graders have made substantial gains, particularly on the 2005 and 2006 ISATs. The percentage of black eighth-graders who met or exceeded standards in reading increased from 54% to 71% between 2004 and 2006. In math, the percentage of black eighth-graders meeting or exceeding standards increased from 37% to 75% between 2004 and 2006. Hispanic eighth-graders showed similar gains.

To make his point that gains at District 65 are eroding at ETHS, Mr. Brinson presented a chart that showed the percentage of District 65 eighth-graders who met standards on the 2004 ISATs, versus the percentage of ETHS eleventh-graders who met standards on the 2007 PSAEs. The cohort analysis showed substantial drops standards on the 2004 ISATs, versus the percentage of ETHS eleventh-graders who met standards on the 2007 PSAEs. The chorot analysis showed substantial drops at the eleventh-grade level. An abbreviated version of Mr. Brinson's chart is shown below:

Relying on a study conducted by MetriTech, Inc. for the Illinois State Board of Education that analyzed the relationship between ISAT and PSAE scores, Mr. Brinson said that eighth-grade ISAT results are predictive of eleventh-grade PSAE results. He said if 54% of black eighth-graders met standards on the 2004 ISATs, he would expect that about 54% of that cohort group would meet standards as eleventh graders on the 2007 PSAE test. Instead, scores dropped from 54% on the 2004 ISATs to 33% on the 2007 PSAEs.

In commenting on the drop, Mr. Brinson said, "There really is something not going on for those students, because we would expect there'd be higher percentages of students meeting expectations."

Mr. Brinson added based on the 2007 ISAT scores of the eighth-grade class that just graduated from District 65, "we would expect that 82% would meet standards in reading and 88% percent would meet standards in math on the 2010 PSAEs."

Superintendent Hardy Murphy said the growth in eighth-graders' ISAT scores was obtained through an instructional environment that included differentiation of instruction, enrichment and an array of support systems.

"If students find similar kinds of support awaiting them as they matriculate from our system out into the next system, then you will see those correlation coefficients being validated," he said. "If they move into an environment that's substantially different, then you're going to see a difference in terms of their performance from what would be predicted based upon the State analysis."

The MetriTech study Mr. Brinson says he relied on states that an eighth-grader's ISAT "performance level" (i.e. "warning," "below standards," "meets standards" or "exceeds standards") matched his or her "performance level" as an eleventh grader on the PSAEs 62% of the time in reading and 70 % in math. Mr. Brinson acknowledged in an interview with the RoundTable that this meant that eighth-grade ISAT performance levels would not match eleventh-grade PSAE performance levels in reading 38% of the time and in math 30% of the time. Mr. Brinson said, though, that for the students whose performance levels did not match, he would expect that one-half of the time their performance level would go up and one-half of the time down, so that overall eighth-grade ISAT scores would predict eleventh-grade PSAE scores.

Judith Levinson, director of research, evaluation and assessment for District 202, told the RoundTable there is no data supporting the assumption that one-half the scores would go up and one-half down, but it would depend on how the scores were spread. She also said that Sam Krug of MetriTech told her that performance on the ISATs is not predictive of performance on the PSAEs. Andrea Preston, communications specialist for ISBE, told the RoundTable the same thing.

Statewide results on the 2004 ISATs and 2007 PSAEs appear to support claims that performance levels on the eighth-grade ISATs are not predictive of performance levels on eleventh-grade PSAEs. The Statewide trends are similar to the trends at ETHS.

 For example, on a Statewide basis, the percentage of eighth-grade white students who met or exceeded standards on the 2004 ISATs in reading was 76%; the percentage dropped for those students as eleventh graders to 64% on the 2007 PSAEs. For black students the percentage dropped from 48% on the 2004 ISATs to 28% on the 2007 PSAEs. For Hispanic students the percentage dropped from 51% on the 2004 ISATs to 33% on the 2007 PSAEs.

"This is a big problem for Illinois," Dr. Levinson told the RoundTable. "It's not good for kids, and it's confusing for parents. The tests need to be aligned so that there is one set of standards along the way, K through 12."

The EXPLORE Test
During the same period that District 65 has made substantial gains in increasing the number of students who meet or exceed standards on the ISATs, the District has also increased the number of students with a national percentile rank of 50 or above on the national norm portion of the ISATs, or in other words, it has increased the number of students performing among the top one-half of students in the nation.

In reading, the percentage of white students in the top 50th percentile in the nation increased from 77% on the 2003 ISATs to 90% on the 2007 ISATs. For black students the percentage increased from 37% in 2003 to 60% in 2007. For Hispanic students, the percentage increased from 44% in 2003 to 70% in 2007.

Despite these increases, the performance of District 65 eighth-grade students on the EXPLORE test has been generally flat. The percentage of white, black and Hispanic students who scored above the 50th percentile rank in the nation on the EXPLORE test in December 2003, 2004, 2005, and 2006 is as follows:

• White students: 95%, 95%, 94%, and 95%;
• Black students: 50%, 49%, 47%, and 51%.
• Hispanic students: 63%, 55%, 55%, and 60%.

Dr. Levinson said the passing standard for EXPLORE is the 50th percentile. In a memorandum dated Oct. 4, she said, "For EXPLORE, you need to be at least at the 50th percentile to meet standards and be on track to pass the PSAE/ACT and be successful in college work."

Using EXPLORE as the measure, about 50% of District 65's black eighth-grade students and about 40-45% of the District's Hispanic eighth-grade students have not met standards for the last five years. District 202 provides remedial programs for many of these students.

Various theories have been offered to explain why District 65 students have shown substantial gains on the ISATs, but shown flat performance on the EXPLORE test.

Mr. Brinson said, "The EXPLORE test is not aligned with State standards. It is not the standard we work toward. It is not the appropriate standard." He said the test is used as a "counseling tool" for eighth-graders entering the high school.

Dr. Murphy said, "The EXPLORE test is an attempt to grab national standards and come up with some kind of alignment with the State standards." He added, "With the PSAE, there is more of an alignment with our State standards so the instructional precision is greater."

Dr. Levinson has a different view. She told the RoundTable that the EXPLORE test is part of the same family of tests as the ACT, that it is directly aligned with the ACT and that the ACT is given as a part of the PSAE tests.

She thinks that students may do better on the ISATs than EXPLORE because the ISAT cut score to "meet standards" corresponds to the national percentile rank of the 37.5 percentile rank, while the passing standard for students on the EXPLORE test is the 50th percentile rank.

Another theory is that changes to the 2006 ISATs have made it easier to pass the ISATs, while the EXPLORE test has not changed. See sidebar.

The Technical Manual for the EXPLORE tests, which cover English, mathematics, reading and science, states "These tests are designed to measure students' curriculum-related knowledge and complex cognitive skills important for future education and careers...The fundamental idea underlying the development and use of these tests is that the best way to determine how well-prepared students are for further education and for work is to measure as directly as possible the knowledge and skills needed in those settings."

The EXPLORE test thus provides benchmarks for college readiness, which are minimum scores required for students to have a high probability of success in college, by the time they graduate from high school. For the EXPLORE tests the composite benchmark is 16.25 out of 25 scale points. For the ACT tests, the composite benchmark is 21.25 out of 36 scale points.

The table below gives the composite scores for white, black and Hispanic eighth-grade students at District 65 on the 2004 EXPLORE tests and as eleventh-graders at ETHS on the 2007 ACTs. The benchmark for college readiness is in parenthesis:

2004 EXPLORE        2007 ACT
White  19.4 (16.25)     27.6 (21.25)
Black   13.6 (16.25)     18.6 (21.25)
Hisp.    14.9 (16.25)     21.2 (21.25)

Many black and Hispanic students fall short of EXPLORE's college readiness benchmark when they transition to ETHS. While white and Hispanic students make gains in relation to the benchmarks for college readiness during their high school years, the composite score for black students still falls short of that mark.

Green Grant for Rhodes

This year, the curriculum at the Rhodes Magnet School has emphasized environmental awareness. Students and teachers pledged to increase their efforts to promote recycling but found themselves hampered by a lack of appropriate containers. Still determined to increase the amount of recycling, teachers at the Rhodes Magnet School applied for and were awarded a $200 grant from the Solid Waste Agency of Northern Cook County. The grant will fund the purchase of a recycling center, recycling bins and educational resource materials that will help the school meets its recycling goals.

D65 Unveils District Improvement Plan for 2007-09

By Larry Gavin

School District 65 administrators presented a District Improvement Plan for 2007-09 at the Board's Nov. 5 meeting. The plan identifies nine strategies and 40 action steps designed to meet the needs of all District 65 students, as well as strategies to target specific student subgroups.

Superintendent Hardy Murphy said, in introducing the plan, "This initiative includes focused professional development for teachers and other instructional staff and developing creative student support programs. The use of an improved database for planning, delivering, and monitoring instruction and programmatic implementation also characterize this initiative."

Assistant Superintendent Valerie Moore said a focus of the plan is based on the hypothesis that "the effective use of data to plan and guide instruction will improve student achievement." All nine strategies focus on the effective use of data.

For example, one of the strategies is to use data to improve differentiation of instruction for all students. The plan says, "The District will develop a framework for planning for differentiated instruction, using data about student readiness, interest and learning profile." The goal is to differentiate instruction to "ensure all students receive appropriate learning opportunities."

Data will also be used in the continued implementation of the Unified System of Delivery (USD) of instruction, which Dr. Murphy has characterized as a major cultural change for the District. The premise of USD is that the instruction of all students, including special education students and English-language learners (ELLS), should be rooted in the District's general curriculum. The District says it plans to use data concerning a student's achievement level to provide high-quality instruction matched to students' needs, to provide appropriate interventions and supports, to monitor student progress frequently, and to make changes in instruction or interventions in light of a student's progress or lack of progress.

"We have had substantial improvement in student achievement based on the interventions that have been put in place," said Paul Brinson, director of information services. To maintain that progress, "We have to begin to more finely differentiate the interventions that we're using," he said.

The plan also states that data will be used to develop an effective framework to monitor District programs and instruction; to effectively allocate resources; to improve instruction for English-language learners; to improve the academic achievement of low-income students; to develop and implement culturally responsive instruction for African-American, Latino and other student subgroups; to develop a comprehensive professional development plan; and to determine the most effective communication strategies to reach targeted audiences.

Dr. Murphy said, "What you see here is a transformation to use data in a more clinical way to meet the needs of individual students. ... We need more of a clinical analysis to create intervention plans for each individual child."

"The heart of this is about improving the delivery of instruction in classrooms," said Dr. Murphy. "That has been the heart of what we've been doing for the last six, seven, eight, nine years - consistent implementation of the District's curriculum."

Board members responded favorably to the plan and are expected to formally approve it at a later meeting.

Physical Education Gets Respect:

D202 Will Include Grades For PE in GPA

By Jennie Berkson

Physical education (PE) grades will be calculated as part of the grade point average (GPA) for Evanston Township High School students, beginning with the freshman class entering in fall of 2008. The District 202 School Board voted in favor of the proposal on Nov. 5.

"We are thrilled with the Board's approval of PE as part of the GPA," Physical Education Department Chair Shirley Nannini told the RoundTable. "We believe that this new policy is ultimately good for our students. There is no refuting the importance of exercise and its relationship to short and long term health. We also have a responsibility to educate students regarding fitness concepts, so that they can be knowledgeable and understand the physiology behind exercise."

Board members were generally supportive of the measure when it was proposed in October, as were department chairs and faculty. However, the Board sought parental input before making its decision because some concern was expressed about the impact on GPA for the highest-ranked students. At the current time, PE does not offer honors credit, so a student who normally takes only honors or AP courses might experience a slight decrease in his or her GPA with the inclusion of the PE grade.

Despite this concern, Board members subsequently reported that students and parents had provided overwhelming support for proposal. Of 51 emails received in response to a request for feedback, "the vast majority were in favor" of the proposal, Board members said. Missy Fleming, an enthusiastic supporter of the proposal offered a "third" to the second to the motion for approval.

Ms. Nannini acknowledged that "we still have some details to work out" but she was confident that plans would be in place by the time the freshman arrive, suited up, in September 2008.

Termination of IMAGE Test Creates Challenge For D65

The State Board of Education advised School District 65 that it will no longer administer the IMAGE test for English Language Learners (ELLs), Susan Schultz told the Board on Nov. 5. She said the test did not meet the requirements of a federal review panel.

"The concern for us is we have a dual-language-immersion model that is a research-based model - and for which we have been commended by the State - that indicates that language acquisition takes five to seven years. This is not consistent with the assessment system that the State is requiring for us. That's a problem for us," Ms. Schultz said.

With the elimination of the IMAGE test, which contains simplified English, ELL students will need to take the ISATs. Ms. Schultz said the administration is waiting to hear from the State Board what accommodations will be made for ELL students in taking the ISATs; the District will then develop a plan so ELL students can succeed on those tests.

Ms. Schultz said the administration is meeting with other school districts to exchange ideas on how to deal with this development.

D65 Board Decides Not to Implement Moment of Silence

By Larry Gavin

The District 65 School Board decided Nov. 5 not to implement recent amendments to the Silent Reflection and Student Prayer Act. The Act, as amended, provides, "In each public school classroom, the teacher in charge shall observe a brief period of silence with the participation of all pupils therein assembled at the opening of every school day."

The Act goes on to say, "This period shall not be conducted as a religious exercise but shall be an opportunity for silent prayer or for silent reflection on the anticipated activities of the day."

A number of Board members expressed concern about the separation of church and state and about the state mandating how teachers should manage their classrooms.

Board president Mary Erickson said she was upset with the amendment and explained, "We are faced with the idea of ignoring the law or going along with something that many of us feel is simply not appropriate. So we are sort of backed into a corner."

Ms. Erickson suggested that the Board not enforce the state mandate at this time. She added that if the legislature incorporated the Silent Reflection and Student Prayer Act into the School Code, the Board could consider seeking a waiver from the State Board of Education. A majority of District 65 Board's members concurred.

The Board's action does not preclude any teacher from complying with the Act. On Nov. 6, State Senator Jeff Schoenberg asked the State Board of Education to adopt rules that provide clarity on how to satisfy the state-mandated moment of silence.

It is unclear whether the State will attempt to enforce compliance with the law, and, if so, whether it would take action against the school district or classroom teachers.

Changes to the ISATs May Make Test Easier

A number of changes were made to the 2006 ISATs, which some critics say are responsible for or contribute to the higher scores: 1) The State Board of Education sharply lowered the bar for eighth-graders to meet standards in math on the 2006 ISATs; 2) the State Board changed other cut scores for other grades and subjects, which it maintains had no effect on pass rates; 3) students were given an extra 10 minutes to take each section of the reading and math tests, which on a cumulative basis add up to an extra 60 minutes to take the tests; 4) a new firm was retained to prepare the test questions; and 5) other changes were made.

A study called "The Proficiency Illusion" concluded that a substantial portion of the higher pass rates on the 2006 ISATs were due to these changes; this is disputed by the Illinois State Board of Education. See Oct. 17 issue of the RoundTable.

D65 2007 ISAT Comparison

By Larry Gavin

The Illinois State Board of Education released report cards for all schools in the State on Oct. 31, which contain each school's results on the 2007 Illinois Standard Achievement Test (ISAT). The report cards contain data that show how students in each school are doing; they also provide Statewide averages.

In addition, this year the Chicago Tribune created an online tool that enables the user to compare a selected school with other schools in the State. Among other things, the online tool ranks each school in the State against 3,093 schools in the State on the basis of overall ISAT scores. It also ranks schools on the basis of the school's ISAT scores for different ethnic groups.

This article provides some comparative information concerning District 65's schools, derived from the State report cards and the Chicago Tribune's online tool.

On an overall basis, 85.9% of School District 65's students met or exceeded standards on the 2007 Illinois Standard Achievement Tests (ISATs) across all subject areas, compared to 83.6% in the prior year. The Statewide average increased from 77.0% to 78.7%.

Performance by school
On average, District 65's schools did better on the 2007 ISATs than about two-thirds of the schools in the State. Willard's average ISAT score put it among the top 96 percent of the schools in the State. Oakton's put it among the bottom 20 percent.

The accompanying table No. 1 gives the percentage of students who met or exceeded standards on the 2007 ISATs at each school in the District, the percent of low-income students at each school, and the rank of the school among 3,093 schools in the State. The State rank of each school was determined using the Chicago Tribune's online tool.

Scores by ethnic group - 4th and 8th grade
District 65's white, black and Hispanic students each did substantially better on average than their Statewide counterparts.

White students: At the fourth-grade level, 98.6% of the District's white students met standards in reading and 99.3% met standards in math. The Statewide average was 82.6% for reading and 92.7% in math. In seven of the District's schools, 100 % of the white fourth-grade students met standards in reading; for math the count was nine schools. The low point at any school was 94%.

At the eighth-grade level, 97.6% of the white students met standards in reading and 99.6% met standards in math. The Statewide average was 87.1% for reading and 89% for math. In two of the five schools with eighth-grade classes, 100% of the white students met or exceeded standards in reading; for math, the count was four schools. The low point at any school was 94%.

Tables No. 1 and 2 reflect the percentage of white fourth- and eighth-grade students who met or exceeded standards on the 2007 ISATs for each school in the District, as well as the school's State rank on the basis of white fourth- and eighth-grade ISAT scores. The number following the slash in the State rank column is the number of schools included in the Chicago Tribune's online comparison. The number of schools ranked for white students is substantially lower than the State total, because schools are not ranked unless they have the requisite number of white students in fourth or eighth grade to report ISAT data.

Black students: At the fourth-grade level, 65.1% of the District's black students met standards in reading; and 81.8% met standards in math. This substantially exceeds the Statewide averages of 50.2% in reading and 68% in math.

At eighth grade, 73% of District 65's eighth-graders met standards in reading and 75.6% met standards in math, compared to Statewide averages of 69.8% and 61.5%.

The results for black students differed substantially, depending on the school they attended. For example, the percentage of black fourth-graders who met or exceeded standards in reading ranged from a low of 33% at Oakton to a high of 85% at Bessie Rhodes. In math the range was a low of 46% at Orrington to a high of 100% at Bessie Rhodes, Dewey and Willard.

Tables No. 1 and 2 reflect the percentage of black fourth- and eighth grade-students who met or exceeded standards on the 2007 ISATs for each school in the District, as well as the school's State rank on the basis of black fourth- and eighth-grade ISAT scores. The number of schools ranked for black students is also much lower than the State total, for the reason stated above.

Hispanic students: In reading, 90.4% of the District's fourth-grade Hispanic students met standards on the 2007 ISATs, in math 92.8% met standards. The Statewide averages were 68.5% and 85.5% respectively.

At the eighth-grade level, 80.3% of the District's Hispanic students met standards in reading; 82.7% met standards in math. The Statewide averages were 76.2% and 76.8% respectively.

Table Nos. 1 and 2 do not include data for Hispanic students because comparative data was not available for 12 of the District's schools.

NAEP Tests: 2/3 of Illinois Students Not 'Proficient'

The Nations Report Card shows that only about one-third of Illinois students performed at a "proficient" level on the 2007 National Assessment of Educational Progress (NAEP) tests. The NAEP is given to a sampling of fourth- and eighth- graders every two years in all 50 states as part of the No Child Left Behind Act. About 700,000 students were given the NAEP in 2007. The results are reported in the Nation's Report Card under the auspices of the U.S. Department of Education.

For reporting purposes, the NAEP uses four achievement levels: below basic, basic, proficient and advanced. The "proficient" level is defined as having "demonstrated competency over challenging subject matter, including subject matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter."

On the 2007 NAEP tests, 32% of Illinois' fourth-graders met or exceeded the proficient level in reading and 36% met or exceeded the proficient level in math. More than twice as many students met or exceeded standards on the 2007 ISATs: 74% of fourth graders met standards in reading and 86% met standards in math.

The dichotomy is similar at the eighth-grade level. On the 2007 NAEP tests, 30% of Illinois eighth-graders met or exceeded the proficient level in reading and 36% met or exceeded the proficient level in math. On the 2007 ISATs, 82% of eighth-graders met or exceeded standards in reading and 81% in math.

Four D65 Schools Among Top in State

Dewey and Willard schools ranked among the top 50 statewide elementary schools, and Bessie Rhodes Magnet and Haven Middle schools ranked among the top 50 suburban middle schools in the 2007 Chicago Sun-Times poll that ranks schools based on average scores on the state reading and math assessments.

In that poll, among the over 2,600 elementary schools in Illinois, Willard School is ranked #8 and Dewey School is ranked #31.  Among all elementary schools in the six-county suburban districts, Willard is ranked #2 and Dewey is #22.  For suburban middle schools, Bessie Rhodes is ranked #33 and Haven is ranked # 43.