ISAT Gains Questioned

By Larry Gavin

A study released on Oct. 4 titled The Proficiency Illusion claims that a substantial portion of the dramatic increases on the 2006 Illinois Standard Achievement Tests (ISATs) is due to changes made to the test, rather than improvement in student achievement. The 238 page study, conducted by the Northwest Education Association  (NWEA) and  the Thomas B. Fordham Institute, analyzed the tests used by 26 states to measure progress under the No Child Left Behind Act.

School District 65 Superintendent Hardy Murphy told the RoundTable, "We don't agree with that. We're seeing not only an increase in ISAT scores, but also an increase in the percentage of students scoring above the 50th-percentile rank nationally in reading and math."
Matt Vanover, spokesperson for the Illinois State Board of Education (State Board), told the RoundTable, "The Fordham Institute is formalizing a plan for a national curriculum and national standard. The Fordham folks are pushing a national agenda. The study has some shortcomings."

The Proficiency Illusion
In 2006, the State Board made a number of changes to the ISATs. Among these changes were altering the "cut scores," those used to determine whether a student met standards. With the exception of eighth grade math, the State Board maintains that the new cut scores were changed to convert to a new vertical scale and that they are equivalent to the old cut scores when it comes to measuring student performance. The State Board acknowledges it lowered the bar to meet standards for eighth grade.

The State Board also extended the time to take each section of the reading and math tests, which on a cumulative basis added up to 60 minutes to take the tests; a new firm was retained to prepare the test questions; and other changes were made.

 As part of the Proficiency Illusion study, researchers with NWEA analyzed whether the changes  made by the State Board in 2006 made it easier to meet standards on the 2006 ISATs compared to prior years. The NWEA did this by analyzing reading and math results from a group of elementary and middle schools in which almost all students took both the ISATs and the Measure of Academic Progress (MAP) test in 2003 and 2006. The MAP test was developed by NWEA and is used in more than 2,500 school systems in 49 states.

NWEA researchers concluded that the changes made it significantly easier for third and eighth graders to meet standards in reading on the 2006 ISATs and for fifth and eight graders to meet standards in math on the 2006 ISATs. Gains shown on the ISATs at those and other grade levels were not present on the MAP tests.

NWEA also estimated the impact the changes made by the State Board would have on the 2006 ISAT results, using the MAP test results as the yardstick. As an example, NWEA researchers determined that third grade students in 2003 had to score at the 52nd percentile on the MAP test in order to meet standards in reading on the ISATs; in 2006, third graders had to score only at the 35th percentile rank on the MAP test in order to meet standards on the ISATs. Since the MAP test was constant, they concluded that the changes to the ISATs increased the number of students meeting the ISATs by 17 percent - the difference between the percentile ranks on the 2003 and 2006 MAP tests.

Using this approach the NWEA researchers estimated that the changes made by the State Board increased the number of students meeting standards on the 2006 ISATs by the following percentages: third-grade reading -17%; fifth-grade reading - 3%; eighth-grade reading - 14%; third-grade math - 2%; fifth-grade math - 8%; and eighth-grade math - 27%.

The table below compares NWEA's estimated increases due to the changes made by the State Board and the actual increases experienced by School District 65 students on the 2006 ISATs for the grades and subjects indicated:
     

 

 

 

 

The Proficiency Illusion study also concluded that meeting standards in math on the 2006 ISATs was easier than meeting the math standards of 80% of the other states reviewed. For reading, the study concluded that it was more difficult to meet standards on the ISATs at the third grade level than 70% of the other states reviewed, but at the eighth grade level meeting reading standards on the ISATs was easier than 80% of the states reviewed.

D65 Points to Increases Percentile Ranks
Dr. Murphy said the study was apparently done as part of a national dialogue about using national standards versus state standards to measure progress under NCLB. "In our looking at it, regardless of what the cut scores are, we're seeing more students scoring at or above the fiftieth- percentile rank nationally," he said. "We're seeing demonstrated improvement."

Paul Brinson, director of information services for District 65, told the RoundTable that 50% of the District's third graders scored above the 50th- percentile rank in reading on the 2003 ISATs; the number grew to 79% on the 2007 ISATs. In math, the percentages grew from 61% on the 2003 ISATs to 74% on the 2007 ISATs.

Dr. Murphy said the District also had substantial gains at the fifth-grade level. In reading, an estimated 59% percent of the District's fifth-graders scored above the 50th-percentile rank nationally on the 2003 ISATs; the number grew to 80% on the 2007 ISATs. In math, the percentages grew from 73% on the 2003 ISATs to 85% on the 2007 ISATs.

Mr. Brinson said the percentile ranks for 2003 are all estimates.

"Our percentile shift is in the right direction," Dr. Murphy said.

The State  Board's response
Mr. Vanover told the RoundTable there are a number of problems with the Proficiency Illusion study. He said, "We don't know the size of the sample they used. We don't know how many students or what schools take the MAP test." He added, "What they fail to take into account is that there is a framework for Illinois learning: the Illinois learning standards. What they discount is that students are actually learning. What they're saying is that if you have a large number of students taking a test and the scores are going up [there is something wrong]."

The Proficiency Study disputes Mr. Vanover's comments. The study states, "NWEA curriculum experts evaluate the particular state's content standards" and "The MAP test is purposely aligned to each state's standards." The study also states that enough students were in the sample for each state to make the results reliable; an appendix to the study states the sample sizes in Illinois ranged from 1,179 to 1,254 students.

Before changing the cut scores in 2006, the State Board conducted a "Bridge Study" in an attempt to make the new cut scores it adopted equivalent to the old cut scores. With the exception of eighth-grade math, the State Board maintains that "the only thing that has changed is the numerical value of the cut score. Thus, meeting the Illinois Learning Standards according to the new vertical scale requires as much knowledge as (and no more than) the old ISAT scale did."

The State Board acknowledged that it lowered the bar to meet standards for eighth-grade math. The Bridge Study states the State Board did so because the score had been placed too high and was out of line with the other grades. The Bridge Study estimated that the change to the cut score for eighth-grade math would increase the number of students meeting standards by at least 26 percentage points.

Board Questions Placement Process and Lack of Progress

Report Prompts Board Ire About Low Minority Representation in ETHS Honors Classes

By Jennie Berkson

A report profiling the District 202 freshman class was a source of frustration to Board members, as it underscored the limited number of students of color in honors classes. It also raised questions about testing and placement procedures and parental influence in that process.

The report, presented to the Board at its Sept. 24 meeting, showed that only 14.2 percent of black students and 10.1 percent of Hispanic students were in Honors Humanities classes, compared to 68.2 percent of white students. Minority students were similarly underrepresented in mathematics honors classes: 13 percent of black students and 10 percent of Hispanic students, compared to 63 percent of white students.

"It's been at least 10 or 15 years that we've been trying to get more minority kids into honors class," said veteran Board member Margaret Lurie. "[These percentages seem] kind of pathetic."

"It seems to me that there's not a great push to do that sort of inclusion," agreed Jane Colleton, who is in her fifth term on the Board. "We're doing things the way we've always done them and not very successfully."

"I did not expect it to be this harsh," Board President Martha Burns said. "This is personal and very important. There is something institutionally and foundationally wrong here. We're not making the kind of inroads we should be in a community that spends the kind of money [we do and] that has the kind of resources we have here in this building."

"I can't speak for the last 10 to 15 years," replied Superintendent Eric Witherspoon. "What I can speak to is what you're going to be able to measure as we place these kids [in the future].

"We know there is a high correlation between students' having high expectations and being placed in rigorous courses and their level of achievement," he said. "We know that the vast majority of students will meet those expectations. One of the things we are determined to do is set in place a system where students will get the support they need in their classes, period."

Ms. Burns had requested the report in conjunction with a recently formed committee consisting of representatives from both District 202 and District 65: she, Omar Khuri and Dr. Witherspoon from District 202 and Mary Erickson (Board president), Keith Terry and Superintendent Dr. Hardy Murphy from District 65. Ms. Burns said that the committee's work will be devoted to developing a policy regarding the transition of students from District 65 to 202, an issue of major significance in the recent Board elections.

The report, prepared by Dr. Richard Bowers, associate principal for grades 9 and 10, cited other freshman class statistics, described the current process by which freshmen are placed in levels at ETHS and detailed current programs designed to facilitate the transition from District 65.

"Using the EXPLORE test as our primary basis, we discuss with [District 65 administrators and staff] our recommended placement for each incoming student in these areas [humanities and mathematics] and also in science," explained Dr. Bowers.

He also said school policy dictates that parents have the final say in their children's placement.

"It sounds as if the parent component has some ability to trump the other two,"

"Our school policy states that if parents disagree with placement they can state where they want their child placed in terms of level, and that's where they are placed," Dr. Bowers said.

Board member Mary Wilkerson asked for an explanation of the relationship between the ISAT test and the EXPLORE test. "Some parents say ‘according to the ISAT test my son or daughter is doing well. I don't know why the high school is saying they have a problem.'" (See sidebar for discussion of tests used by the two districts and testing terminology).

"In Illinois there is a significant difference between the ISAT and the Prairie State Achievement Examination," said Dr. Judith Levinson, director of research, evaluation and assessment. "The state has recognized this alignment is not in place between state tests."

Dr. Levinson explained that the EXPLORE test is linked to the ACT, which is part of the Prairie State Achievement Examination (PSAE), used to determine if the District is meeting its goals under the No Child Left Behind Act. That is why, she said, the EXPLORE test is used to determine levels for students when they enter ETHS.

She also said the reason there might be a discrepancy between student performance on the ISAT and placement at ETHS is that the "cut score" for meeting standards on the ISAT is lower (35 - 36th percentile) than the standard for being at grade level at ETHS (50th percentile). That difference, she said, could account for a student's being placed in a lower level than might have been expected given the ISAT score.

Dr. Levinson also said by the time students get to 11th grade and take the PSAE, they are required to be "somewhere around the 60th percentile" in order to be considered as meeting standards. District 202 has consistently not met standards, based on the PSAE in all tested areas for all subgroups.

In a separate e-mail to the RoundTable, Dr. Witherspoon further supported the use of EXPLORE: "Since the State of Illinois requires that all juniors take the PSAE/ACT, we use the EXPLORE and PLAN for baseline information and to measure growth using assessments aligned with the Illinois high school assessment system."

However, he also said we need to "consider multiple assessments and student information in placing more students in more rigorous classes."

Despite the administration's defense of the test, Ms. Colleton was not appeased. "I came away feeling as if the EXPLORE test is a gatekeeper in terms of getting into honors classes," she said. "I have to wonder how well parents understand their power in this, their options."

Ms. Burns shared with the RoundTable some of her concerns about parental involvement. "I wish I could see more parents of color coming forward," she said. "People of color feel that it [a change in placement of students] will not happen, ... but that is not an excuse. That is a defeatist attitude."

However, she said three parents had contacted her after viewing the Board meeting on television. They said their efforts to get their child's level changed had been met with more resistance than might have been expected, given Dr. Bowers's statement about the school's policy.

Ms. Burns said one parent reported the school denied the request to change the student's level. The other two, she said, told her they felt they had been required to go through "bureaucracy" that was "draining" before their request was granted.

Dr. Witherspoon told the RoundTable, in response to the parents' experience, "We will make sure we have an uncomplicated way to collaborate with parents on behalf of their children, and we will make sure it is easy for parents to partner with us in the course selection process."

Ms. Burns decried the imbalance in placement at ETHS that she said makes for classrooms that are populated for the most part either by black students or by white students. "We need more heterogeneous classrooms,' she said. "Do we have to have our own Brown v. Board of Education at ETHS to correct this?"

Testing Terminology and Definitions

ISAT: The Illinois Standards Achievement Test measures individual student achievement relative to the Illinois Learning Standards. Students are tested every year from third grade through eigth grade, at least in reading and mathematics, less frequently in science and writing.

PSAE: The Prairie State Achievement Examination (PSAE) measures the achievement of grade 11 students relative to the Illinois Learning Standards for reading, mathematics, science and writing. The PSAE includes three components: (1) the ACT (an exam used for college entrance) Plus Writing,; (2) an Illinois State Board of Education-developed science assessment, and (3) two WorkKeys assessments.

Scores on the ISAT and the PSAE are used to determine if a school or a district is making Adequate Yearly Progress (AYP) in compliance with the No Child Left Behind Act (NCLB)

The EXPLORE program is designed to help eigth and nineth graders explore a broad range of options for their future. Part of the EXPLORE program is an achievement test that covers reading, mathematics, science and writing. The content of the EXPLORE tests is closely tied to that of the achievement tests in the ACT. As discussed above, the ACT is a major component of the PSAE.

NORM-REFERENCED ASSESSMENTS measure one person's performance relative to others. A norm-referenced test is designed to discriminate among individuals in the area being measured and to give each individual a rank or relative measure regarding how he or she performs compared to others of the same age, grade, or other subgroup. Norm-referenced tests are often used to tell how a school or school district is doing in comparison to others in the state or nation. The EXPLORE test is a norm-referenced assessment.

CRITERION-REFERENCED ASSESSMENTS allow interpretation of a test-taker's score in relation to a specific standard or criterion. The individual's score is based not on how he or she does in comparison to how others perform, but on how the individual does in relation to absolute expectations about what he or she is supposed to know. The ISAT is a criterion-referenced assessment.

PERCENTILE - A percentile rank indicates that the individual student did better than a certain percentage of students in the group. For example, being at the 36th percentile means that one did better than 35 percent of the others in the group.

Sources: Illinois State Board of Education website www.isbe.net, ACT website www.act.org, Gale Encyclopedia of Children's Health: Infancy through Adolescence and Grolier Multimedia Encyclopedia.

D65 Board Invites Applications For Open Board Position

The District 65 School Board will select a new Board member to fill the vacancy created by Sharon Sheehan's resignation. Persons interested in serving on the Board should submit a letter of interest and/or resume by Oct. 9 to Pat Markham, District 65 Board Secretary, Joseph E. Hill Administration Center, 1500 McDaniel Ave.
The Board plans to interview candidates on Oct. 15 and to select a new Board member at its meeting scheduled for Oct. 15.

District 65 School Board Extends Superintendent Murphy's Contract Through 2012

By Larry Gavin

The District 65 School Board, by a split vote, agreed to extend Superintendent Hardy Murphy's contract for an additional three years at its Sept. 24 meeting. Dr. Murphy's current contract was scheduled to expire in June 2009, so the three-year extension will carry his tenure through June 2012.

Mary Erickson, Bonnie Lockhart, Jerome Summers and Keith Terry voted in favor of the three-year extension. Mary Rita Luecke and Sharon Sheehan voted no. Katie Bailey voted present, saying she disagreed with the process and timing of the vote.

The emotionally charged meeting was attended by almost 100 persons, 16 of whom spoke either in support of or in opposition to the motion. Community members supporting the extension pointed to the substantial rise in test scores of African-American students in the last few years and the narrowing of the achievement gap.

Persons opposing the extension said that "all too often African-American children are emerged in skill and drill, rote memory regimes"; that "many, many Evanston families have left the District both because of its inability to challenge some of their students, and its refusal to provide others with special education services they desperately need"; that "achievement [for black and white students] has been ignored at Oakton for years on end"; that "defining success as meeting the minimum standards on the ISATs is not good enough for this community and is a disservice to our children."

Many persons asked the Board to delay the vote until next spring so new Board members would gain additional knowledge about the District's operations and so the community could provide input on the issue.

The contract extension was discussed earlier by the Board in a closed session meeting held on Sept. 10. Board members apparently made up their minds during that closed session meeting. The Board was permitted to discuss the matter in closed session under the Open Meetings Act.

The "Yes" votes
Board president Mary Erickson defended the process. She said contracts with superintendents are often extended on a rolling basis as "a vote of confidence." She said every year the Board evaluates the Superintendent, and that was one of the bases for the decision to extend his contract.

Responding to statements that the Board should obtain community input before moving ahead with the vote to extend Dr. Murphy's contract, she said gathering community input is an ongoing process: "We constantly gather information about how our administration is performing."

She added that the new Board members just went through a campaign during which they had numerous meetings with community members and they had "a close-up-eye view of how the community thinks the administration is performing."
In explaining her vote, Bonnie Lockhart said, "I really feel strongly it's time to celebrate the many successes under the leadership of Dr. Murphy. ... Over the past five years, the number of students who meet or exceed standards has steadily increased. The administration has narrowed the minority achievement gap by 59 percent - a fact that has not been accomplished for over 40 years - while continuing to raise the achievement level of all students, so no one has suffered as a result of this." She said, "My vote tonight is a vote of confidence for Dr. Murphy and his staff."
Keith Terry said, "It's my firm belief that you reward people who work for you. Dr. Murphy is our superintendent. There is no other mechanism by which we can reward him."

Mr. Terry also said, "Student achievement is dramatically up. I really do appreciate that ISAT is not the measure by which we should grade ourselves. Unfortunately, the real world dictates that we do. It's the law. As an African-American parent I am so pleased to be in this City where across the board every cohort is up. I celebrate that. I have to applaud Dr. Murphy and his staff for this achievement. The teachers have to be applauded too."

Mr. Terry added that District 65 has prided itself on trying to attack the achievement gap, which has "confused and puzzled this country for a long time." Referring to recent data showing the gap has substantially narrowed in District 65, he said, "It appears to me District 65 is onto something. I'm proud of that."

Mr. Terry also noted that under Dr. Murphy's leadership, "We've had six years of balanced budgets. We've had five years of operating surpluses. ...This is a huge feather for this District." He also said that Dr. Murphy has a strong work ethic and has met the Board's objectives.

Responding to requests that the Board defer the decision until next spring, Mr. Terry said, "I don't know how we would have more information next year. We need to move on to more substantive business and get it done. By all accounts Dr. Murphy has done a good job."

Jerome Summers said, "I see scores going up across the board, black kids, Latino kids, special ed. kids, kids who have historically struggled, their scores have gone up dramatically over time." He added that school buildings have been maintained or improved, that technology is improving, that the budget has been balanced five years in a row, and that Dr. Murphy has received five positive annual evaluations in a row.

Mr. Summers also threw race into the picture, questioning why the extension of Dr. Murphy's contract generated such an uproar when an extension of a white superintendent's contract at District 202 went largely unnoticed. Parents opposing Dr. Murphy's extension have said they are raising legitimate concerns affecting the education of their children.

Ms. Erickson said that "good and stable leadership" is "the best gift a Board can give its organization." In evaluating a superintendent, she said, "the two key things you look at are the budget and educational progress." As to the budget, she said the District is going to receive the highest rating Illinois gives school districts. In terms of educational progress, she said, "We've made more progress in closing the gap than we ever have in the history of District 65."

Ms. Erickson defended an extension that would take the contract through for five years. "Take politics out of this so we can concentrate on education now," she said.

The "No" and "Present" votes
Mary Rita Luecke opposed a five-year contract, saying, "I fundamentally disagree with five-year contracts. I don't know anyone who has that kind of job security. I think when one gives a five-year contract, you've given away significant opportunity to hold a superintendent accountable." Ms. Luecke acknowledged many positive things going on in the District, but she said there were problems. "We define success too narrowly when it comes to student achievement. ...What we need to be looking at is not just meeting AYP [annual yearly progress under the No Child Left Behind Act], but every student meeting their potential.

"The rise in test scores comes at the expense of learning," Ms. Luecke continued. "Every parent with children in grades three to eight knows that from January through March, a significant amount of time is spent on test preparation, which cuts into real teaching and learning in this District. Nor is test preparation the kind of learning that leads to long-term gains in academic achievement."

Ms. Luecke also said there has been a decline in student enrollment above what demographic data would predict, that some families with special education children are not receiving mandated services, and that the District is projecting budget deficits in the next few years. She said the operating budget was balanced this year only because the District was paying for technology with debt, rather than out of operating funds "I'm very concerned about the future of this District," she said.

Ms. Luecke moved that Dr. Murphy's contract be extended to June 2010, rather than 2012. That motion failed.

Sharon Sheehan said she had discussed why she believed Dr. Murphy fell short in confidential evaluations which she felt she could not discuss in public. She said, though, "My vote is based on what I believe education can and should be in Evanston-Skokie District 65."

Ms. Sheehan listed numerous areas where she felt the District needed to change. She said the District should conduct earnest evaluations of academic program content and implementation and follow through on the evaluations; build a budget out more than one year as provided in the Long Range Financial Plan; make decisions based on information the Board has confirmed is accurate, thorough and honest; treat teachers as professionals; include parents in the education of their children, without treating them as the enemy; rebuild the community's trust in District 65 schools, so that families choose to stay in the schools; put education and not interest group politics at the forefront of discussions; strive for educational excellence and measure learning by more than test scores; and craft an educational vision responsive to the community's expectations.

She said, "Most importantly, I believe we can and should unite and inspire the members of our community to believe in educational excellence for all. I believe we can do better. Therefore, I vote no."

Katie Bailey voted "present." She said during the campaign she promised she would not support a contract extension for the Superintendent four or five months after being elected. "It's too soon for a brand new Board, in my belief, to make the most consequential decision we make."

Ms. Bailey said before the Board voted to extend the Superintendent's contract "we need to have a dialogue that leads to agreement on a vision about where we want the District to be ... We have an opportunity to bring the community together. Instead, we are making a decision that may divide us for a while and erode trust." She added, "I believe we must take time to sit as a new Board, to listen, to watch and to learn."

Ms. Bailey said, "I see no compelling reason to set aside my principled belief that this is the wrong time to bring this issue to a vote. I believe it's economically irresponsible to commit to five years, beyond the life of this Board, without taking the time to do it right." Ms. Bailey said her vote was on the process, not the person.

Dr. Murphy's remarks
Dr. Murphy said, "We have here in District 65 all the ingredients to demonstrate without equivocation that all students can and will succeed, regardless of rank, privilege or color, that those who traditionally struggled can excel without compromising those who traditionally do well ... I truly believe we will, in fact, become again the award-winning District that has led all of its children to the summit of educational excellence."

"I look forward to the opportunity to continue working with our very fine teachers, with our administrators and support staff to address the challenges and opportunities to meet the learning needs of our students ... and helping each and every child reach their full potential, unleashing their imagination in the 21st century and to make their dreams come true."

D65 Geometry to Remain at ETHS

By Larry Gavin

The District 65 School Board decided on Oct. 1 to keep offering geometry and algebra II classes for eigth-graders at Evanston Township High School. District 65 administrators had recommended that the classes be shifted to the middle schools. The cost of providing the classes at ETHS versus teaching them at District 65 is about the same.

John Benson, a 30-year math teacher at ETHS, told Board members, "I strongly urge you to do everything you can to keep this course at ETHS."

He said ETHS teachers can do a better job teaching the class because they know how geometry fits into the next two years of math at ETHS. "We have a whole bunch of kids who really get excited about this course," he said. "Kids get turned on to mathematics."

In a poll of the Board, Keith Terry, Katie Bailey, Mary Rita Luecke, Jerome Summers and Bonnie Lockhart all said they wanted the program kept at the high school. The administration may continue to examine ways to improve scheduling the classes at the high school.

D65 Adopts Final Budget for 2007-08

The District 65 School Board unanimously adopted a final budget for fiscal year ending June 30, 2008 at its Sept. 24 meeting. Operating revenues are projected at $89.2 million, up $2.6 million or 3.0 percent over the prior year. Operating expenses are projected at $88.8 million, up $4.4 million or 5.3 percent over the prior year. The budget shows a surplus of $404,000 for the year.

A number of adjustments were made after the Board approved the tentative budget on Aug. 20 to reflect additional revenues from federal and state funds, and additional expenses to maintain broadband and to purchase middle school math and elementary school social studies textbooks. The tentative budget showed a surplus of about $390,000.

While voting to adopt the budget, Board members Sharon Sheehan and Mary Rita Luecke expressed concerns that the budget built in new expenditures of $595,000 which would be carried through to subsequent years and increase the operating deficits for those years. The District is currently projecting a deficit of $1.2 million for 2008-09, unless adjustments are made. The Board's Finance Committee is addressing the projected deficits.

In the last ten years, from FYE 1998 to FYE 2008, the District's operating budgets have increased from $64.5 million to $88.8 million. During the same period, the District's student enrollment, including pre-K, has declined from 7,223 to 6,474 (budgeted number for 2007-08).

For more information concerning budget assumptions, new initiatives, expenditures by function and the projections, see the Aug. 22 issue of the RoundTable.