The Research Base of Open Court and
Its Translation into Instructional Policy in
California
by
This study analyses the research that
supports Open Court, describes its translation into instructional policy
in California, and compares the average SAT 9 reading scores of
English-only children in schools using Open Court against comparable
schools using non-scripted programs in one large urban school district. It
found no significant difference in the average second grade SAT 9
reading scores in Open Court and comparison schools. Furthermore, it
found no Open Court school had positive differences of 10 or more
percentile points between second and fifth grade whereas 21% of the comparison
schools did. Long-term Open Court schools had negative differences of 10
or more percentile points between second and fifth grade twice as often as
schools using non-scripted programs. Finally, long-term Open Court
schools serving communities where 97-100% of the children receive free /
reduced-price meals were significantly more likely to be in the bottom quartile
of the SAT 9 reading assessment than schools using non-scripted programs
serving similar children.
Ordinarily, professors of reading education write about principles in reading instruction and avoid naming specific commercial programs and schools. However, these are not ordinary times. In 1996 the reading / language arts teacher specialists on California’s Instructional Resources Evaluation Panel recommended that Open Court’s Collections for Young Scholars (1995) not be placed on the state’s textbook adoption list (Holland, 1996). Yet by January 2000, Open Court, as it is commonly referred to, had grown from being used in one in every hundred to one in every eight elementary schools in California (Helfand, 2000a). When the Los Angeles Unified School District, with over 300,000 K-5 children, moved into Open Court nearly district-wide the summer of 2000, Open Court’s share of the California textbook market increased dramatically.
The purpose of this article is twofold: (1)
to help educators and educational stakeholders understand the research that
supports Open Court and how it was translated into instructional policy
in California, and (2) to analyze the effect of this policy on reading
achievement in California.
The Research Base of Open Court
The story of Open Court’s phenomenal
growth in market share begins in Houston, Texas. In 1994-95, under the auspices
of the federal government’s National Institute of Child Health and Development
(NICHD), a team of researchers led by Barbara Foorman of the University of
Texas—Houston Medical Center studied the effect of various types of reading
programs on the reading achievement of low-achieving,
economically-disadvantaged, first and second grade children in the Alief
Independent School District in Texas. The programs they studied were: (1) the
pre-publication version of Open Courts’ Collections for Young Scholars,
(2) the researchers’ adaptation of Hiebert, Colt, Catto and Gray’s 1992
program, (3) the researchers’ adaptation of contemporary reading instruction,
and (4) the ongoing contemporary reading instruction in place in the district
before the study began. They collected data on children in various classrooms,
each with one of the four programs. Foorman and her colleagues found the
children in the classrooms with Open Court improved in word reading more
than the children in the other classrooms. Consequently, they concluded that
the “[r]esults show advantages for reading instructional programs that
emphasize explicit instruction in the alphabetic principle for at-risk
children” (Foorman, Francis, Fletcher, Schatschneider, & Mehta, 1998, p.
37).
There
are many problems with the research. For one, the study received financial and
personnel support from Open Court’s publisher at the time, Blouke Carus.
Another problem is that the version presented to the California State Assembly
Education Committee May 8, 1996, a few months after Open Court was
purchased by McGraw-Hill, and the version published after peer review in the Journal
of Education Psychology in 1998 used considerably different data (Taylor,
1998, 333-338).
Still another problem is that, in both the
prepublication and the published versions, the sample favored Open Court
(Taylor, 1998, pp. 333-338; Foorman et al., 1998, Table 1). In the
published version most (9/13) of the classrooms with Open Court were in
schools where 40 to 43% of the children were on the federal lunch program; most
(10/15) of the classrooms with the adapted Hiebert et al. program were
in schools where 64 to 65% of the children were on the federal lunch program;
most (16/25) of the classrooms with the adapted contemporary reading
instruction were in schools where 50 to 64% of the children were in the federal
lunch program; and most (10/13) of the
classrooms with the ongoing contemporary instruction were in a school where 71%
of the children were on the federal lunch program (Foorman et al., 1998,
Table 1). Children who are not economically disadvantaged typically do better
on standardized tests of reading achievement than children who are economically
disadvantaged. The school in the study that had 71% of the children on the
federal lunch program also had the lowest achievement scores on the statewide
test in Grade 3 (Foorman et al., 1998, p. 39).
On all measures in both the prepublication
and the published versions, the children in the classrooms with Open Court
instruction had higher average pre-test scores than the children in the other
classrooms (Taylor, 1998, pp. 333-338; Foorman et al., 1998, pp. 43-44).
In the prepublication version which was used to influence legislation,
two-tailed t-tests show significant differences between Open Court
and each of the other groups’ pretest averages on word reading: for the Open
Court vs. the adapted Heibert groups, t(88) = 3.184, p<.01; for the Open
Court vs. the adapted contemporary groups, t(85) = 2.199, p<.05; for the
Open Court vs. the ongoing contemporary groups, t(78) = 4.091, p<.001. In
the published version, in 1 of the 12 possible comparisons the difference
reached significance (first grade Open Court vs. adapted Hiebert on phonological
processing, t(91) = 3.287, p<.01) and in five of the 12 possible comparisons
the differences tended towards significance.
Another problem is the way the data was
aggregated. The data on the children who had been in the district’s classrooms
with contemporary reading instruction when they were in first grade but in the Open
Court classrooms when they were in second grade were averaged with the data
on the children who were in the classrooms with Open Court (Foorman et
al., 1998, Tables 3 & 4), making it impossible to distinguish the
effect of each program.
Most problematic of all is the operational definition of reading employed by Foorman and her colleagues. They equated pronouncing print words out of the context of text with reading. That is, they assumed that if a child can pronounce a print word the child is “reading”, regardless of whether the child is making sense of the print or not. Most reading experts today have moved beyond this traditional definition of reading as pronouncing print to a new definition of reading as making sense of print.
At the end of the six-month study, despite
their lower pre-test scores and their greater economic disadvantage, the
children in the classrooms with the ongoing contemporary reading instruction
answered comprehension questions about text they had read silently better than
the children in the Open Court classrooms (Foorman et al., 1998,
Table 5). If one defines reading as making sense of print, one would
have to conclude from the NICHD Houston study—contrary to Foorman et al.’s
conclusion—that contemporary reading instruction is more effective than
traditional, parts-to-whole reading instruction. This reanalysis of Foorman et
al.’s findings is consistent with independent research that has found that
all children, but especially lower-achieving children, achieve more with
contemporary reading instruction than with traditional reading instruction
(e.g., Anderson, Wilkinson, and Mason, 1991; Cantrell, 1999; Eldredge, Reutzel,
and Hollingsworth, 1996; Mullis, Campbell, and Farstrup, 1993; Reutzel and
Cooter, 1990; Sacks and Mergendollar, 1997).
Despite its flaws, the NICHD Houston
research was used to justify policy and legislation that excluded contemporary approaches
to reading instruction and favored the purchase of Open Court,
especially in schools serving poor families (Coles, 2000; Taylor, 1998; Taylor,
Anderson, Au and Raphael, 2000).
How did this flawed research come to effect
instructional policy in California? First, it was presented to policy makers
during a time when the print media was repeatedly portraying California’s
reading / language arts framework in place from 1990-1995 as a failure.
California’s relatively low scores on the 1992 and 1994 NAEP reading assessment
were blamed on California’s reading / language arts framework in place at the
time—a framework virtually the same as those of the states with the highest
scores on the same NAEP reading assessments.
Second,
it was widely reported in the press. One typical newspaper article entitled: Study:
Phonics is the best way of learning to read quoted Foorman as saying “We
discovered that (phonics) children were 6.03 times as likely to be reading more
than one word at the end of the year than whole-language children” (Oakland
Tribune, 1997, p. A5). One of McGraw Hill’s divisions is a media division
with the capability of putting messages on the news wire services.
Third, other researchers were not allowed to
review the research, despite repeated requests, until after it had been used to
justify legislation (Coles, 2000; Taylor, 1998; Taylor, et al., 2000).
Finally, the research was presented to
legislators and other policy makers as “reliable, replicable research”. May 8,
1996, a year before it was submitted to the Journal of Educational
Psychology for peer review, Foorman and the NICHD Branch Chief for Child
Development and Behavior, Reid Lyon, presented it to the California State
Assembly Education Committee. Foorman spoke of the “impressive gains” of the Open
Court group and the “alarming results” of the adapted Hiebert and
contemporary reading instruction groups (Taylor, 1998, pp. 334, 336).
Open Court became the program of choice for policymakers without reading /
language arts expertise. December 1996,
the California Board of Education overruled the reading / language arts teacher
specialists on its Instructional Resources Evaluation Panel and added Open
Court to California’s textbook adoption list. In 1997, the Board of
Education of Sacramento City Unified School District adopted Open Court for
55 of its 60 elementary schools, K-6, and accepted a grant from the David and
Lucile Packard Foundation for Open Court coaches (Saunders, 1999).
In 1999, after the second statewide
administration of Harcourt-Brace’s Stanford 9 Achievement Test (SAT 9),
grades 2 through 11, Sacramento’s use of Open Court was used to leverage
the market around the state in schools and districts with low SAT 9
scores. Marion Joseph, a California School Board member with a B.A. in
political science, widely viewed as the most influential person in California
on reading education policy (e.g., Colvin, 2000), testified on the success of Open
Court in Sacramento to the Los Angeles Unified School District’s Board of
Education. Duke Helfand, a reporter with no professional preparation in
education (personal communication), wrote in a front-page article in the Sunday
edition of the Los Angeles Times about Sacramento’s success as measured
by the SAT 9 in its almost exclusive use of Open Court. He
reported “The primary grades led the way, with second-graders making the
largest gains: 15 percentile points in reading compared with 4 points among
their counterparts across the state” (Helfand, 1999, p. A1). William Ellerbee,
Jr., Associate Superintendent of Elementary Schools in Sacramento, wrote in a
front-page article in the newsletter for the California Association for
Supervision and Curriculum Development, that “SCUSD tripled the state average
gains in elementary school reading...on the SAT 9” (Ellerbee, 1999, p.
1).
In their analyses of Open Court’s
success in Sacramento, Helfand and Ellerbee compared Sacramento’s scores in one
grade in 1998 to scores in the same grade in 1999. In so doing, they analyzed
teaching, not learning. Teachers tend to teach in the same grade from one year
to the next while children move up a grade from one year to the next. To
compare the same—or nearly the same—children, we need to compare scores from
one grade, one year, to the next grade, the next year.
When we follow Sacramento’s children from
second grade to third grade, we find 35% of the second graders scored above the
national average in 1998 and 37% of the third graders scored above the national
average in 1999. Hence, while Sacramento’s second grade teachers raised the SAT
9 by 15 percentile points between 1998 and 1999, Sacramento’s children in
second grade in 1998 and third grade in 1999 raised their SAT 9 scores
by 2 percentile points during the 1998-1999 school year. Statewide, 40% of the
second graders scored above the national average in 1998 and 41% of the third
graders scored above the national average in 1999. Hence, Sacramento’s actual
gain was one percentile point more than the state’s.
June 15, 1999, Los Angeles Unified’s Acting
Superintendent Ruben Zacarias wrote the LA Unified Board of Education:
The Los Angeles Unified School District is
in the process of developing a collaborative program with the K-3 Reading
Alliance, which is composed of the Los Angeles County Office of Education
(LACOE) and the National Center to Improve the Tools of Educators (NCITE). The
K-3 Reading Alliance includes support from the Los Angeles Times, and the
Von/Pavilions “Teach the Teachers Collaborative” … In order to participate in
the project, low-performing schools must use a research-based reading program,
such as Success for All, Open Court, or Reading Mastery.
The National Center to Improve the Tools of
Educators is housed at the University of Oregon and affiliated with the
National Institute for Child Health and Development.
August
8, 1999 the Publisher and the Editor of the Los Angeles Times
recommended Los Angeles Unified adopt Open Court, Reading Mastery,
or Success for All for it’s low-achieving, high poverty schools because
they are “research-based programs”. Open Court and Reading Mastery
(formerly DISTAR), both owned by McGraw-Hill, were on the state adoption
list and so could be purchased with state textbook monies. Success for All
was not on the state adoption list so schools selecting Success for All
would have to find $17,000 of non-textbook monies to purchase the program.
The research that supports each of these
programs has been sponsored by or conducted by people with a vested interest in
the outcome of the research. All three programs are parts-to-whole, scripted programs.
Scripted programs differ from non-scripted programs in that they do not allow
teachers to adjust the program to accommodate the learning needs of the
children. Anderluh (1998), writing about Open Court in Sacramento,
observed “[t]he program lasts two to three hours a day, with half that time
spent in whole group instruction. The detailed scripting, combined with the
over sight of roving coaches, means that from school to school, teachers are
presenting basically the same lesson at a given grade level in a given week.”
In October 1999 the Los Angeles Unified
School District Board of Education adopted a policy requiring all of its 444
elementary schools to use Open Court K-5 beginning fall 2000 unless they
use Success for All, Reading Mastery, or achieve above the 50th
percentile in reading on the SAT 9 at grades 2 and 3. The kindergarten
and first grade teachers would receive Open Court training July 2000;
the second and third grade teachers would receive training January 2001; and
the fourth and fifth grade teachers would receive training July 2001. Those who
did not adopt Open Court by July 2000 would miss out on the K-3 training
[personal communication from teachers in LA Unified]. The overwhelming majority
of schools in the Los Angeles Unified School District scored below the 50th
percentile on the SAT 9. During the 1999-2000 school year, 311 LAUSD
elementary schools “chose” to join the 49 schools in the district already using
Open Court (Helfand, 2000c).
April 30, 2000, the LA Times ran
another front-page article (Helfand, 2000b) which lauded the adjacent Inglewood
Unified School District for its high SAT 9 reading scores and attributed
the scores to the use of Open Court. While the clear implication of the
article was “If Inglewood can do it, so can LA”, important differences in the
two districts were ignored. While 74% of the children in LA Unified receive
free / reduced-price meals, only 61% of the children in Inglewood Unified
receive free / reduced-price meals. While 45% of the children in LA Unified are
limited English proficient, only 36% of the children in Inglewood are limited
English proficient. Furthermore 30% of the children in Inglewood Unified have
parents who graduated from college, a rate higher than the national average,
and12% have parents with graduate degrees (Rothstein, 2001). While we do not
know the percent of children in LA Unified who have parents with college
degrees, it’s doubtful, based on our knowledge of the area, that the rates in
LA Unified reach the national average, much less exceed it. The article also
failed to note that there are large (-11, -24, and –26) negative differences in
the average SAT 9 reading scores between second and fifth grade in all
three long-term Open Court Inglewood Unified schools. Nevertheless, Los
Angeles Unified sent its teachers to Inglewood to learn how to teach reading.
During the 1999-2000 school year other
school districts serving large numbers of economically disadvantaged children
such as the Long Beach and Oakland Unified School Districts switched to Open
Court. They were joined by San Diego City Unified during the 2000-2001
school year.
Since LAUSD has begun using Open Court in
kindergarten and first grade we have been hearing from multiple sources that
K-1 teachers no longer have time to teach science, social studies, math, or
writing. From an instructional point of view, this is of concern for several
reasons. Background knowledge in science and social studies is necessary to
make sense of texts on science and social studies topics. Math is typically the
first area of curricular success for non-native speakers of English. Writing
supports reading development.
Assessing Reading Achievement in Schools
Using Open Court via the SAT 9
Using SAT 9 scores to compare the
success of one reading program with the success of other reading programs is
problematic. SAT 9 is not a criterion-referenced test but a
norm-referenced test. Criterion-referenced tests, such as weekly spelling
tests, high school API tests, and driver’s license tests, assess if the test
takers have mastered a given body of knowledge. Norm-referenced tests, such as
college entrance tests, sort and rank the test takers. They are
constructed so that 50% of the test takers will score in the bottom 50%,
regardless of their mastery of the subject.
In California, another problem in using the SAT
9 to assess reading instruction is that the SAT 9 norms were
developed with a population where 1.8 percent of the children were limited
English proficient. Over 25% of California’s K-12 children are limited English
proficient. Hence, schools with large numbers of limited English proficient
children will compare unfavorably to SAT 9’s norming group as well as to
other California schools with lower numbers of limited English proficient
children. Moreover, the scores of limited English proficient children, as
presently reported, are meaningless. Immigration occurs at all ages. It takes
five to seven years for non-native speakers of English to achieve in English at
levels equivalent to native speakers (Collier, 1989). It’s impossible to
discern progress—or lack of progress—from data which combines the scores of
children who have been in U.S. schools a few years with children who have been
in U.S. schools many years.
There are other problems with using SAT 9 scores to assess the effectiveness of one reading program at one school against another reading program at another school. SAT 9 scores do not tell us if the teachers at each school are fully credentialed or not, if they are teaching the programs as the programs were designed to be taught or not, if they are providing instruction above and beyond the programs or not, if they are spending significantly more time in reading instruction or not, or if they are receiving significantly more assistance from aides, volunteers, or tutors or not. SAT 9 scores also do not tell us if the children at each school are economically disadvantaged or not, if they are the same children that were at that school the year before or not, or if significant numbers of children were excluded from taking the test by being placed in Special Education or not.
Nevertheless, under AB 1626 (1998), SBIX (1999), and ABX12 (1999), California rewards students, teachers, and school staffs with cash bonuses in schools that meet target scores on the SAT 9 and threatens students, teachers, administrators, and school districts that do not meet target scores with retention in grade, loss of jobs, and loss of autonomy. Hence, despite problems in judging reading education on the basis of SAT 9 reading scores, how successful various programs are in enabling students to do well on the SAT 9 reading assessment may be of interest to California educators caught up in high-stakes testing.
More importantly, as education stakeholders, we need to ask if the cost of neglecting instruction in other curricular areas is justifiable in terms of reading achievement. Will sacrificing instruction in other core curricular areas pay off in greater reading achievement or not? Our research asked three questions:
(1) Does Open Court foster higher reading achievement initially among economically disadvantaged children?
(2) Does Open Court foster higher reading achievement in the long-run among economically disadvantaged children?
(3) Does Open Court foster higher reading achievement generally among economically disadvantaged children?
Method. Working with the 1999 average SAT 9 reading scores of
English-only children and the percent of children on free / reduced-price meals
obtained from the California Department of Education’s web site, we
investigated the effectiveness of Open Court schools against
non-scripted programs in schools serving similar economic groups in the Los
Angeles Unified School District. The fact that LA Unified used several
different reading programs in 1998-99 allows us to compare the SAT 9
reading scores of Open Court schools to the SAT 9 reading scores
of similar schools using several other programs. The non-scripted programs used
across schools in the district in 1998-99 were Invitations to Literacy
(Houghton Mifflin Co.), Literacy Places (Scholastic, Inc.), Signatures
(Harcourt-Brace), and Spotlight on Literacy (McGraw-Hill).
We limited our study to schools on the
traditional calendar. We further limited our study to schools that either (1)
used Open Court not in combination with another program or (2)
used one of the non-scripted programs used across the Los Angeles Unified
School District in 1998-99 not in combination with other programs.
We found 159 elementary schools in LA
Unified that met the research criteria for inclusion in the study. Of these, 21
used Open Court. Nine of the Open Court schools had been using Open
Court over ten years. All the Open Court schools served populations
where 50% or more of the children were on free / reduced-price meals. Hence, we
further limited the schools in the study to schools serving populations where
50% or more of the children were on free / reduced price meals. This left 153
schools in the study. Table 1 shows the number of schools in the study by type
of program and economic disadvantage of the children.
Table 1
Numbers of Schools by
Type of Reading Program and Percent Children Receiving
Free / Reduced-Price Meals
|
|
Children Receiving Free /
Reduced-Price Meals |
||||
|
Reading Program |
50-59% |
60-69% |
70-79% |
80-89% |
90-100% |
|
Non-scripted programs |
12 |
10 |
43 |
24 |
41 |
|
Open Court |
1 |
1 |
4 |
4 |
2 |
|
Long-term Open Court |
|
|
|
|
9 |
Among
9 of the schools with 95 – 100% of the children are on free / reduced-price
meals, some grades had too few English-only children for the scores to be
reported by the state. Each of these cases occurred among the schools that used
non-scripted programs.
To investigate whether Open Court
fosters higher reading achievement initially among economically disadvantaged
children, we compared the average second grade scores of the schools in the
study. In this comparison we sorted the scores into one of three groups: those
below the 25th percentile, those at the 25th to 49th
percentile, and those at the 50th percentile or higher. Four of the
153 grades in this comparison had too few English-only children for the scores
to be reported.
To investigate whether Open Court
fosters higher reading achievement in the long-run among economically
disadvantaged children, we compared the average second and fifth grade scores.
In this comparison, we sorted the schools into one of three groups: those with
differences of +10 or more percentile points between second and fifth grade,
those with less than 10 percentile point differences between second and
fifth grade, and those with differences of -10 or more percentile points
between second and fifth grade. Open Court schools were further sorted
into Open Court schools and long-term Open Court schools.
Eight of the 153 schools in this
comparison had too few English-only children at the second or fifth grade for
their scores to be reported.
To investigate whether Open Court
fosters higher reading achievement generally among economically disadvantaged
children, we compared the average scores of all the grades of long-term Open
Court schools with the average scores of all the grades in schools using
the non-scripted programs that had the same percent of children on free /
reduced-price meals. The long-term Open Court schools in LA Unified
serve populations where 97-100% of the children receive free / reduced-price
meals. Therefore, we compared the scores of the long-term Open Court
schools with the scores of schools using non-scripted programs where 97-100% of
the children received free / reduced-price meals. As shown in Table 2, the
groups were comparably disadvantaged. The average long-term Open Court
school in this group had 98.3 children receiving free / reduced-price meals.
The average school using a non-scripted program had 98.5 children receiving
free / reduced-price meals.
Only one school in the all-grades comparison
(a school using a non-scripted program) had average scores at 50% or higher.
Hence, in this comparison we sorted the scores of each grade into one of two
groups: those with average SAT 9 reading scores in the bottom quartile
on the SAT 9 and those with average scores above the bottom quartile.
Nine of the 120 grades in this comparison had too few English-only children for
their scores to be reported.
Table 2
Numbers of Schools with 97-100% of the Children on Free / Reduced-Price Meals by
Type of Reading Program
|
|
Children Receiving Free / Reduced-Price Meals |
|||
|
Reading Program |
97% |
98% |
99% |
100% |
|
Non-Scripted Programs |
4 |
7 |
5 |
5 |
|
Long-Term Open Court |
4 |
1 |
1 |
3 |
The fifth-grade score of one long-term Open
Court school was omitted in the second to fifth grade and the all-grade
comparisons because the score was an outlier. While the second grade children
in this school averaged in the 28th percentile and the third and
fourth grade children averaged in the 17th percentile, the fifth
grade children averaged in the 62nd percentile, a difference far
greater than in any other school, as can be seen in the appendix.
Findings.
Second grade comparison. Looking at all the schools in the study, there
is no clear difference between the SAT 9 reading scores for second grade
English-only children in schools using Open Court and in schools using
the non-scripted programs. As shown in
Table 3, second grade English-only children in non-scripted programs were more
likely to score in the 25-49% range than their counterparts in Open Court
schools, but these differences are not significant. Among the schools where
97-100% of the children are on free / reduced-price meals, average SAT 9
reading scores for second grade English-only children in the 18 schools using
non-scripted programs were 2 points higher than those in 9 long-term Open
Court schools (33 vs. 31), but again this difference is not statistically
significant (p = .545).
Numbers of Schools with Average Second Grade
English-Only SAT 9 Reading Scores
in the Second, Third, and Fourth Quartile
|
|
Average SAT
9 Reading Scores |
||
|
Reading
Program |
0-24% |
25-49% |
50% and
above |
|
Non-Scripted Programs |
16 (13%) |
94 (73%) |
18 (14%) |
|
Open Court |
4 (19%) |
13 (62%) |
4 (19%) |
Chi2 (df2) =
1.214, p = .545
Second to fifth grade comparison. As shown in Table 4 no Open Court
school had a double-digit positive difference between second and fifth grade.
However, 23% (28 out of 122) schools using non-scripted programs had positive
differences of 10 percentile points or more between second and fifth grade.
Numbers of Schools with Differences of +10 or More Percentile Points in Average
English-Only SAT 9 Reading Scores between Second and Fifth Grade
|
|
Children
Receiving Free / Reduced-Price Meals |
||||
|
Reading Program |
50-59% |
60-69% |
70-79% |
80-89% |
90-100% |
|
Non-Scripted Programs |
4/12 |
3/10 |
9/43 |
5/24 |
7/33 |
|
Open Court |
0/
1 |
0/
1 |
0/
4 |
0/
4 |
0/
2 |
|
Long-Term Open Court |
|
|
|
|
0/
8 |
As shown in Table 5, double-digit negative
differences between second and fifth grade occurred almost twice as often in
long-term Open Court schools as in schools using non-scripted programs.
While 25% (2 out of 8) of the long-term Open Court schools had double-digit
negative differences between second and fifth grade, 13% (16 out of 122) of the
schools using non-scripted programs had double-digit negative differences
between second and fifth grades.
Numbers of
Schools with Differences of -10 or More Percentile Points in
Average
English-Only SAT 9 Reading Scores between Second and Fifth Grade
|
|
Children Receiving Free / Reduced-Price Meals |
||||
|
Reading Program |
50-59% |
60-69% |
70-79% |
80-89% |
90-100% |
|
Non-Scripted Programs |
1/12 |
1/10 |
8/43 |
3/24 |
3/33 |
|
Open Court |
1/
1 |
1/
1 |
1/
4 |
0/
4 |
0/
2 |
|
Long-Term Open Court |
|
|
|
|
2/
8 |
Among schools using the non-scripted
programs, double-digit positive differences between second and fifth grade
occurred almost twice as often as double-digit negative differences.
All-grades comparison. As shown in Table 6, in schools where
97-100% of the children were receiving free / reduced-price meals, grades in
schools using the non-scripted programs scored above the bottom quartile almost
twice as often as grades in long-term Open Court schools. While 72% (54
out of 75) of the grades in the schools using non-scripted programs scored
above the bottom quartile, only 43% (15 out of 35) of the grades in the
long-term Open Court schools scored above the bottom quartile. A 2x2 Chi2
shows that this difference is statistically significant at the .01 level.
Table 6
Numbers of Grades with Average
English-Only SAT 9 Reading Scores
In and Above the Bottom
Quartile
In Schools Where 97-100% of the
Children Receive Free / Reduced-Price Meals
|
|
Average SAT
9 Reading Scores |
|
|
Reading
Programs |
In bottom
quartile |
Above bottom
quartile |
|
Non-Scripted Programs |
21 (28%) |
54 (72%) |
|
Long Term Open Court |
20 (57%) |
15 (44%) |
Chi2 (df1) = 8.669, p<.01
Discussion. In every measure used in this study, schools using non-scripted
programs did as well or better than schools using Open Court on
the SAT 9 reading assessment. Schools using Open Court
were conspicuously absent among the schools that had a 10 percentile or more
positive difference between their average SAT 9 reading scores in second
and fifth grade. Double-digit negative differences occurred twice as often in
long-term Open Court schools as in schools using non-scripted programs.
While the second grade children were not the same children as the fifth grade
children in the study, the differences in the second and fifth grade SAT 9 reading
scores between schools using Open Court and schools using the
non-scripted programs is remarkable, especially in light of the fact we did not
find a significant difference in the average second grade scores between
schools using Open Court and schools using the non-scripted programs. If
Open Court is as effective as the other programs in the study, we would
expect to see scores about the same across grades. Finally, long-term Open
Court schools serving communities where 97-100% of the children receive
free / reduced-price meals were significantly more likely to be in the bottom
quartile of the SAT 9 than schools using non-scripted programs serving
similarly disadvantaged children.
The negative difference between the average
second and fifth grade SAT 9 reading scores in LA Unified’s long-term Open
Court schools parallels the negative differences between the average second
and fifth grade SAT 9 reading scores in Inglewood Unified’s long-term Open
Court schools. The findings of this study are consistent with the above
reanalysis of children’s reading in Open Court vs. the contemporary
reading instruction in the NICHD Houston study in the Alief Independent School
District. Altogether, the outcomes in the Alief, Inglewood, and Los Angeles
school districts suggest that Open Court limits what children are able
to achieve in literacy relative to what they are able to achieve via many other
programs. The outcomes support the professional judgment of the reading /
language arts teacher specialists on California’s 1996 Instructional Resources
Evaluation Panel who recommended Open Court not be placed on
California’s textbook adoption list. Finally, we find no justification in
sacrificing instruction in other core curricular areas to implement Open
Court.
Acknowledgements
We are indebted to Stephen Krashen and Denise Ross for
comments that enhanced this study.
References
Anderson,
R.C., Wilkinson, A.G. and Mason, J.M. (1991). A microanalysis of small-group,
guided reading lesson: Effects of an emphasis on global story meaning Reading
Research Quarterly, 26, 417-441.
Anderluh,
C. (1998). Building with words: City schools hope phonics boosts reading. Sacramento
Bee, Feb 1.
Cantrell,
S.C. (1999). Effective teaching and literacy learning: A look inside primary
classrooms. The Reading Teacher, 52 (4), 370-378.
Coles, G. (2000). Misreading
Reading: The bad science that hurts children. Portsmouth, NH: Heinemann.
Collier,
V.P. (1989). How long? A synthesis of research on academic achievement in a
second langauge. TESOL Quarterly 23 (3): 509-531.
Colvin,
R. (2000). Marion Joseph: The force behind phonics predicts a reading
renaissance. Los Angeles Times, Orange County Edition, June 18, M3.
Downing,
K. M. , Parks, M. and Clayton, J. (Aug. 8, 1999). Reading plans that work. Los
Angeles Times, Orange County Edition, M4.
Eldredge, J.L.,
Reutzel, D.R. and Hollingsworth, P.M. (1996). Comparing the effectiveness of
two oral reading practices: Round-robin reading and the shared book experience.
Journal of Literacy Research, 28 (2), 201-225.
Ellerbee, W. J.
(1999). Focus, focus, focus pays off in results. FORWARD! 13 (1), 1
& 12-15.
Foorman,
B.R., Francis, D.J., Fletcher, J.M., Schatschneider, C. and Mehta, P. (1998).
The role of instruction in learning to read: Preventing reading failure in
at-risk children. Journal of Educational Psychology, 90 (1),
37-55.
Helfand,
D. (2000a). Decades later, frustrated father is phonics guru. Los Angeles
Times, Orange County Edition, Jan. 10, A1 & A16.
Helfand,
D. (2000b). Inglewood writes the book on success. Los Angeles Times,
Orange County Edition, April 30, A1 & A 38-39.
Helfand,
D. (1999). Sacramento gets high marks in school reform. Los Angeles Times,
Orange County Edition, Aug. 22, A1 & A42.
Holland,
B. (1996). No More Wright Group or Rigby?? An open letter sent by the 1996
President of the California Reading Association’s Inland Empire Reading Council
to California Reading Association Council members and concerned educators.
The Oakland Tribune (1997). Study: Phonics is the best way of learning to read, Feb. 19, A5.
Mullis, I., Campbell, J., and Farstrup, A. (1993). NAEP 1992 Reading Report Card for the Nation and the States. Washington, D.C.: National Center for Education Statistics.
Reutzel, D.R. and Cooter, R.B. (1990) Whole language: Comparative effects on first-grade reading achievement. Journal of Educational Research 83, 252-257.
Rothstein, R. (2001). Poverty and Achievement, and Great Misconceptions. The New York Times (Jan 3).
Sacks, C.H. and Mergendoller, J.R. (1997). The relationship between teachers’ theoretical orientation toward reading and student outcomes in kindergarten children with different initial reading abilities. American Educational Research Journal 34 (4), 721-739.
Saunders, A. (1999). Eureka! School reform works. San Francisco Chronicle, June 29, A21.
Taylor, D. (1998). Beginning to Read and the Spin Doctors of Science. Urbana, IL: National Council of Teachers of English.
Taylor, B., Anderson, R.C., Au, K.H., and Raphael, T.E. (2000). Discretion in the translation of reading research to policy. Educational Researcher, 29, (6) 16-26.
Appendix
LAUSD Schools* in Study with Positive 10
Percentile Point Differences Between
Average 2nd and 5th
Grade SAT 9 Reading Scores among English-only Children
by Percent of Economically Disadvantaged
Children
|
50-59% |
60-69% |
70-79% |
80-89% |
90-100% |
|
Calahan +11 |
San Jose +26 |
Short +12 |
Laurel
+15 |
Castelar +10 |
|
Colfax +14 |
Nevada
+16 |
Plainview +13 |
Grand View +23 |
City Terrace +14 |
|
Bonita +10 |
Valley View +13 |
Canterbury +18 |
Gault +19 |
Sheridan
+15 |
|
Brainard +19 |
|
San Pascual +19 |
Saticoy +13 |
Pinewood +13 |
|
|
|
W. Hollywood +23 |
Sterry +21 |
1st St. +10 |
|
|
|
Monlux +17 |
|
Griffin +13 |
|
|
|
El Sereno +12 |
|
Soto +14 |
|
|
|
Shirley +10 |
|
|
|
|
|
Multnomah +18 |
|
|
*All the schools in
this table used one or more of the following non-scripted programs: Invitations
to Literacy (Houghton Mifflin Co.), Literacy Places (Scholastic,
Inc.), Signatures (Harcourt-Brace), and Spotlight on Literacy
(McGraw-Hill).
January 2001