Claims of “Marked Jumps” in
Children’s Literacy Skills in Open Court Schools
Not Confirmed by Children’s SAT 9 Reading Scores
April
10, 2002, the Business Wire carried a news article extolling the “raised test
scores” and “marked jumps in … children’s literacy skills” with Open Court
reading instruction. The article cited McGraw-Hill Education’s report, Results
with Open Court Reading.
Using
the SAT 9 data on the California Department of Education’s website (www.cde.ca.gov), I analyzed the reading
scores of the English-only children in the school and school districts in
California in the report. The report compares (1) scores in selected grades
year by year within a school or a school district and (2) one school district’s
scores against the state’s scores.
First,
let us look at the within-school or within-school-district comparisons in all
publicly reported grades. As shown in the following tables, if one follows the
teachers from year to year (e.g., from second grade in 1999 to second grade in
2000) by reading the tables horizontally, the percent of children scoring at or
above the 50th percentile usually increased, often to a large
extent. However, if one follows the children from grade to grade (e.g.,
from second grade to third grade) by reading the tables diagonally, we see a
different picture. The numbers in parenthesis represent the one-year change
from grade to grade. Following the children we see that:
·
From
second to third grade, in all but one of eight cohorts, the percent declined,
sometimes to a large extent.
·
From
third to fourth grade, the percent remained virtually the same, partially
recovered from the previous year’s loss, or, in one cohort, declined a great
deal.
·
From
fourth to fifth grade, Lemoore and Sacramento City experienced modest to no
gains, Kelso partially recovered from losses in earlier grades, and Curtis
Creek experienced strong gains.
In
sum, if one follows the children rather than the teachers, with the exception
of one grade in one school, the percent of children scoring at or above the 50th
percentile remained virtually the same or declined.
Lemoore
Union Elementary School District * Sacramento
City Unified School District +
|
|
1999 |
2000 |
2001 |
2 year change |
|
|
1999 |
2000 |
2001 |
2 year change |
|
grade 5 |
|
35 ( 0) |
42 (+3) |
+ 4 |
grade 5 |
46 |
46 (+1) |
48(+ 1) |
+ 1 |
|
|
grade 4 |
35 |
39 (+1) |
43 ( 0) |
- 4 |
|
grade 4 |
45 |
47 ( 0) |
51(+ 1) |
- 8 |
|
grade 3 |
|
43 (-4) |
55 (+1) |
|
|
grade 3 |
47 |
50 (-9) |
51(-10) |
|
|
grade 2 |
47 |
56 |
63 |
|
|
grade 2 |
59 |
61 |
61 |
|
* 57% free/reduced-price meals + 61% free/reduced-price meals
17% ELL 29% ELL
Kelso Elementary School ** Curtis Creek Elementary School District ++
|
|
1999 |
2000 |
2001 |
2 year change |
|
|
1999 |
2000 |
2001 |
2 year change |
|
grade 5 |
71 |
54 (- 6) |
46 (+7) |
- 15 |
grade 5 |
57 |
59 (+9) |
60 (+6) |
+ 7 |
|
|
grade 4 |
60 |
39 (-22) |
74 (+5) |
- 9 |
|
grade 4 |
50 |
54 (+1) |
61 (+8) |
+ 1 |
|
grade 3 |
61 |
69 (-14) |
72 (- 3) |
|
|
grade 3 |
53 |
53 (- 7) |
68 (- 8) |
|
|
grade 2 |
83 |
75 |
77 |
|
|
grade 2 |
60 |
76 |
81 |
|
** 85% free/reduced-price meals ++ 35% free/reduced-price meals
39% ELL 0 % ELL
Now let us look at the district-to-state comparison
in the report. The report compares Curtis Creek’s scores against California’s
scores as a whole. According to the report, Curtis Creek adopted Open Court
in March of 1998. Curtis Creek’s 1999-2001 scores compare favorably to the
state’s, but this should be expected: only 35% of Curtis Creek’s children are
on free/reduced-price meals while 47% of California’s children are on
free/reduced-price meals. More economically advantaged children typically do
better on the SAT 9 than less economically advantaged children. If the
report had compared Lemoore Union’s or Sacramento City’s scores against the
state’s scores, the comparison would have been unfavorable, but equally invalid.
Curtis Creek Elementary School
District ++ California
#
|
|
1999 |
2000 |
2001 |
2 year change |
|
1999 |
2000 |
2001 |
2 year change |
|
|
grade 5 |
57 |
59 (+9) |
60 (+6) |
+ 7 |
grade 5 |
54 |
55 (+2) |
57 ( 0) |
+ 4 |
|
|
grade 4 |
50 |
54 (+1) |
61 (+8) |
+ 1 |
|
grade 4 |
53 |
57 (+4) |
59 (+2) |
+ 4 |
|
grade 3 |
53 |
53 (- 7) |
68 (- 8) |
|
|
grade 3 |
53 |
57 (+2) |
60 ( -1) |
|
|
grade 2 |
60 |
76 |
81 |
|
|
grade 2 |
55 |
61 |
63 |
|
++35 free/reduced-price meals # 47% free-reduced-price meals
0% ELL 25% ELL
A
more valid comparison would be to compare Curtis Creek’s scores to those of a
school serving similar children. Sonora Elementary School District, in the same
county as Curtis Creek, has a similar percent of children on free/reduced-price
meals. Sonora uses a variety of non-scripted reading instructional programs
plus Reading Recovery. Children are seen as individuals and teachers are seen
as professionals. When we compare these districts, we see different outcomes on
the SAT 9 reading assessment. In second grade we see a larger percent
scoring at or above the 50th percentile in Curtis Creek. In third,
fourth, and fifth grades we see a larger percent scoring at or above the 50th
percentile in Sonora. In Curtis Creek we see strong declines from second to
third grade. In Sonora, we see increases in every grade. Half the time
the increases were large.
Curtis Creek
Elementary School District ++ Sonora Elementary School District
##
|
|
1999 |
2000 |
2001 |
2 year change |
|
1999 |
2000 |
2001 |
2 year change |
|
|
grade 5 |
57 |
59 (+9) |
60 (+6) |
+ 7 |
grade 5 |
64 |
65(+
3) |
62 (+
2) |
+ 4 |
|
|
grade 4 |
50 |
54 (+1) |
61 (+8) |
+ 1 |
|
grade 4 |
62 |
60(+
2) |
69 (+
9) |
+ 19 |
|
grade 3 |
53 |
53 (- 7) |
68 (- 8) |
|
|
grade 3 |
58 |
60(+10) |
71 (+10) |
|
|
grade 2 |
60 |
76 |
81 |
|
|
grade 2 |
50 |
61 |
71 |
|
++35% free/reduced-price meals ## 38% free-reduced-price meals
0% ELL 2% ELL
The
SAT 9 scores in the schools featured in McGraw-Hills Education’s report
show that the teachers in the school and school districts in the report are
doing what is being asked of them, but they do not show “marked jumps in …
children’s literacy skills”. In fact, the scores in these schools should raise
questions about Open Court’s efficacy.
But
perhaps the most important question of all is: How did the article get on the
Business Wire? Could it be because one of McGraw-Hill’s divisions is a media
division and only members of the media can put items on the news wire? Does this privilege the marketing of one
program over another? Are policymakers and the public hearing the full story?
Margaret
Moustafa, Ph.D.
California
State University Los Angeles April
24, 2002