Robert E. Land and Margaret Moustafa

California State University Los Angeles

This study asked if children in schools with scripted reading instruction programs score as well on high-stakes tests of reading achievement as children in comparable schools using unscripted programs.

Method. The study compared the percent of English-only and English proficient children, grades 2 through 5, scoring at or above the 50th percentile on the SAT 9 reading assessment in schools using scripted programs with the percent of children scoring at or above the 50^th percentile in comparable schools using unscripted programs without decodable texts in a large urban school district in California from 1999 to 2001. California used the SAT 9 for accountability purposes during the years of the study. The scripted programs in the district we studied were Open Court and Success for All. The unscripted programs without decodable texts were Invitations to Literacy, Literacy Places, and Signatures. Schools that used a program in combination with a program not in the study, or used both scripted and unscripted programs, were excluded from the study. Schools and/or grades where programs were not used in a particular year (e.g., the 2001 second grade scores of schools that switched from an unscripted to a scripted program fall 2000) were excluded. Schools with fewer than 50% of their students on free/reduced-price meals in 1999 were excluded. Overall, the analyses included data from 98,345 student scores at 183 schools and 1899 grades. Schools in the study ranged in the level of teachers credentialed from 46% to 87% in 1999. Schools were divided into two groups: those with lower levels (71% or fewer) teachers credentialed and those with higher levels (72% or more) teachers credentialed.

Findings. The study found that the percent of children scoring at or above the 50^th percentile on the SAT 9 test of reading achievement was significantly lower in schools with scripted programs than in schools with unscripted programs (p.<.0001). As shown in the table, the study found the percent of children scoring at or above the 50^th percentile rose each year of the study regardless of the type of reading instruction program. Within this rising pattern, the percent of children scoring at or above the 50^th percentile was

· lower in schools with lower levels of teachers credentialed than in schools with higher levels of teachers credentialed

· lower in schools with scripted programs than in schools with unscripted programs.

· lowest in schools with low levels of teachers credentialed and scripted programs.

Percent of Children Scoring At or Above the 50^th Percentile on the SAT 9 Reading Assessment by Level of Teachers Credentialed, Program Type, and Year

	lower levels of teachers credentialed		higher levels of teachers credentialed
	scripted programs	unscripted programs	scripted programs	unscripted programs
2001	31	38	44	44
2000	27	35	37	40
1999	22	30	31	35

Note: Overall scripted programs mean=31.2, unscripted programs mean=36.8. F (1/1870df) = 31.8, p. < .0001

Discussion. The finding that the percent of children scoring at or above the 50^th percentile increased during the period we studied is consistent with Linn’s summary of the last fifty years of high-stakes/accountability testing. Linn (2000) explains rising test scores over successive years as a consequence of using the same questions with the same norms for high stakes purposes year after year. (Linn also points out that when a new test is introduced for accountability purposes, there is a sharp drop in scores in the first year a new test is introduced, followed by gains in subsequent years.)

The finding that the percent of children scoring at or above the 50^th percentile was lower in schools with lower levels of teachers credentialed than in schools with higher levels of teachers credentialed is consistent with the First Grade studies that found that the teacher is the most important factor in the classroom.

The finding that the percent of children scoring at or above the 50^th percentile is significantly lower in schools with scripted programs than in schools with unscripted programs is consistent with the fact that children vary in their instructional needs and scripted programs don’t allow teachers to use professional judgment to adjust instruction to the instructional needs of the children in their classrooms. Scripted programs may also discourage inexperienced teachers from learning how to address students’ various instructional needs, discourage experienced teachers from mentoring inexperienced teachers, and discourage teachers from remaining in the profession.

When we presented our finding that the percent of children scoring at or above the 50^th percentile is significantly lower in schools with scripted programs than in schools with unscripted programs to the 2002 International Reading Association Convention, an educator from a school district in California that has used only the two scripted programs in this study since 1997 responded that her district’s scores on the SAT 9 reading assessment have risen year after year since the district adopted the programs. The findings of this study suggest that the district’s scores would have risen regardless of the program used and may have risen more if they had used unscripted programs.

The finding that the percent of children scoring at or above the 50^th percentile in reading achievement was lowest in schools with lower levels of teachers credentialed and scripted reading programs suggests that schools confronted with low levels of credentialed teachers will do better in high stakes testing if they use unscripted programs.

Reference:

Linn, R. (2000). Assessments and accountability. Educational Researcher, 4-16.

_____________

For inquiries contact Bob Land, Ph.D., Associate Professor, Charter College of Education, (323) 343-4443 or rland@calstatela.edu

May 9, 2002