Most
discussions of "school reform" focus on the need to close or
restructure failing schools or else
provide students with choice as a way
out of them. Embedded within these discussions are theories or suggestions
about why these schools are failing. Often left unexamined is the actual claim
that these schools are failing.
Since
I have had the opportunity to study many schools in Louisiana throughout my
career as a member of the faculty in the College of Education at Louisiana
State University, and through the lens of my service as a School Board Member
for sixteen years, I will focus on schools in Louisiana. When a school in
Louisiana is assigned the letter grade F,
it is almost universally accepted in the media and in school reform policy
debates that children in this school are receiving a sub-standard education,
almost by definition since the school itself is seen as "failing."
Yet the second part of the title of this paper, which comes from a chapter in
the Late Gerald W. Bracey's 2003 book "On the Death of Childhood and the
Destruction of Public Schools," raises an interesting question.
Should
we be confident that the letter grade F
assigned by the State of Louisiana actually indicates that the quality of
teaching in the school is the reason for the failure? If it is not the quality
of teaching, then what is it? After all, it is not the building that is
responsible for the education of the students within its walls.
Drawing
upon research on teaching and learning, and data on student achievement we will
examine how failing schools "fail"
and ask if they indeed are failing. We will also draw from some of the good work
being done in Louisiana to improve the quality of teaching and the quality and
effectiveness of our schools.
The teacher is
the most important factor for school success. "On the job" professional
development and improving working conditions of teachers are the keys to school
improvement.
We
find further support for the role of teaching in
failing schools in other places as well. For example, we heard
strong comments from Dr. Tabitha Grossman, Senior Policy Analyst at the National
Governors Association, when she came and spoke before the Blue Ribbon
Commission on Educational Excellence. In her
Building
a High Quality Workforce: a governor's guide to human capital development
she begins on page one with this succinct point:
"Teacher effectiveness is the primary
influence on student achievement, followed by principal effectiveness."
She
gives as her reference to this, a Wallace Foundation publication, certainly a
reputable source in many education and policy circles. On page three of the guide, Dr.
Grossman again emphasizes the important role of the teacher in the academic
achievement of students: “Research shows that teacher quality is the primary
influence on student achievement." For her reference for this statement she
uses a major report on teacher quality published by the Organization for
Economic Cooperation and Development (OECD). Again, this gives us additional confidence in her statement.
We can
see in these quotes a clear relationship being outlined between "teacher
effectiveness" as the "primary influence" on student
achievement, and the teacher as the "most important single factor"
for student success. Since the letter grade of a school is, for the most part, determined
by "student success" as measured by standardized test scores, the
reformers seem to be on solid ground in assessing the blame for low scores to inferior
teaching. Before we conclude that so many of Louisiana's failing schools are filled with poorly performing teachers,
however, we might want to examine other possible factors that might explain the
low scores in these schools.
If
teaching is key, we should find evidence that quality teaching can reverse the
impact of other factors that might be hindering student achievement in the failing schools. After all, if the
earlier quotes represent the best research on student achievement, that
research certainly suggests that teacher quality is the dominant factor. To
test this, let's see if other factors appear to play a large role in failing schools.
One
claim that we often hear from those challenging the reformers is that poverty
is the reason for failing schools. If poverty is the cause of failing schools, then we should find that strong patterns exist
linking poverty to school performance, even in charter schools and in voucher
programs where reformers claim that competition for school success should help
to overcome any challenges in educating students from low-income homes.
We
will now explore some data to see whether poverty seems to be related to school
failure within Louisiana. To accomplish this we used data on the letter grades
and school poverty that used to be available from the Louisiana Department of
Education's website. An earlier version of this paper had links to all of the
data used in the following examples, but unfortunately, State Superintendent of
Education John White has had nearly all research data sets removed or modified
to ensure that studies such as this cannot now be done. Luckily, this study was
completed before the removal of the historical public data. We removed selective enrollment and alternative schools from the list of
schools in the state containing letter grades and information on student
poverty, since both types of schools obviously start off with students selected
by achievement levels. Dr. George Noell, the architect of the state's
value-added assessment system, has pointed out that prior test scores, where
available, are clearly the most powerful predictors of future achievement, so a
school that selects for high-achieving students or is filled with students
whose academic achievement has been severely hampered before their arrival in
the school would not be giving us legitimate information about the quality of
teaching within the school.
If
teacher quality is the most important factor in student achievement, then we
should find patterns suggesting that high-performing schools such as those with
letter grades of A are overcoming
the impacts of poverty, while those schools labeled F are not as effective in overcoming the challenges of poverty. We
will use the category of students qualifying for free lunch, since that
contains students in families with incomes up to 130% of the poverty line, or
closest to the federal guidelines for poverty.
The
average percentage of students qualifying for free meals in the A schools, is approximately 34%, which
is almost the same as the average percentage of students in all schools across
the United States who qualify for free meals. When we look at schools identified as failing schools, we find much higher levels of poverty. Approximately 89% of the students in
schools rated F qualify for free
meals. Examining schools labeled B
or C or D, we find that they also fit this pattern. We find their respective
percentages of low-income students to be 45%, 60%, and 80%. To make it easier
to visualize this pattern for the state of Louisiana, we have charted it below.
From
this chart it appears that there is a powerful pattern in the relationship between
the concentrations of poverty in schools and the assigned letter grades for schools.
However, we should note that for four years Louisiana put out a report that
highlighted High-Poverty High Performing
Schools, which suggested that there are, perhaps, many schools that defy
this pattern. After carefully examining the lists, which reported higher
numbers of schools each succeeding year, with 56 schools in the 2011 release, we
noted that many of the schools actually had a lower percentage of students
qualifying for free meals than the state average. In addition, most of the
schools were magnet schools or schools where Gifted/Talented programs were
masking lower test scores for other groups of students in the schools. Finally, there were schools like Lake
Forest Elementary, in New Orleans, that had extensive application and testing
procedures that eliminated low-scoring students from the schools. We also noted that there were no
schools that had been on the list every year. Not one school out of over 1300
schools in the state that had overcome the challenges of poverty every year.
The
pattern shown above suggests, in fact, that poverty is playing a very important,
and perhaps dominant role in student achievement. Perhaps Louisiana is the only
state that has these patterns? However, we note similar patterns on the
National Assessment of Educational Progress (NAEP) subject area assessments.
For example, the chart below shows 8th grade reading assessments for public and
for private schools (including religious-affiliated schools). As can be seen in
the chart, family income seems to matter in achievement scores for all schools.
The gaps between those students who DO NOT qualify for free or reduced meal
prices and those who DO qualify for reduced meal prices are 14 points in the
public schools and 18 points in the private schools. The gaps between those who
DO NOT qualify for either free or reduced meal prices and those who qualify for
FREE meals are 25 points for public schools and 26 points for private schools.
On these
NAEP Reading Scores, a 10 to 12 point gain in the scale scores is roughly equal
to a year's gain in achievement. (The numbers you see are called "scale
scores" and are used to enable researchers to more easily compare
achievement across schools and groups and to see differences.) To put this in
the simplest terms, students scoring 10 to 12 points higher than other students
have achievement levels we would expect from those having an extra year of
schooling. This data suggests that students living in families at or near the
poverty level are about two years behind their middle class peers. Clearly
then, poverty seems to greatly affect student achievement in public and in
private schools. We note that although the private schools seem to have an
advantage on the scores, they enroll very few special education students, and they
get to select their students.
We
also explored data in districts across the country that have had charter
schools as well as voucher programs and other common school reforms in place for
a number of years. We again looked at NAEP scores in order to see what kind of
impact these reforms have had on reversing the apparent relationship between poverty
and student achievement.
NAEP
scores can be useful checks against a natural tendency of states, districts,
and schools to focus on teaching to the
test, because NAEP assessments are much more difficult to game or teach to than state level tests. An example of this can be found in
states where 90 percent or more of students receive passing scores in their
state at the basic level, when only 20 or 30 percent of their students are
passing NAEP at the Basic Level. We note that the Proficient Level of NAEP is
not at all equal to what we might assume is being on grade level, but is instead significantly higher.
Cleveland,
Ohio school reformers achieved success at the U.S. Supreme Court in 2002 with a
ruling that vouchers could be used in religious schools. Currently, over 5000 students
in Cleveland attend schools that accept vouchers and most of these students are
attending church-related schools. (Vouchers
are named for the payment vouchers used by families to fund their children's schooling
at private, parochial or other schools outside of existing public school choice
opportunities such as magnet schools. Vouchers should not be confused with
scholarships, which are used to reward merit, or scholarship.) In 2003, right after
vouchers for religious schools were approved to begin, the average 8th grade
NAEP reading assessment scale score in Cleveland was 240 for low-income students.
Eight years later the average score for low-income students remained at 240. We
should note that Cleveland does not have enough students who do not qualify for
either free or reduced meal prices to actually have a published NAEP score for
those students.
In New
York City, where school reformer Chancellor Joel Klein shook up the school system
and radically changed the structure of the nation's largest school system, with
charter schools taking over many existing schools, and also in Chicago, where
current U.S Secretary of Education Arne Duncan used many of the same reforms, score
trends on 8th grade reading were also flat. Average scores for low-income
students from each of these two cities in 2011 straddled the national average
for low-income students. Students in New York City, after years of reform
scored 151, Chicago students scored an average of 149, and the national average
for low-income students is 150. Clearly, the evidence suggests that poverty
matters, and it matters in a powerful and predictable way. In spite of early claims
that teachers in high-profile schools in these cities had reversed the impact
of poverty, closer examination nearly always reveals that the impacts of
poverty certainly remain.
To
show how concentrations of poverty play out for individual schools, researchers
plotted the scores of students at elementary, middle, and high schools in
Ascension Parish, Louisiana (In Louisiana, "parishes" are the
equivalent of "counties" in other states.). Ascension Parish is
useful because the school level data is not affected as much as it is in other
districts by charters and magnet schools that distort the data. You can easily
imagine how a school that selected mostly high-achieving students who also
qualified for free meals would distort patterns of achievement for that school,
as well as other schools around them. (Obviously, there are students from
low-income families who are also extremely high scoring! It is for this reason,
among many others, that low expectations for students are never justified!) For
each level, elementary, middle and high, the straight line on each chart
represents the statistical fit of the relationship between the percent of
students qualifying for free meals and the schools' performance scores. (It is
this School Performance Scores, or SPS, that determines the actual letter grade
of a school.)
It is almost frightening that the
schools were fitting so closely on the line that represented the correlation or relationship between
poverty and school performance scores. There are actually two schools where
both the school performance scores and the corresponding percentages of
low-income students were so nearly identical that they appear over one another
in the chart.
The
pattern for middle schools was even stronger than that for elementary schools
in Ascension, as illustrated on this next chart.
The
powerful relationship between poverty and performance scores in these Middle
Schools is almost beyond description. In statistical terms, the correlation
between the percentage of students eligible for free meals and the school
performance scores for Middle Schools in Ascension was negative .987 which,
simply stated, means that as poverty increased, scores went down, in an almost perfectly
linear relationship.
For the Ascension Parish High Schools the relationship was as perfect as most of us have
ever seen. The incredibly powerful numerical correlation of -.998 once again
suggested that the percentage of free lunch eligible students in a school could
easily be used to fairly accurately predict the school performance scores of a
school.
These
charts can be incredibly powerful in helping us visualize the degree to which
poverty matters. In school districts where there were schools that had a group
of "gifted" or "talented" students, or where there were
minimum requirements to get into the school, such as the Baton Rouge Foreign
Language Academic Immersion Magnet (Baton Rouge FLAIM), we almost always found
these schools to be above the line, meaning they achieved at a higher level
than their free lunch percentages would have predicted, for obvious reasons.
Conversely, schools that dealt with high concentrations of students who had not
been achieving well on standardized tests in the high-stakes world of
"reformy" Louisiana, and who often directed their frustration in
socially unacceptable ways, were almost always below the line. Schools with
high concentrations of students qualifying for special services due to mental,
physical, or emotional handicapping conditions almost always lie below the
line.
In
examining District Performance Scores (DPS) across Louisiana, we notice the
same patterns. The DPS scores are, like SPS, mostly influenced by student
achievement on standardized tests. Zachary, with the lowest percentage of
students qualifying for free meals in the entire state, had the highest DPS. On
the other end of the spectrum, the Recovery School District and St. Helena
Parish have the two lowest DPS scores, with St. Helena having the highest
percentages of students qualifying for free meals.
The strong
nature of the relationships we found yet again convinced us that we needed to seriously
question the statements we quoted earlier in this paper about the powerful
effects of teachers on student academic achievement. When we went to the original sources of the texts used by
Tabitha Grossman, for example, we discovered that her statements were not
consistent with the statements in the documents she referenced.
Dr.
Grossman wrote: "
Teacher effectiveness is the primary influence on student achievement,
followed by principal effectiveness." The Wallace Foundation research report she referenced actually said: "Teachers have the most immediate in-school
effect on student success." We trust you can note the not so
subtle-difference. "In-school" and "immediate" do not necessarily
equate to "primary influence."
What about her other quote: "Research shows
that teacher quality is the primary influence on student achievement." When we went to the OECD report she referenced, this is what we found:
Three broad conclusions emerge from
research on student learning. The first and most solidly based finding is
that the largest source of variation in student learning is attributable to
differences in what students bring to school – their abilities and attitudes,
and family and community background. Such factors are difficult for
policy makers to influence, at least in the short-run. (Emphasis added)
The second broad
conclusion is that of those variables that are potentially open to policy
influence, factors to do with teachers and teaching are the most important
influences on student learning. In particular, the broad consensus is that
“teacher quality” is the single most important school variable
influencing student achievement. (Emphasis added.)
The
OECD report and the earlier report from the Wallace Foundation suggest that
teacher effectiveness is important, but they are clearly not reporting that
among all factors impacting student achievement, that teacher impacts are the
most important. In fact, the OECD
report is quite explicit on this.
If
researchers can use rates of poverty, independent of any knowledge of actual
teacher and teaching quality, to predict school performance scores, what does
this say about the claims of reformers that it is the teaching in the schools
that is deficient? What it
suggests is the need to ask the question in the second part of the title of
this paper. What if failing
schools… aren't?
Policy
makers in Louisiana, and in other states, have suggested that value-added
measures (VAM) be used to explore the actual impact of teachers on student
achievement. Supporters often use the following argument to suggest that this
complex mathematical procedure is necessary because it would be ridiculous to
use the actual test scores of students. "Think about it" they say. "Would
anyone want to teach the students who have the most challenges to overcome, and
who score the lowest on nearly all standardized tests?" Teachers might
only want to teach advanced students, or gifted students. "Obviously" they say, this
would not be "fair." How is it "fair" to not account for
these challenges when measuring "school performance?" How is it "fair" to condemn
schools as failing when the accountability system is so flawed that it does not
even begin to measure the quality of teaching within the schools?
I
believe the data and arguments presented in this paper make clear that many schools
are being unfairly stigmatized with the label of failing. More to the point, it suggests, very strongly, that the
letter grades themselves seem to have no particular value in predicting the
quality of teaching within schools! I recently tweeted that "You can know
nothing about the quality of teaching in a school by looking at the school's
letter grade." To put it bluntly, the evidence suggests that school
performance scores are clearly and powerfully related to the degree of poverty
in the schools. I believe that there are likely to be some schools that have
excellent teaching faculties, and some schools that do not have such a strong
teaching faculty. I also believe the data suggest that neither the School
Performance Scores nor the "transparent" Letter Grades are useful in
determining which schools have which faculties.
Wonderful
schools which are nearly filled to overflowing with students struggling from
the negative impacts of poverty do not deserve to be automatically labeled as failing schools by anyone who has not
carefully, and honestly, examined the teaching going on within their walls.
Parents,
teachers, students and even school reformers perhaps, can fairly make
determinations of whether a particular classroom is conducive to learning. It
may take multiple observations and interactions with the teacher and the
students, but fair and accurate evaluations of the quality of teaching can, and
should be made. School reformers often seem to know numbers, but don't always
appear to know what those numbers actually represent, or if they represent
anything at all. I argue here and now, stridently and with the support of
incredibly powerful data, that to label a school as failing based on the scores of students in the school, without
taking into account factors that Dr. George Noell, the Wallace Foundation, and
the OECD and others have clearly suggested impact student achievement
independent of the classroom teachers, is neither ethically nor morally
defensible!
Our
current accountability system for schools does nothing to account for the
effects of poverty on students in our schools, and the challenges it represents
to those teaching and learning in these schools. Louisiana's accountability system
does not, in any way, account for the incredibly varied abilities students
bring with them when they first arrive at our schools. The current system fails
to recognize evidence such as Kindergarten inventories that clearly point out
the effects poverty has had on children before entering school. The work of Dr. Noell and others who
point out that these scores are powerful predictors of future performance, and
would provide evidence of the extremely difficult challenges faced by
high-poverty schools, is totally ignored.
Recognizing
poverty, and the incredibly powerful effects it has on student achievement in
our schools, especially prior to students even entering Kindergarten, would
provide increased support for initiatives that have incredibly powerful and
proven effects on student performance. Community programs that focus on
pediatric care or parenting awareness programs are not as strong as they need
to be in a state where children rank last in virtually all early-childhood categories
on comparisons across all states. The number of books in the home, and the
hours of television watched might become topics of conversation in a community
seeking to increase the education quality of the entire community.
The
current accountability system labels schools as failing in spite of no evidence that it is the teaching that is
failing. The labeling is done on the basis of test scores alone, which have
more to do with the incredible poverty faced by the students than with any
evidence of poor teaching. When so many high-poverty schools across the state
receive the Letter Grade F, it has
the effect of eroding public confidence and desperately needed community
support for our neediest public schools, yet the grades are assigned by
computer, with no one going to the schools to actually see what is happening in
the classrooms! This does a great disservice
to teachers, to students, and to parents who more than anything else, simply
want to have their children in schools where teachers meet their children where
they are, and then teach their children well.
The
real choices to be made, then, are perhaps not the "choices" promoted
by reformers who often see profits in the condemning of public schools.
Instead, a more important and meaningful choice is whether we choose to do
something about the failure of our accountability system to fairly and
accurately provide parents and communities with useful information about the
quality of our public schools.
The
important question, for now, is whether we will have the moral and political
courage to stop falsely labeling schools as failing,
with all of the negative consequences such a label carries, absent real and
convincing evidence concerning the quality of teaching within the school. The
thousands of dedicated teachers and administrators in our schools deserve better,
and the hundreds of thousands of students certainly deserve nothing less!
I used
the term "failing schools"
in this paper over a dozen times, and chances are you didn't stumble over the
term, since we have heard it so many times. I believe that we should banish the
use of this term in the media, and in our policy debates, until the reformers
can assure the public that they ONLY mean those schools that are actually
failing to provide quality instruction for student, and not simply those who
face incredible challenges, and in spite of those overwhelming challenges
provide excellent opportunities to learn for all children.
This
paper is the first in a series which will begin to offer an alternative vision
of the very real choices Louisiana can make as it moves beyond its bicentennial
as a state. Will we make the hard choices, that require more than the labeling
and blaming of schools for the challenges we face? I believe we must. I thank you for reading this, and I
welcome your comments and your suggestions for ways we can further inform the
future of education.
Copyright©2013 by Noel Hammatt. All rights reserved. Permission is hereby granted
for the publication of all or portions of this paper by those seeking to share
the truth about public schools in Louisiana, and elsewhere, as long as this
complete copyright notice is included.
