Sunday, January 25, 2015

Adjusting Teacher Observation Scores

Recent research has shown that it may be more difficult to observe the greatest teaching outlined in observation rubrics in schools and classrooms with lower levels of achievement and the symptoms that often accompany low achievement like high rates of poverty, more behavioral issues and more transient populations. That makes sense. However, I completely disagree with the policy conclusion of the need to adjust observation scores.

Yes, it is likely that there is some systemic bias within teacher observation practices. The possibility also exists that teachers in low performing schools have a more difficult job. This makes sense although there are plenty of examples where this doesn't play out in practice, largely due to the variance of individual observers or even the cultural expectations of an entire district. Score adjustments are not the solution.

Introducing a score adjustment explicitly lowers the expectations for low-income, minority children. The adjustment described is not actually based on prior testing history, it is based on demographic information which correlates with achievement but does not determine achievement. This is the opposite of what we aspire to as an education system, which is to achieve a system where who your parents are, which zip code you were born in and the the color of your skin do not determine how well you perform and how you are treated. What is being communicated is that black and brown children can't have classrooms like white children.  If we not only believe that but systematize that belief into how we observe classrooms there is no reason to believe teachers and students will do anything other than meet those lower expectations, returning us to where we are already, students meeting and exceeding expectations only to learn they are woefully unprepared to compete.

There are biases in all measures.  Knowing what those biases are, or are likely to be, is an important part of understanding data. If we aspire to measure teachers perfectly then measurement will continue to improve but if we hold an expectation requiring perfect, unbiased measurement then we will be paralyzed into measuring nothing and making no decisions.  The proper comparison for any measure is not perfection, where any measure will come up short, but the measure that came before.  In that respect, value-added metrics are head and shoulders above the degrees and experience of the past. Rigorous observation is in a different universe than the non-differentiating exercise in paperwork of a decade ago. This is called progress and it is progress not just for students but for teachers who finally have access information about their performance that doesn't rely entirely on something they can't change (their age), something that is both costly and does not have a track record of helping them improve (degrees) and pure politics (whatever we called human capital decision making when there was very little information to use to make decisions). 

If we do believe that it is harder to teach effectively in schools where it has traditionally been considered more difficult to teach then a better policy conclusion is to pay the teachers in those schools more than the teachers in other schools. Those schools need better teachers to break even. The teachers in those schools are likely performing better than they appear. And it starts to equalize the incentives in terms of recruitment. 

If we do believe there is bias in the observers and it has nothing to do with how difficult it is to teach effectively then we need to do a better job of training and supporting observers. Rubrics are flexible and good judgment is necessary. There are many ways to arrive at an effective lesson where students learn what they're supposed to learn. In my experience many observers focus so heavily on the teacher action and so heavily on the easily observable that they sometimes miss the outcome, the learning, that should drive scoring on an effective rubric. Over time this will likely improve as rising leaders, who have more experience with observation and are more likely to have taught while engaging with teacher evaluation, take over leadership roles with a deeper understanding of instruction and how to interpret rubrics.

And if we believe that observation needs to provide a more systemic leveling of the playing field that protects teachers from all the quirks and biases we can think of, well that has been invented already and it is called value-added. Value-added is the answer to fairness. Teachers with students all over the achievement map do well on value-added metrics. By using students' full testing history and multiple years of data the metrics are stable. By taking into account error they address different class sizes and contexts. By applying the same process to all assessments and all teachers it takes out the variance of the individual observer. If we want the closest thing to perfection that we have, it is value-added and we already have it. If we want observation to serve in a similar capacity as value-added then we need to layer on a similar amount of protective complexity and invest even more heavily in norming and training for observers. That means being okay with a major increase in resources, both time and money, and also being okay with a level of complexity that is difficult to communicate and undermines buy-in.

Overall this feels like missing the forest for the trees. Yes, almost undoubtedly observations are covered in bias, just as all measures are. But what we currently have is a major step forward and likely even stronger qualitative feedback systems than what most of us have ever experienced in non-teaching jobs. The idea isn't to provide perfect fairness to teachers, although we strive for that, or even to help teachers improve, although that is central to the whole strategy.  The idea is to help all students learn. Lowering expectations for the students who need us the most is not a way to do that.

Saturday, January 10, 2015

Free Agency

Last time I wrote on this blog I was leaving DC for Tennessee for what I thought would be a summer fling of a job. That was nearly four years ago. The summer fling of a job turned into three and a half years of adventure and learning. Both DC and Tennessee produced dominant results on NAEP. I managed my own team for the first time, getting so deep into evaluation that I'm not sure I'll ever climb out. It was really good and I feel like we made tremendous progress that will continue for years to come. And now I'm a free agent again, unsure of which direction to go next. Maybe because I've done this before I feel very peaceful about it, after all last time I was a free agent I ended up here in Nashville. Maybe because I have a supportive wife this I feel less urgency to get back to work but more motivated to be productive in one way or another. Maybe because I have a some more experience, some more knowledge about how to do things, I am feeling patient about finding the right fit for me to make a difference. In the meantime, this seems as good a time as any for reflection and thinking, for documenting and organizing my thoughts and ideas. This seems like a good time to write down what I will forget later, to remember what I learned so that it will help me in the future. This seems like a good time to return to this blog, which will be my free agency diary of sorts, to see how I evolve over time, to see what I think when I take time between gigs, and hopefully to stumble upon something useful to those pushing forward to forever improve education in America.

Sunday, June 12, 2011


My apologies for not posting for quite some time.  Life got in the way.  A new job, a new city, a new adventure got in the way.  And so I am going to stop writing for now, for at least ten weeks.  Thank you to those who read what I wrote.  I hope it helped you become a better thinker as it did for me.  Don’t hesitate to email if you wish.  I’m always ready to talk too much about education policy, my dislike of Valerie Strauss  and pretty much anything else.  Farewell until I write again.

What We Are Really Talking About

Someone brought Diane Ravitch’s recent New York Times opinion to my attention.  The gist of it is that the idea of miracle schools are overblown and that the primary way to solve the educational woes of low performing schools and districts is to target the families.  I agree that the idea of a miracle school that suddenly turns everything around for its students is probably overblown.  Although there are some reasons that these stories become such a large part of the conversation and Diane Ravitch is one of those reasons.  I also agree that families are incredibly important factor in student outcomes.  In a sense, that reality is the reason for the whole education reform movement.

When I went to work for DC Public Schools it wasn’t because I didn’t think student outcomes were defined by student background to a large degree.  It was because I did think that.  And I thought it wasn’t right.  To me that is not, and never will be, a reason to not try whatever it takes to give everyone a chance at a successful life.  Our society is based on the idea that anyone can do anything they put their minds to, that each person has the freedom to pursue happiness, to enjoy equal protections under the law.  Is this rational? Is it really true?  Probably not and definitely not.  But we attempt to create a reality that closely aligns with our values.  Maybe the vision we Americans have of equality, meritocracy, open dialogue and social mobility is a little utopian but pursuing that utopia has led to tremendous historical success.  Just because something isn’t possible or doesn’t seem possible doesn’t mean it shouldn’t be attempted.  The mere act of attempting can lead to tremendous change and improvement.

Let there be no mistake that the goal of education reform is revolutionary.  I don’t say education reform to refer to those who are part of the education reform movement.  I say education reform to refer to all those people, mostly teachers, who work every day to give our students the best education and best chance in life possible.  If we accomplish the goal of a strong education for all students regardless of background, zip code, skin color, parents’ education, family income and every other indicator that correlates too strongly with education outcomes, then the world starts to look like a very different place.  Racial inequality starts to disappear.  Neighborhoods that have been left to struggle without answers start to build a foundation of hope based on the human capital of its newly graduate citizens.  The cycle of poverty as we know it begins to break down.  That is what we are really talking about and I can’t think of many endeavors more noble.

If changing the world to make it a better place and pursuing a reality that realizes our more utopian values is too abstract for you that is okay.  You see, the arguments put forth by Diane Ravitch and many others claiming to be protecting public education and teachers are suicidal.  I understand the frustrations among teachers when they are being held accountable, and often told they are failing, for things they cannot control.  Governments, districts, administrations undoubtedly need to do a better job of providing the necessary supports for teachers to be successful and for students to be ready to learn.  That includes the basics like making sure school breakfast and lunch is accessible and edible.  All of us need to do a better job.

However, staking out positions that imply, or outright say, that teachers are largely unimportant in the education outcomes of students is a terrible position to take if you are trying to protect teachers, public education and the system as we currently understand it. 

When Diane Ravitch argues families families families she is basically saying teachers are powerless to overcome the backgrounds of their students.  I realize this is likely a case of public negotiation pushing someone to a position that is too extreme and lacking in nuance.  Nonetheless, the argument is there and it is an argument that actually does threaten public education.  Private companies selling helpful products to under performing districts like Wireless Generation does is not going to take down public education.  Charter schools slicing off a chunk of traditional public school students is not going to take down public education.  School vouchers are definitely not going to take down public education.  Arguing that public education is not capable of overcoming demographics will.  Because ultimately if our schools make no difference they become an unconscionably enormous bad investment.  Because if teachers don’t make a difference then why do we have them at all?  Because if better schools that actually make better opportunities for students that wouldn’t otherwise have those opportunities can’t exist then why try?  In that case maybe slashing school budgets to fund tax cuts for the rich makes a little more sense, or to fund more public transportation, or to fund more defense spending, or to fund more social services, or to fund more police.  Why not lay off more teachers?  Why not just abolish public schools and let each locality, each household figure it out for themselves?  Why is Diane Ravitch making her enemies’ argument for them? 

The end point of pointing the finger at poverty or demographics or social services or crime or health or all of the other hundred things that are stacked against our most at-risk children is giving up on education as a solution.  The end point is reinvesting all that education money elsewhere, away from schools, away from teachers, away from the current, broken, failed system.  But I think Diane Ravitch is wrong.  I think teachers, schools, districts can make a difference.  I think we can do better.  I think we can create a world where a child is not doomed just by the happenstance of being born to the wrong family in the wrong part of town.  


I have never been a teacher.  I have never faced the day to day of standing in front of students and showing them the world.  I can’t understand what that is really like, what each and every challenge feels like, how success can push you forward or failure can break your heart.  I can’t understand what it is like to be a teacher.  All I know is what I have been told, what I have seen and what I have experienced as a student.  For me good teachers are heroes, capable of things I am afraid to attempt.  Teachers are people of course, but growing up they seem like more than normal people and that has stuck with me to this day.

Teaching is a profession that is at once deified and vilified.  Teachers are underpaid, overworked guardians of future.  Teachers are lazy lifers who are in it for friendly hours and summers off.  Teachers are noble, making sacrifices most would never make, saving people who nobody else can save, bringing light where there is darkness.  Teachers fail to teach everyone to read, fail to work hard, fail to create people ready to produce in society.  Today, I’m not so interested in the false dichotomy that plays in the media.

I’m interested in the profession and what it should be.  First and foremost it is important to remember that teachers are people, like you and me, and have the same wants and desires as we do.  Wanting to be paid more is not a bad thing.  Wanting time off to go to Hawaii or Spain or wherever is not a bad thing.  Being ambitious is not a bad thing.  Wanting to be rewarded, supported, free and flexible are not bad things.
I want to pay teachers more because I want to attract the highest caliber people into the profession.  Unless someone is independently wealthy money does factor into the decision making process of choosing a job.  That isn’t an insult to the art of teaching.  That is how the world works.

I want to advance the best teachers whether through higher pay, more powerful positions or some other kind of hierarchy.  Recognizing who your best are and asking more of them, compensating them more, giving them more power and leeway, that is good human capital.  That is making the most of your resources.  Not every teacher needs to be ambitious.  But those that wish to work their way up, to coach other teachers, to join in the policy discussions, to become principals and superintendents and thought leaders should do so.

I want a future President of the United States to be a former teacher from an urban school district.  Why not?  Republicans have a pipeline from the military to politics.  Democrats should have the same pipeline from urban teaching corps to politics.  Recruit a young, ambitious, socially conscious person out of college to serve our future as a teacher.  Watch them flourish, mastering the art of teaching, working as part of a team, helping others better themselves.  Watch them ascend and learn how to lead, how to manage, how to execute.  Watch them learn all of the issues that face politicians today by actually experiencing those issues, seeing what poverty and unemployment do, seeing the dysfunction of bad government.  They will master the politics of a school, of a neighborhood, of a ward, of a city, just like politicians do, just like military men and women do.  They can join their local school team, then the school board, then the city council, then become mayor or state senator or governor, then to the national stage.

I want teachers who are so great they can do anything they want but they choose to teach.  I want every individual teacher to be so valued that they hold power as individuals, able to say “I am so good that I am irreplaceable”.  

I want college students to battle over scarce slots in teaching schools, worrying over difficult tests, intimidating interviews and convoluted essay questions to get into that top choice school that will set them up for future success. 

I want alternative credential pathways to step up their game to ensure they are as good as the improving teaching schools, to make sure they don’t get left behind.

I want practitioners to go out of their way to learn to teach, to devote some of their time to sharing with others.  I want ambitious thought leaders to help form the minds of tomorrow.  I want them to do it for prestige, for networks, for recruiting their future heirs. 

I want teaching to be a stepping stone, a pathway, a destination, an expectation.

I want teachers who learn to teach, can leave to go do something else they are passionate about, and are able to return, better for it and reenergized.

When I talk about alternative pay scales, accountability, data, recruitment, retention, teacher evaluation, the end of tenure and LIFO, I’m not talking about disrespecting the profession or destroying it.  I’m talking about respecting the profession of teaching the way it should be respected.  I’m talking about making the profession more powerful.  I’m talking about treating teachers the way we tend to treat out best.  Driving them hard for more success.  Rewarding them well for being the best.  Giving them freedom to explore.  Supporting and nurturing their passions.  Allowing them to stretch themselves.  We do not do this now and in our never ending negotiation between management and labor it is both sides that are holding the profession back.

Wednesday, May 25, 2011

The Pursuit of Happiness

I never thought I’d be defending Rupert Murdoch and really I’m not.  I’m defending Wireless Generation sort of I guess.  Mostly I think I’m defending the idea that making money isn’t inherently evil.  Full disclosure: I haven’t made enough money in my life to know this for a fact. But I’m pretty sure that making a profit is an integral part of American society and power.  Of course there are villainous billionaires out there and Rupert Murdoch is a pretty good villain.  But there are villainous public sector workers too, and villainous Nigerian princes, and villainous writers too.

Valerie Strauss’ recent blog post paints a picture that makes for profit education companies evil because they are trying to make a profit.  She makes a couple observations that could be construed as a conflict of interest.  Basically Joel Klein left New York City Schools to work for Murdoch and now Murdoch’s company News Corp has purchased Wireless Generation, a company that does some big business with New York City Schools, and with other districts as well.  That is literally the extent of what I know so you can be the judge.  Doesn’t feel that iffy to me.  If anything it speaks to Klein thinking highly of Wireless Gen’s work.

Ms. Strauss actually acknowledges that “for-profit business can and do bring valuable products and services to public schools.”  Her major issue appears to be that for-profit companies care more about money and investors than they do about public schools and the kids.  Maybe that’s true although for-profit companies are also capable of believing in a mission.  Also self interest doesn’t exist exclusively in for-profit companies.  I can think of a few local DC watchdog groups whose interest seems to lie more in legitimizing their own existence than in actually creating real positive change for the kids. 

Teachers unions are great examples of institutions attached to education whose interest do not lie with students but with teachers.  That obviously doesn’t mean there is no place for unions in education. They serve their purposes and Ms Strauss, who is a champion of the union perspective, should be able to acknowledge that self interest can and often does intersect with the interest of the kids.

Obviously, not all business is great for students.  Just like not all activists are great for students, not all unions are great for students and so on and so forth.  There is always potential for exploitation.  But from what I’ve seen from Wireless Generation, and what I’ve heard smarter people than I am say, they by and large provide high quality service.  It is good that they are making money for providing a high quality service.  Too many of the vendors who cater to urban school districts only do so because they are bad enough at what they do and can overcharge by enough that it is worth navigating the nonsensical procurement processes that dominate government bureaucracies.

The bottom line for Ms. Strauss is that she believes big business is driving education policy.  Big business is savvy enough to enter emerging markets and with all this still relatively new focus on education and turning around low performing schools a new market has emerged.  It is education policy that is driving big business.  And by bringing a for-profit mentality into the market, new and innovative ideas are being introduced as well.  It has diversified perspectives, thought processes and personalities in a sector that has spent a long time not serving our most vulnerable communities.  This is a good thing on net.  There will definitely be slip ups, examples of profit corrupting a noble mission.  But impure motivations corrupting noble missions has always occurred and will always occur.  It is not a reason to hate all things profitable.  That is a viewpoint of a caricature from an Ayn Rand novel.

Tuesday, May 24, 2011

More Mythical Myths

Another article was published in the Washington Post taking aim at the myths education reformers are using to dupe the American people, or the media, or something.  It wasn’t so long ago that I used another article using the same myth gimmick as a launching point to talk about some of the major education reform issues that are out there.  These myth articles don’t really do it for me if only because they do not accurately represent the positions they are attempting to debunk.  Thus they are of minimal use.  This one in particular seems to be arguing against…nobody at all.

We’ll do this quickly myth by myth:

Myth 1: Our schools are failing

It isn’t just the education reformers who think a lot of schools are failing.  It is pretty much everyone who has a child in an urban school district.  It is most people paying attention to the educational outcomes of minorities.  Yes, things seem to be trending in the right direction.  But citing rising high school graduation rates that are derived from different methodologies and are often pulled from historically questionable data is not going to change the fact that way too many of our schools don’t educate students.  

Of course not all schools are failing.  There are good ones and bad ones.  Enough are failing and enough are failing in a systemic way that it feels quite cold to argue that there isn’t a problem.  For a little more oomph on this topic, responding to the same article, check out eduwonk who beat me to the punch with his pitch perfect response.

Myth 2: Unions defend bad teachers

Unions do defend bad teachers.  That is part of their job description.  And due-process can easily be code for really long, arduous, complicated process that is easy to file a grievance against.  That being said, unions are not universally bad, nor do they have bad intentions, nor are they unwilling to engage in productive reform.  A lot of the conflict between the reform movement and the unions comes down to negotiating.  There are of course high profile disagreements but on the ground and behind the scenes the relationships are not as antagonistic as they appear in the papers.

The real argument here is about reform minded administrators wanting more power over who their employees are.  This is really a myth about teacher evaluation, tenure and last in, first out policies.  Montgomery County is an interesting example.  I don’t know much about the program but according to the article it did result in 245 teachers exiting the schools over a five year period.  This is an interesting example and an interesting number to cite because 245 teachers over a five year period in a school system with 11,000 teachers seems shockingly low.  That comes out to less than half a percent of teachers per year.  I’ve never seen an organization whose human capital was so good that less than half a percent of its employees were better than replacement level.

Of course, there has been progress and unions have been a part of that progress.  More powerful and fairer evaluations are being developed.  They are still young, they are expensive but they are also essential.

Myth 3: Billionaires know best

There are some notable billionaires putting their money where their mouths are and forking over the dollars to some of the most difficult school districts in the country.  They are attaching those dollars to some pretty specific reform oriented programs that would not be possible without private dollars.  I didn’t realize this was a bad thing.  I’d rather risk Bill Gates’ money on new merit pay programs than tax payer dollars.  He was wrong about small schools, admitted his mistake and has moved on to a new idea that will hopefully be a part of a longer term solution to some of our country’s education woes.

This myth is really an attack on merit pay which I’ve already weighed in on.  But the final sentence really kills me:  “There’s no doubt that these schools can use every dime that rich guys give.  But attaching strings for pet projects is elitist and wasteful.” 

First of all I wouldn’t call merit pay for teachers the pet project of billionaires.  Secondly, wasteful would be pouring a billion dollars into a failing school district with no clear idea of what the money would be used for.  Failing school districts aren’t known for fiscal responsibility or effectively leveraging resources to create systemic success.  That is why they are failing.