The problem is that no one who is pushing teacher evaluation actually wants to do the difficult work of reliably assessing teaching. They just want to fire some people.
You need at least five years worth of data, for instance, to control for the random effects of a teacher getting a good or a bad bunch of students in any one particular class. But none of the proponents of teacher assessment want a five or ten year assessment cycle. They want to start busting heads now and have a threat they can hold over employees heads constantly in the future.
What Rob said. I get disagreeable about reformers pushing teacher assessment because they largely seem to me to be dismissing the difficulty of objective teacher assessment as unimportant. And then you see things like the linked story, where a system under heavy pressure to put an assessment system in place comes up with bullshit.
no one who is pushing teacher evaluation actually wants to do the difficult work of reliably assessing teaching. They just want to fire some people
This times a thousand.
But Yglesias says that teacher evaluation will not just lead to random teachers being fired, it will also lead to random teachers getting raises!
3 -- 1,000 might be low. You sure your math teachers were up to snuff?
I hate federalism. The federal department of education surely would have done a better job: they have more financial resources, smarter people, and know they can't micromanage every school.
Yeah, if I were seeing reformers identifying teacher assessment as a vital tool for figuring out who to direct additional coaching resources and support to, I'd be much less disagreeable about it.
7: Count me as one in that column, then. Of course few people read my op-eds (naturally so, as they don't exist).
There in fact are federal contracts ongoing that are studying the reliability of measurement problems and how to overcome them. But fuck that, it's a RACE.
That's actually a surprisingly sensible article from the NYT.
OT but this is an excellent article about the balancing act middle-class foster parents face. I have lots of little quibbles, but there's plenty of overlap with what we've experienced.
1,2:
Random effects aren't really decisive counterarguments on their own. After all, the current system also rewards teachers for things imperfectly correlated with teaching quality (like pursuing unrelated degrees, or sticking around for longer).
The problems of poor measurement as I see it are:
(a) Problems due to the disruptiveness of merit pay can be larger than the improvements. The stability of teaching positions/salaries is part of total compensation, current teachers are likely to be more risk averse than average, so adding negative performance incentives, even when balanced with positive incentives, amounts to a pay cut.
This is a good reason to start measuring before we start designing incentives, but people don't like to think about that, and I suspect reformers see the promise of immediate improvements as something of a noble lie.
(b) Measurement costs money that could be used directly on things that we know or suspect will improve outcomes directly.
I consider this one a weak argument, since you have to start somewhere, and there's no particular reason to assume we won't get better at measuring over time.
(c) the incentives created by imperfect measurements are counterproductive, e.g. incentivizing teaching to the test, or senior teachers trying to get students who are already good rather than the ones who need the most help.
This is probably the most serious problem, and by definition difficult to quantify. But as long as you take basic measures to prevent cheating (which, yes, are not actually taken everywhere), if students have been drilled for a single paper test instead of taught general skills, at least that means they have the minimal reading skills necessary to take the test. It's much easier to catch the low outliers by testing, than to improve the median outcome.
Note that this is not just an argument about teachers, but about measuring all sorts of hard-to-quantify-results performance.
I do agree with Rob that you need a decent model of what produces good outcomes, before you can start comparing actual to expected outcomes per teacher.
(a) Problems due to the disruptiveness of merit pay can be larger than the improvements. The stability of teaching positions/salaries is part of total compensation, current teachers are likely to be more risk averse than average, so adding negative performance incentives, even when balanced with positive incentives, amounts to a pay cut.
It's not just negative performance incentives, it's negative performance incentives with a significant random element. That's going to be a morale killer all out of proportion to the expected value of the negative outcome. (Aside from the fact that actual injustice is a bad thing, treating people transparently unjustly is not a great way to get them to improve their performance.)
This is a good reason to start measuring before we start designing incentives, but people don't like to think about that, and I suspect reformers see the promise of immediate improvements as something of a noble lie.
Yes, it would be a very good reason. But I'm not hearing anything suggesting that as a goal -- the reform rhetoric is all about finding and firing the bad teachers ASAP.
(b) Measurement costs money that could be used directly on things that we know or suspect will improve outcomes directly.
I consider this one a weak argument, since you have to start somewhere, and there's no particular reason to assume we won't get better at measuring over time.
I don't see the force of your second sentence. Are you claiming that money invested in teacher assessment will self-evidently be more productive that money invested in teacher coaching, improvements in staffing, improvements in physical resources, if we just look at a long enough baseline? Because I think that's got to depend on the specifics; you need to make a case that teacher assessment is the most useful place to put more money.
And on (c), good luck stamping out cheating when you make people's jobs depend on it, and (see (a), destroy their faith in the justice of the system as it applies to them.)
The difficulty with this issue from my perspective is that the arguments in 13 are so far removed from how (I've seen) these policies implemented that they're very nearly in the "and a pony" territory.
On top of that, there's the overarching issue that the worldview itself that these priorities emerge profoundly warps people's thinking. I realize that it's very appealing to a subset of people to think of education as a series of inputs and outputs, but the long-term consequences of that kind of mechanistic thinking are IME extremely damaging.
There is IME a willful deafness among a lot of advocates, such that if you said to them, "You can spend $2K a student on making sure they're all well-fed, or $2K on testing," they'd pick testing without a second thought, and without any real understanding of what poor nutrition or simple hunger do to people's ability to think, concentrate, and learn.
I hate the way this all trickles down to students. My NYC public schools kindergartner is coming home with 15-30 minutes of homework every night. Some of the homework activities are explicitly labeled as "test prep" worksheets, in which he is asked to fill in bubbles next to the correct answer.
While at school, his class spends most of its time on academics - math, reading. At his age, he hasn't yet developed the fine motor skills needed for writing, but they spend a lot of time on it anyway. Working on art projects seems comparatively rare, and "free time" is relegated to Friday afternoons. There is no recess.
He misbehaves in class and stress keeps him awake at night.
Yeah, kindergarten is really just first grade these days. That sounds ghastly.
You were having trouble getting into your local school, weren't you? Did that get straightened out, or does he have to commute on top of everything else?
Yeah, we did get him into the local school, after a long period on the waiting list. Coincidentally, he got off the waiting list right after testing into GT. I suspect they cherry picked him off the waiting list once they figured out he could bring up their test scores.
The location of the school is great (right next door), but, if we had really understood what it would be like, we likely would have gone with another option.
18: So sorry. We've got some of that going on too, though not with explicit test prep. I got in trouble with the teacher last week for not making Val do homework, when Monday was trick-or-treating and Tuesday was Mara's birthday (not that the school knew that) because I'm just not going to prioritize fighting through it when she already is in school from 8 am to after 3 pm. Argh.
Ooh, the motor skills for writing is so maddening. Newt takes after me in having great difficulty with a pen or pencil -- my handwriting is still humiliatingly insane-looking, and dealing with it was a nightmare through grade school. Newt's not as badly off as I was, but his ELA test scores took a huge leap between third and fourth grade, which is when he finally got to the point where he could write so someone who wasn't really personally attached to him could read it.
It's funny, I don't remember kindergarten homework; I don't know if we didn't have any, or if I've just blocked it out.
We would get a piece of paper each week with a letter at the top, and those lines for practicing handwriting. You had to write the upper- and lowercase versions of the letter 5 times, and then paste pictures below of things that begin with that letter.
Oh, I mean I don't even remember for Newt and Sally. I don't think there was much -- maybe a worksheet like the one you describe once a week, but not daily or anything.
Twitter tells me that Obama just announced he wants Head Start programs to start using some kind of results-based proof they're doing what they're supposed to if they want to keep getting funding. Yay for trickling down!
And Blume, something like that is Val's homework every night, plus a math page, plus writing her name four times. Sometimes other things too. Annoying when I have to manage it daily, because I'm just not the right kind of parent.
15: My point about cost is that you have to try measuring something before you can know for sure whether the measurement is meaningful.
You don't get rid of cheating by punishing cheaters, you get rid of cheating by making cheating either hard to do or easy to detect.
I do agree that you need to look at the specifics in each case. I don't have any particular insight on any particular measurement program. But there are lots of domains where we want to measure improvements/disimprovements in outcomes, and some of these criticisms generalize.
||
Apparently my habit of talking to absolutely everybody about voting is paying off. Two co-workers have been by my desk in the last 10 minutes to ask about ballot questions. Have just re-sent our local nonpartisan voter guide info to the entire office + volunteers.
||>
It seems like there's consensus here that scaling rewards and punishments continuously with test results has produced some bad outcomes. What about using it strictly to identify bad outliers? (say, 4 standard deviations below the mean)
Are bad outliers sufficiently numerous and hard to identify as to justify the gigantic machinery of testing?
Consider also that huge portions of NCLB and RACE have been crafted by ETS and other companies that administer these tests for profit.
What about using it strictly to identify bad outliers? (say, 4 standard deviations below the mean)
That just sounds like a huge waste of effort. I'm going by 'seems to me' here, but for four standard deviations below the mean, wouldn't you think that whatever the teacher was doing badly would be visible to the naked eye? Like, they would be doing concrete, observable things wrong? At which point a principal could take action on that basis.
The magic of 'objective', results-based assessment, I thought, was that it could find good and bad teachers who weren't doing anything that could be identified as different. When you're talking about wild outliers, I can't see why they'd be necessary.
30: Is there any reason to suppose that bad outliers are not otherwise identifiable? My experience is that it's not that people don't know who the truly bad teachers are, it's that there are other factors that cause those teachers not to get fired (sometimes, but not often, because of union protections).
High-stakes testing is an enormous, enormous externalizing of stress, fear, responsibility and guilt. Before we start shoveling that on to families who already have enough to deal with (and I mean pretty much all of humanity, not just poor families), I'd like to be pretty sure that a) the information we're going to get can't be gotten any easier way, and b) the information is going to be reasonably accurate.
33: Yeah, I really don't know if anywhere has adopted the setup I had in mind. I was thinking that if you had independent reviewers at the state level, these measures of student performance would be good enough to pick cases to promote to a human being's attention.
Although really you should be able to do that with nothing more than comparing letter grades as long as you get a decent level of dispersion, so this wouldn't really justify a formal testing infrastructure unless you'd already tried it with letter grades and decided that they're no good.
On reflection, I'm upgrading (b) to a moderately strong argument.
I'm really not opposed to using 'objective' assessment instruments to try to figure out who the best and worst teachers are, just to taking punitive action on the basis of instruments that haven't been shown to be valid and useful.
Plus you could already calibrate your assessment of HS grades based on SAT scores (with some adjustment for missings), and then chain your way backwards through the grades. So yeah, state-level testing is probably redundant.
I do think a required regime of formal testing could have been a way to force states that otherwise wouldn't to pay more attention to their failingest schools, but allowing states to design their own tests screws that up pretty badly.
I don't think we're that far apart, really. School reform issues are all the sort of thing where the devil's in the details; I can agree with the most intense of the school reform types on all of their statements of principle, and still want to dig my heels in at most of what's actually on offer as reformist policy as counterproductive or wasteful in practice.
27 would be good news if the alternative were spending the money on other things well-supported by the evidence like child nutrition or dentistry, but I suspect that's not what will in fact happen...
41: Sounds like we agree almost entirely.
At least now that I changed my mind on some of the cost stuff...
40: There's a funny issue here, and I don't have much of anything useful to say about it, just to point to it. But a lot of what seems to be going to me with the appetite for testing is a sense of despair(real or feigned) at the idea of improving management in a badly managed school or school system.
Like, I don't think it's actually that difficult for a school system to identify which schools have academic problems, there's just a sense that 'if we had numbers to point to, we could MAKE them do a better job'. And I think that's mostly a fantasy: if we could magically identify the objectively worst 10% of teachers every year and fire them, I don't think that would do all that much to fix the schools with problems.
OT:
I was not able to vote today because I do not have a government-issued photo ID with my current address pre-printed on it. They accepted my passport last year, but today told me it was no good, so I gather they've tightened the restrictions recently. This is goddamn ridiculous. What is the problem they think they are solving by doing this?
On second thought, maybe not so OT, because I'm pretty sure the people who want to keep me from voting are the same fuckers who want to cut schoolteachers' pay.
42: I just saw a comment from some journalist and then went to lunch, so I couldn't tell you. Neither Alex nor Mara explicitly attends a Head Start program, though both are in preschool/daycare programs that do have USDA food aid because they cater to low-income families.
Alex brings a bag of food home every Thursday since he's on a four-day preschool schedule and then all the kids bug me all weekend to please please cook the tiny can of beans and franks. I don't know how many of his classmates truly rely on that food, but I assume it's enough to make the administrators feel the program is worthwhile.
What is the problem they think they are solving by doing this?
That too many people vote for Democrats?
46: If you don't mind me asking, what state is this?
The same poll worker has been manning the table at my voting location for quite some time now. This was the fourth election in a row that he told me I have the same first and last name as his father.
Shit, I completely forgot that today is election day.
What is the problem they think they are solving by doing this?
Too many people voting who don't own land or can't pay the poll tax.
I was trying to find out if there was anything for me to vote for, but I think it's only city elections today--nothing at the state level, and my town's got nothing. They would have put something on their website or in the town e-mail, but I'm still anxious about the fact that I haven't voted, and I feel like I was supposed to.
That stinks. Did you at least get a provisional ballot, L?
Yes, I got a provisional ballot. They gave me a "Provisional Ballot Notice," which is very confusing, but I think I may be required to provide proof of identity to the Board of Elections within 10 days. It says:
In these situations, the law requires that your ballot can be counted only if you appear at the office of the board of elections and provide to the board any additional information necessary to determine your eligibility.
L., do try to follow up if you can, if nothing else because provisional ballots are a hotly contested issue in your state and more anecdotes/data could make a difference for others.
Come to think of it, a copy of my lease would probably work. This reminds me that I requested a copy of the lease a while back and my landlord blew me off.
A lease should do it. Barring that, in some states you can do an online driver's license address change.
The US's K-12 system isn't that bad. If you compare acheivement
The US's K-12 system isn't that bad. If you compare the skills of Chinese-American students with Chinese students, or Mexican-American students with Mexican students or European-American students with European students, the US students come out a little bit ahead. The US just has a growing percentage of hispanic youths that makes cross-national comparisons to China and Europe look bad and worsening. If the US educational system really were declining, we could fix it easily. Maybe even by returning teacher quality to some earlier gold standard. There is no such easy fix though.
I do not have bank statements or paychecks mailed to my house
Do those institutions have your current address on file, though? Surely you can print something out?
Not to take away from the PITAness of the whole situation.
Might not work -- when I've brought bills as proof of address, they wanted to see a postmarked envelope. A printout could say anything, but a letter that went through the mail tends to demonstrate that you have the capacity to pick up mail delivered to the address on its face. (Still, obviously, eminently forgeable, but the degree of difficulty is different enough that I could see someone demanding a mailed letter.)
6
I hate federalism. The federal department of education surely would have done a better job: they have more financial resources, smarter people, and know they can't micromanage every school.
I don't understand this comment. This stuff is being pushed by the federal department of education. Race to the top is a federal program.
In the US we seem to rely heavily on half-delegating programs to state governments. It'd be better for the national government to just come up with a plan themselves, rather than having states compete to come up with their own plans.