That does seem silly. The previous year's teacher's raise should depend on current year scores. (Or possibly on the difference between the scores the year before that teacher and the year after.)
The part the boggles me is that apparently the student population is stable enough from year to year to allow this.
I'm used to schools where the churn is on the order of 35% or so in a given year. I would think Texas -- where people are not only moving from district to district or state to state, but potentially back and forth to Mexico -- would be even worse in that respect.
I can see why home schooling would seem compelling in this situation.
Teachers could be compensated by their ability to make schools more profitable. For instance, if it turns out to be more revenue-enhancing to have the students make iPods rather than learn history, why teach history?
We have STAR testing here. It's awesome. You'll love it.
I'm used to schools where the churn is on the order of 35% or so in a given year. I would think Texas -- where people are not only moving from district to district or state to state, but potentially back and forth to Mexico -- would be even worse in that respect.
You would be completely correct! The upheaval is phenomenal.
6: Then the situation described in your OP is even more ridiculous. Wow.
(I don't know anything about STAR -- is it better?)
I'll speak up in defense* of basing teacher compensation on this testing regime. Given the level of student population turnover and other effects, the results are likely to be functionally indistinguishable from random variation. Thus, the expected value of the bonus is about the same for all teachers: some years you get one, some years you don't. As long as the base compensation isn't reduced, it's a net gain for everyone. There are obvious drawbacks associated with demotivating teachers and wasting time teaching to the test, but in some respects, a completely fucked up regimen like this one is better than a merely imperfect one, in which the bonuses will be systematically misallocated rather than randomly distributed.
*mostly tongue in cheek.
8: Random rewards are good. If you put birds in a Skinner box that dispenses rewards at random intervals, it reinforces behaviour at random - known as "superstitious behaviour" or "religious behaviour". Knecht's proposal would make teachers more religious. Which is a good thing!
This might be on-topic:
Having two kids in elementary school has given me a new perspective on home schooling.
When I was a kid, we didn't get anywhere near as much homework, and there was zero expectation that parents would be helping out. Nowadays, The Missus and I are required to help our kids out a lot, and I gather that our very ordinary public school isn't unusual that way.
It seems to me that the schools have effectively enlisted parents as assistant teachers. For those of us who have the time to help out, this seems pretty great to me. I worry, though, about kids with less-involved parents.
It seems to me that the schools have effectively enlisted parents as assistant teachers. For those of us who have the time to help out, this seems pretty great to me. I worry, though, about kids with less-involved parents.
Those are the kids whose teachers get fired under knyecht/Obama/Rhee's randomized firing scheme. And good riddance!
"For those of us who have the time to help out, this seems pretty great to me."
I hate the increased homework expectations. There is no evidence that it is useful at all. I would be a lot happier being a parent in the 1970s.
I hate the increased homework expectations. There is no evidence that it is useful at all. I would be a lot happier being a parent in the 1970s.
Yeah, I am really dreading this. Why does it have to be this stupid way?
13
... Why does it have to be this stupid way?
Because people don't want to accept that some children are smart and some children are stupid and there isn't a lot you can do about it.
||
Is this the debate thread?
|>
14: Against all my better judgment let me ask the following. Assume that people did "accept that" (rightly or wrongly), how would that impact the homework question?
16
The additional homework is a response to the perception that the schools are "failing" or "broken" because all the children don't do as well as the smartest children. A more realistic attitude towards what schools can be expected to achieve would reduce such responses.
18
An example of the thinking involved is a quote from this NYT article .
... "If you don't solve the problem of teacher quality, you will continue to have an achievement gap."
This statement is true but highly misleading as it would remain true if you deleted the first clause. So it could equally well read
... "If you don't bring prayer back into the schools, you will continue to have an achievement gap."
The achievement gap can be used in this way to justify any pet crackpot theory you want.
14: Except for math, right? At least, I thought that Math homework was actually supposed to help.
And by analogy I'd expect that, if anyone implements similarly high-quality reading drills it would help with basic literacy. But I'm not aware of any actual evidence for that one.
19
Except for math, right? At least, I thought that Math homework was actually supposed to help
Help how? The purpose of school is to sort out the smart kids. Giving a lot of math homework doesn't change who is smart and who is dumb.
20: Helps them actually learn basic math. Not everyone needs Calculus, but everyone needs basic computational arithmetic. And my understanding of the research (plus my own subjective experience) is that for math, in particular, understanding and performance are increased substantially through practice.
People who are good with numbers can genuinely be productive in ways people who aren't can't. And being able to read and write is important, not just as a sign of achievement, but as a fundamental tool of communication.
I can believe that sorting is one of the primary functions of the school system, but that it is the purpose is incredible.
21
Helps them actually learn basic math. Not everyone needs Calculus, but everyone needs basic computational arithmetic. And my understanding of the research (plus my own subjective experience) is that for math, in particular, understanding and performance are increased substantially through practice.
Drill may improve performance but I doubt it does much for understanding. And drilling people to perform calculations which in the real world people use calculators for seems fairly pointless.
The purpose of school is to sort out the smart kids.
That shouldn't be the purpose of school.
21
I can believe that sorting is one of the primary functions of the school system, but that it is the purpose is incredible.
I exaggerate a bit. Schools should also attempt to teach some basic practical skills but seem to actively resist doing so. By doing things like dropping driver's education. And this is not the focus of the reform movement.
23
That shouldn't be the purpose of school
So schools shouldn't identify the smart kids and encourage them to pursue further education? What do you think should be the purpose of school?
So schools shouldn't identify the smart kids and encourage them to pursue further education?
Further education at... schools? But if the kids are already smart, and the purpose of schools is to identify them as smart, why would they need more school? Is it just a series of rubber-stamps for the already-smart? I suppose that is, in fact, one way to interpret the education system.
School is for learning to sit still, write down numbers and letters, and pay attention. Increasing homework seems justified if more time could be devoted to sitting still and paying attention, perhaps in front of a screen of some sort.
Once you've sorted the children by intelligence you know how much lead to put in their milk-boxes in order to achieve the true liberal goal of education, ensuring that all children are equally smart.
Is this the debate thread?
I caught the beginning of that over breakfast this morning! Then I got momentarily confused over why I was seeing a live debate over breakfast. Being on the other side of the planet is strange.
That's not an argument, that's contradiction!
26
Education?
Ultimately intellectually challenging jobs. But first you have to survive the winnowing out process.
28 School is for learning to sit still, write down numbers and letters, and pay attention.
Ideal preparation for a future life as a blog commenter.
20: The purpose of school is to sort out the smart kids
I sometimes think something must have gone very wrong in your education.
37
I sometimes think something must have gone very wrong in your education.
Because I didn't completely absorb all the self-serving pro education propaganda you get fed in the schools?
Oh James. I feel like I should dress you up in rags and take you to meet my people, the Autodidacts, in our strange, subterranean lairs deep beneath the sorting towers of the Temples of Learning. It'll be just like TV's Beauty and the Best.
I meant Beast but you know what? Way better as it is.
22: You don't think that knowing division and multiplication like the back of your hand would help someone better be able to understand slightly more complicated math like exponentiation? (Important for understanding common real-life things like compound interest.)
IME a conceptual grasp of one level is not adequate for getting a conceptual grasp of the next level; I actually need to be able to see immediately and effortlessly what's going on on the lower level in order to get to the higher one. IOW, to some extent performance is understanding.
I feel like maybe 41.last should have been posted in all caps by OPINIONATED WITTGENSTEIN or something.
Wait, we're arguing with James about education? ... Prank call! Prank call! I don't know us.
In his zeal for annoying liberals, James has (presumably without quite meaning to) ended up stating his, er..., controversial view that different people have measurably different natural capacities for learning (which is consistent with everyone learning something valuable at school, but some more than others), in such a way as to give the impression that he is asserting that learning at school is impossible (so that schools can at most sort people into groups according to their capacities for learning, with the actual learning going on elsewhere). Which is just weird.
(presumably without quite meaning to)
That I doubt.
45: Oh, he's definitely meaning to be annoying.
41
You don't think that knowing division and multiplication like the back of your hand would help someone better be able to understand slightly more complicated math like exponentiation? (Important for understanding common real-life things like compound interest.)
I think rote drill in things like doing long division by hand (which I seem to vaguely recall from third grade) is pretty much useless. And I don't think it promotes understanding, you just memorize an algorithm with no understanding of why it works. How many people who have memorized the formula for the roots of a quadratic equation could derive it? And solving 1000 quadratic equations in some pointless drill won't help.
Solving word problems seems a bit more useful but I am under the impression that teachers avoid them because students find them too hard.
44
In his zeal for annoying liberals, James has (presumably without quite meaning to) ended up stating his, er..., controversial view that different people have measurably different natural capacities for learning (which is consistent with everyone learning something valuable at school, but some more than others), in such a way as to give the impression that he is asserting that learning at school is impossible (so that schools can at most sort people into groups according to their capacities for learning, with the actual learning going on elsewhere). Which is just weird.
Not that learning at school is impossible but that the amount learned mostly depends on the kid independent of the quality of the teaching (within the range commonly found in American schools). And that much of what school teaches (like plane geometry) is aimed at identifying the smart students not at providing useful knowledge. And that the dumber kids mostly aren't going to be able to successfully compete for high status intellectually challenging jobs and schools shouldn't pretend they can change this by piling on the homework.
Doing algebra until it's second nature is helpful.
In my 20th, and last (!!), year of having kids in school, I find that my resentment at being drafted as an assistant teacher (as pf describes) has hit a pretty high level. I like teaching my kids things, and do it happily. I do not like being an enforcer of busywork homework loads.
I'm told that the big homework loads are so kids who don't do well on tests can do well in a class. What about kids who do really well on tests, but don't do well at giving over their lives to pointless busywork? Oh, fuck them, is the only answer I've ever gotten.
49
Doing algebra until it's second nature is helpful.
Helpful for who? Not the thousands of LAUSD students who don't have high school diplomas because they couldn't pass algebra .
Gabriela failed that first semester of freshman algebra. She failed again and again -- six times in six semesters. And because students in Los Angeles Unified schools must pass algebra to graduate, her hopes for a diploma grew dimmer with each F.
Midway through 12th grade, Gabriela gathered her textbooks, dropped them at the campus book room and, without telling a soul, vanished from Birmingham High School.
the amount learned mostly depends on the kid independent of the quality of the teaching
Even if this is true, it doesn't mean that the teaching is causally irrelevant, right? I mean, if a hundred trucks are used carry grain from point A to point B, the amount transported by each truck mostly depends on the capacity of the truck, and is independent of the quality of the driving, but it doesn't mean that the driving isn't causally necessary, or that it isn't easy to foul that part of the process up. But if you concede this, then why the attitude that teaching is some kind of racket?
52
... But if you concede this, then why the attitude that teaching is some kind of racket?
I have some sympathy for the current teaching force as they are being blamed for stuff that isn't their fault. But this is because teachers aren't actually that important (just as most drivers will get the truck to its destination most of the time). The rackets are the many dubious claims that we know how to significantly improve education by adopting whatever silver bullet an interest group is advocating. Whether that be charter schools, merit pay, higher teacher pay, higher standards, doing away with teacher's unions etc. I am annoyed at the extent to which the politics of this is driven by delusions and magical thinking and divorced from any considerations of empirical reality.
if a hundred trucks are used carry grain from point A to point B, the amount transported by each truck mostly depends on the capacity of the truck, and is independent of the quality of the driving, but it doesn't mean that the driving isn't causally necessary, or that it isn't easy to foul that part of the process up.
James covered this, but I have to admit that if I'm concerned about getting a bunch of trucks to carry grain from point A to point B, the quality of the drivers is not going to be a prominent consideration. Sure, you have to have drivers, but as long as they aren't completely incompetent, they'll probably do just fine.
This comment is precisely why analogies were once banned.
Sure, you have to have drivers X, but as long as they aren't completely incompetent, they'll probably do just fine.
Describes the vast majority of paid for work in the world.
And to finish the thought, and yet the people responsible for trucking and other tasks care about the quality of their workforce. Shockingly.
56: sort of, although with trucking in particular I think the workforce quality concerns basically relate to potential liabilities. You certainly don't hear a lot about how if only truckers were paid more and had more professional respect in our society, then the amount of grain being carried from point A to point B in America would be much higher.
Sure, you have to have drivers, but as long as they aren't completely incompetent, they'll probably do just fine.
I get that, but my point was that, even assuming arguendo that the quality of the teacher/driver doesn't matter much, that doesn't mean that teaching/driving is just make-work, which is the sense I got from James's comments earlier in the thread*, and which seemed to go beyond the positions he's taken in the past. But yeah, I ban myself (again).
What about lorries carrying nitroglycerine????
What about lorries carrying nitroglycerine????
Depends on whether the lorry is travelling towards a group of Nobel prize winners who could be saved by toppling a fat man into its path.
So James believes that even a very good teacher can have minimal to no effect on the educational outcomes of any particular student? That's a strange belief to hold, and I disagree with it. But if that's the gist, I see no reason to go on debating with him about it. On the internet.
60: Then the bad drivers tend to weed themselves out.
61: Whether the driver is Yves Montand in an oddly low-cut vest also plays a role.
The key part of 44 is:
Not that learning at school is impossible but that the amount learned mostly depends on the kid independent of the quality of the teaching (within the range commonly found in American schools).
The pretty way to word this is that mostly our teachers are good enough that we've passed the point of massively diminishing returns on teacher quality, and would do better focusing on other ways, if any, to help low-performing children.
It is of course easy to imagine a teacher so bad that the kids' learning suffers. For example if the teacher kills all the students, they will do poorly next year. But of course few if any teachers are that bad. The question is, of the variance in teacher quality we actually see, how much of the variance in student outcomes does it explain? I suspect not a lot.
But some. I've definitely learned different amounts from different teachers, in ways not strongly related to the explicit curriculum or the circumstances of my life at the time, but strongly related to teaching style.
53: Charter schools are promising as part of a search algorithm, but unfortunately don't seem to die fast enough when they don't outperform.
65
... The question is, of the variance in teacher quality we actually see, how much of the variance in student outcomes does it explain? I suspect not a lot.
According to the study I linked in the other thread the answer is about 1% (p. 24).
... Because the standard deviation of teacher effects is approximately 0.1 SD of the student test score distribution (averaging across math and English) ...
68: That paper is pretty cool.
If scores are normally distributed, shouldn't that be more like 4%?
69
If scores are normally distributed, shouldn't that be more like 4%?
I think you square the .1 to get the fraction of variance explained but I could be making some embarrassing error.
70: No, I think I was just thinking of a completely different metric. 4% of the population is between 0 and 0.1 sigma, which is not at all the same thing as explaining 4% of the variance. My error.
In fact, it looks like 1% is kind of an upper bound, because the effect seems to diminish over time.
On reflection, I think your interpretation is correct for student scores, but I can think of two complicating factors:
1) You have to count the effects from all classes in all years to get the total effect of teacher quality on student outcomes.
2) Variance in student scores is boosted by day-of-test random error. This will not be the case when considering overall life outcomes, as these sort of minor fluctuations should mostly cancel out. So if teachers' effect is more than transitory, it should become more important as we look at more long-term outcomes.
73
1) You have to count the effects from all classes in all years to get the total effect of teacher quality on student outcomes.
It seems to me that this will reduce the important of teacher quality as the typical student will get a mix of good and bad teachers and the effects will tend to cancel out. Whereas things like IQ or home environment will be pushing in the same direction throughout their school years.
72
... because the effect seems to diminish over time.
This is one of the many problems with trying to evaluate teachers. End of the year scores are not what you (presumeably) really care about.
This paper compares evaluating first calculus teachers based on their students scores at the end of the first year or based on how well they do in a required second year follow on course and finds no relation (the paper makes a big deal of the evaluations being negatively correlated but since the corellation coefficient is not significantly different from zero this seems a bit over dramatic).
Btw this paper cites (page 2) a bunch of other papers as obtaining similar results about the size of teacher effects (pre college settings) as the paper I mentioned abpve.
... Several studies find that a one standard deviation increase in teacher quality improves student test scores by roughly one-tenth of a standard deviation (Aaronson, Barrow, and Sander, 2007; Rocko , 2004; Rivkin, Hanushek, and Kain, 2005; Kane, Rocko , and Staiger, 2008). ...
The effects (of the first year teacher on first year performance or on second year performance) this paper finds are about half (or a quarter in terms of variance) this.
74: It should always reduce the measured importance because the relevant variance is not in individual teacher quality, but some weighted average of teacher quality for all the teachers a student has had, which will vary considerably less.
But depending on whether there are increasing or diminishing returns, it could either strengthen or weaken the true marginal impact of a single teacher's quality.
75: OK, I am convinced that within a school system, teacher variance is not a very important driver of outcomes, unless somewhere entire schools are being staffed with teachers more than 1 SD below the mean, or the opposite.
Could still be worth investing in getting rid of the worst and spending more money retaining the best, but not as important as it's made to seem.
76
OK, I am convinced that within a school system, teacher variance is not a very important driver of outcomes, unless somewhere entire schools are being staffed with teachers more than 1 SD below the mean, or the opposite.
High and low rated teachers tend to be spread around . A cynic might suspect this is because the ratings are basically random numbers.
They were in similar proportions in successful and struggling schools, and they were just as likely to have taught the most challenging of students and the most accomplished.
Btw another reason using test scores to evaluate teachers is problematic is the incentive it gives to cheat. The first study I cited was for a district where the scores were not used to evaluate teachers but it still seemed that a lot of the teachers ranked in the top 2% were likely cheating. See pages 23-24.
Finally, in our baseline speciĀ cations, we exclude classrooms taught by teachers whose estimated VA u^tj falls in the top two percent for their subject (above 0.21 in math and 0.13 in English) because these teachersĀ impacts on test scores appear suspiciously consistent with testing irregularities indicative of cheating. ... Nevertheless, the fact that high-VA outliers do not have lasting impacts on scores or adult outcomes serves as a warning about the risks of manipulability of VA measures. The signal content of VA measures could be severely reduced if teachers game the system further when VA is actually used to evaluate teachers. This is perhaps the most important caveat to our results and a critical area for further work, as we discuss in the conclusion.
Also note it is likely there are also ways (without cheating outright) such as teaching to the test of boosting short term scores without a positive (or perhaps even with a negative) effect on long term results.