Number 8 is also debatable. The hare or the owl?
Surely the most outrageous thing is that the test says that the story is "by Daniel Pinkwater", when Pinkwater never wrote that story at all (search for "ok here is the deal"; no permalinks for some reason).
Hungry is the Joe Bloggs answer, so in order to know whether it's right or wrong, I need to know where in the exam these questions occurred. If it's supposed to be an easy question then A is right, and otherwise C is right.
I agree that 8 is a bigger problem. A surface reading would suggest D (and I'd bet that that's the "correct" answer), but surely it's A. In fact the story *is a joke*, while the Owl unwisely thinks it's a parable about pineapples and sleeves.
I'm having some trouble with 11 as well: "When the moose says the pineapple has some trick up its sleeve, he means that the pineapple." I'm pretty sure that the test is looking for the answer C, "Has a plan to fool the animals", but that's not actually right -- the moose isn't speculating that the pineapple has a plan to fool the animals, it's speculating that the pineapple has a plan to win the race.
E. Because there wasn't an eggplant at hand.
Now I really do want to see the Dikembe Mutombo questions. (Was it the same test?)
No, this is eighth grade -- my kids are fifth and seventh.
None of the questions have a right answer with the possible exception of 6. This is because the story is nonsense. Small children understand nonsense, whether it's presented by Lewis Carroll or Dr Seuss or anybody else; they understand that the rules of this world don't apply in nonsense world. Therefore the answer to questions 7-11 (and for all I know 1-5 too) is, "None of the above."
This is a pernicious test.
I also kind of hate 9, how did the animals feel about the pineapple before the race? I assume the desired answer is 'suspicious', but I could make a strong case for 'kindly': at first, there isn't a unified way 'the animals' feel about the pineapple -- some are suspicious, and some are sort of contemptuous. Then, all the animals decide the pineapple has a trick and root for it on that basis. If you're convinced someone is going to pull a trick, and you're cheering for them, you're really not 'suspicious' of them, are you?
The funny thing is that after reading KD's post I was totally ready to be like "come on, of course they all have right answers, why are people complaining." In fact, when I first read about it there I didn't even bother to click through because I was sure it wouldn't be interesting. But in fact, wtf is with those questions?
I liked Felix Gilman's take via Twitter: standardized testing as gigantic dadaist intervention to teach children that everything in their world is nonsense.
I suppose I should be glad that twenty years later I've forgotten the exact wording of the question and the two alternate answers, and why I argued for the second best one even though my friends all agreed on the other as the right answer. But I was damn sure of my reasoning on the (wrong) answer, and I didn't yield, even though it was the only question I got wrong in 8th grade and I stuck to my answer when I saw it again in 11th grade, because I was right, no matter what those fuckers who graded the test said. Something about a camel, maybe? Was it the CBEST?
Whatever. With a whole lot of therapy, I've moved past it.
11: Well, it's an egregious example of a sort of question that's fairly typical on this kind of test -- generally, they don't ask for the 'right' answer, they ask for the 'best' answer. Which means that the trick to doing well in cases of ambiguity is to figure out what sort of thing people who write tests like that are likely to think is the best answer. This may be a skill that correlates well with academic success, but it's one that I always resented having to develop.
6 and 9 are fine, and I think 11's ok too (I agree with LB's criticism that there's a better version of the right answer, but still none of the "wrong" answers are very good). Of the rest, 8 is wrong, 7 is ambiguous (decisions don't typically have single well-defined reasons, and surely hunger was also a contributing factor), and 10 is completely misguided (you need to have a more fully constructed world for counterfactuals to make sense).
14: But as 11 points out, children already know that.
Well, at any rate, its important that kids spend less time at the playground and more time in school, so they can do better on this kind of test.
Question 7: Kraab is wrong. Theoretically it could be C or A, but C is clearly better - it's the answer that relates to the story.
Question 8 is the worst, definitely. The moose and the crow were wrong, but the other two speeches (those of the hare and the owl) are simply incomparable on the dimension of wisdom.
Question 11 is just checking that you know what the idiom "a trick up one's sleeve" means. Yes, the prescribed answer's not quite on the mark, but all the others are solidly wrong.
Everyone involved with this debacle should lose their jobs, the building where the test was written should be burned to the ground, and the earth salted around it. Chris Y gets it right.
I don't think B is "solidly wrong" on question 11. Worse than C, certainly, but not that bad.
It's actually a test that punishes children for being too smart.
Am I right that the actual answers haven't been released? Or have they?
The right answer to 10 is that the pineapple would be a cheetah, because people in these kinds of stories always root for the underdog.
Question 11 is just checking that you know what the idiom "a trick up one's sleeve" means. Yes, the prescribed answer's not quite on the mark, but all the others are solidly wrong.
While the desired answer is clear to me, I still think that it's no less wrong than, say, B: "wants to show the animals a trick". Having to pick between two wrong answers on the grounds of which sort of wrong is wronger annoys me.
but C is clearly better - it's the answer that relates to the story.
Relates to the story, sure, but there's nothing in the story that makes it more likely to be right. What makes C clearly better isn't anything I know about the story, it's what I know about what standardized tests are like. Feh.
WTF, these questions.
I don't see how C could be considered better than B for #11. "A trick" is a reasonable synonym for "a plan to fool," and he particularly wants to show this plan, because he challenged the hare to the race. In fact, perhaps B is better than C, because if the hare loses a race to a pineapple because of the pineapple's clever plan then only the hare has been fooled, but all the animal spectators have been shown the plan. Thus given that both options involve "the animals" in general, B makes more sense.
Now I'm being haunted by vague memories of ambiguous questions throughout my primary school career. I'd kind of forgotten what non-math education was like. God this shit drives me crazy.
One Calvin and Hobbes strip centered on Calvin's graphic fantasies of blowing up his school. Many papers refused to print that strip because their editors thought it would be offensive. Bill Watterson's reply to that was that those editors must have forgotten what it was like to be a child. And remember that the series ran from 1985 to 1995, well before the NCLB and the weekly-standardized-testing system.
Man, I hated school so much as a kid. Never really pulled out of feeling that way.
The problem isn't so much the specific questions, though they are stupid, but the premise of taking (or as it turns out more or less making up) a nonsense story and asking people to infer things about the mental state of the protagonists on no textual evidence. They're talking woodland animals contemplating a race involving a tropical fruit! How the hell should I know what they're thinking if the story doesn't say it explicitly or even imply it? Number 10 is particularly funny in this context - it's a counterfactual about a nonsense story's protagonists' mental states.
I guess to know 11B is wrong you also need to read "show a trick" as having an idiomatically different meaning than "trick"; very likely how the writers were thinking but not well-grounded enough in this case.
I continue (from FB) to maintain that every question has a clear least bad answer.
I was (and am) too pleased at being patted on the head to have resented school, even though I had many of the bad experiences that Unfogged types (and Bill Waterson, for that matter) found maddening.
If only someone could tell me what that means!
And I continue to maintain that for question 8 the clear least bad answer (A) is not the "correct answer" (D).
33 also gets it right. The only "correct" response to this test is to set it on fire.
I imagine the construction of these stories and questions is very much like committee-horse-design.
2: Can't we conclude that it's the owl since his words are stated to be the moral of the story?
you also need to read "show a trick" as having an idiomatically different meaning than "trick"
But 'show a trick' is actually a better description of what the moose thinks the pineapple is going to do. The moose doesn't expect that the pineapple is going to deceive the animals, it expects that the pineapple is going to do something clever and unexpected that will make it win the race -- much more like 'showing a trick' than 'having a plan to fool'.
What makes it clear to me that 11B is wrong is that it's slightly awkwardly phrased -- 'show a trick' isn't so much wrong, as it is ungraceful English, and right answers aren't phrased ungracefully on this sort of test.
I went straight from reading these comments to link below, and have no doubt that I've discovered the underlying problem!
http://blogs.discovermagazine.com/80beats/2012/04/23/computers-can-grade-essays-as-well-as-people-can/
40: Except that the story is nonsense -- why should its moral be wise? (I mean, you're right, that's how you're supposed to think, I think, but it only works if you read the test with no sense of humor at all).
I wonder how reliable you could get with a reading comprehension test where the questions and answers were actually nonsense, in the sense of being strings of random words? It would take a big sample and a lot of revisions, but I bet you could eventually do pretty well.
A funny thing is that while I'm outraged about the lousiness of these test questions, I don't actually know if they're bad as test questions: that is, if kids with better reading comprehension were likely to have answered these as the test-writers intended. It seems perfectly possible both that the questions are maddeningly stupid and ambiguous, and that bright kids who read well will be able to spot the answers that the test-writers want without much difficulty.
But you have to read these things without a sense of humor or really any other ideas that a living being might have. God, I have always hated reading comprehension questions.
Which means that the trick to doing well in cases of ambiguity is to figure out what sort of thing people who write tests like that are likely to think is the best answer. This may be a skill that correlates well with academic success...
It is more than that--it is a subset of the broadly useful life skill "thinking like the people with power over you."
Knowing how to think like a test designer is the master skill for all tests. It is useful for every possible test that can be designed. Life is full of bureaucracies, and bureaucracies are full of tests. This alone is reason to learn to think like a test designer.
But even when you are not dealing with formalized tests, the people with power tend to be people who have passed standardized tests. It is good to know how to deal with them.
As I teacher, I think that one part of my job is teaching students how to work the system. It is not my favorite part of the job, but it is the part of the job that best satisfies the reason students come to education in the first place. They are here for economic advancement. Since they have paid their money, I owe it to them to spend some time giving them the thing that they wanted.
Meanwhile, the University of Florida has eliminated its computer science department in order to save 1.7 million at a time when it was raising its athletic budget by 2 million.
Knowing how to think like a test designer is the master skill for all tests. It is useful for every possible test that can be designed. Life is full of bureaucracies, and bureaucracies are full of tests. This alone is reason to learn to think like a test designer.
If I were Bob (and aren't we all Bob, sometimes?) I'd speculate that the point of teaching clever children to empathize with the people with power over them is to control them, or to train them to control themselves without realizing what they're doing.
The answer for 7 could be either (a) or (c) -- there's just not enough textual evidence to go on. The animals probably were annoyed, but it's not clear why that would drive the animals to eat the pineapple. Maybe they eat it because they want it dead, but why can't they just kill it? On the other hand, the animals probably *were* hungry -- they had been waiting around for two hours, after all -- and hunger obviously will drive an animal to eat food.
But the answers for 8 and 11 are just wrong. In 8, the hare points out that the pineapple can't pose a reasonable challenge in a race, and that this must be a joke. This is much wiser than what the owl says, which *sounds* all mysteriously meaningful, but is actually totally cryptic and basically unhelpful.
For 11, why would the moose believe that the pineapple had a plan to *fool* the animals? Fool the animals how? The moose believed that the pineapple had a way of winning the race, and had a trick for doing so. By winning the race, the pineapple would have to perform its trick, and in doing so, reveal, or show, the trick.
As evidence of 45, there doesn't seem to be any disagreement in the thread about what the "correct" answers are supposed to be. Just how good or bad they are.
This is why we need Urple.
To speculate that the animals really should have eaten something other than the pineapple?
I agree with 45/50 that it's easy enough to tell what answers would net you the highest score -- we've all had enough experience in school testing to figure that out. The problem is that those answers aren't factually right in this instance, so that the test is only assessing a very limited kind of scholastic preparation, rather than what it is ostensibly testing, i.e., reading comprehension. In fact, better reading comprehension would lead a student to do more poorly on this test.
I guess I'm a poor stand-in for Urple, because I don't have strongly held beliefs about what the correct answers ought to be. But I am legitimately unclear on what the correct answers to 8 and 10 are supposed to be. (I have misgivings about questions 7, 9, and 11, also, but I think the answers to those are supposed to be C, A, C.)
What clues should I be picking up on that would tell me the least-bad answers?
As evidence of 45, there doesn't seem to be any disagreement in the thread about what the "correct" answers are supposed to be. Just how good or bad they are.
No, there is. That's why I asked if we knew the correct answers. I thought the animals ate the pineapple because they were hungry. And that the owl spoke the wisest words.
And I have no idea about 9. If it were fill-in-the-blank, I would have said "curious".
54: Some of us had inadvertently cheated by reading the New York Times article that told the "correct" answers.
7: C (ate the pineapple because they were annoyed)
8: D (the owl said the wisest words)
9: A (the animals were suspicious)
10: D (if the animals had cheered for the hare, they would have been happy to have cheered for a winner)
11: C ("trick up his sleeve" means "plan to fool")
I'd argue with all of these but 10, but if I were trying to score well on the test rather than to give the answers I think are best, this is how I'd answer.
I agree with everything in 57, except that I'd also argue with 10 where the only correct response is to refuse to answer.
Oh, I missed that paragraph. The correct answer is that they ate the pineapple because they were ANNOYED THAT IT TRICKED THEM?!?!
That makes no goddamn sense whatsoever. I never in the passage got any inkling that the animals turned on the pineapple. I thought they were curious and excited ahead of time, and afterwards thought "Oh well, I guess he's just a regular pineapple. Might as well have a snack." WTFuckingF?
Even though it's an asshole, the pineapple is still a sentient being. The animals ate the pineapple because they were murderers.
I believe that the hare is the wisest animal, but I would have known to pick the owl.
The argument on 8 is that the moose and crow are just wrong, and the hare isn't talking in the right register to count as 'wise' -- he's argumentative rather than pontificating. The owl is pontificating, and can be read as correct in an enigmatic kind of way, and gives the moral of the story. So, 'wise'.
10 actually is the only one where I'm perfectly happy with the 'right' answer: none of the other answers are attractive at all, and we are explicitly told that the animals wanted to root for the winner, so speculating that if they had rooted for the winner they'd be happy seems reasonable.
While I agree with k-sky that the only way to get the intended answers is the least-bad route, I disagree that this leads to unambiguously determined answers, as 49 does a good job of demonstrating.
I assert that there's no way to prove from the text that the animals ate the pineapple from annoyance rather than hunger, or vice versa. The only way to prove one or the other is to speculate about the tentmakers.
I'm pretty sure that, if testing companies and educational "reform" advocates stated their case as, "Children must take lots of standardized tests so that we can determine which of them are best at divining the mental states of test makes," they'd be run out of town on the proverbial rail. We can debate all day long whether or not that's a legitimate thing to test, but it's undeniably not what the testing is predicated on.
60: Like the man says, pineapple is murder.
Killing pineapples is not murder, because pineapples don't have sleeves. Didn't you learn anything from the story?
The owl is pontificating, and can be read as correct in an enigmatic kind of way, and gives the moral of the story. So, 'wise'.
Also, owls are always wise in animal stories; the fools who designed this test didn't even have the gumption to make it a goose. Fire the lot!
48: Disciplined Minds.
Standardized tests are murder. And this inane arguing over these somewhat more inane than usual questions is indirect ratification.
OK, on a reread, I actually do see some ambiguity in the question about which animal is smartest. Here's the problem: the story isn't nonsense. The moral is just so much more oblique and metaphorical than the rest of the story that it looks like nonsense.
It might look like either the moose or the crow is the smartest, because they both say something relating to why and how the pineapple might have some kind of trick to win the race. The owl simply said "pineapples don't have sleeves," which looks like the kind of little bitchery we're familiar with non-substantive nitpicking that's beside the point of the animals' discussion. But in hindsight, the pineapple didn't have a trick. It's just a fruit, no arms or legs or clothes, it can't even move like a tortoise can. It's got nothing up it's sleeves, either literally or metaphorically.
So you can say that there's an oblique, sage moral even though the story is really elementary in all other ways, or you can say that the owl is the smartest because owls are traditionally symbols of wisdom in modern Western culture, or you can say that the owl is the smartest because its quote is used as the moral.
So there is a right answer, I'd say, but there are both good and bad reasons to pick it, depending on what level you're thinking at. That's bad test design. It's also bad design to have one question that's so much sneakier than all the rest.
51: Can I use that as an excuse to say that Mara was puzzling over the story of Eve and the tree of knowledge (because she wanted to know why the woman in the statue was sad and had a snake) and decided that maybe it was a minsundestanding because if Eve was really hungry, it was better to eat the fruit than the snake. She is Team Eve all the way ("It's okay to make mistakes, Eve! Poor Eve, so sad and beautiful!")
I'm guessing that helpy-chalk isn't teaching clever children empathy as much as about reminding them that taking tests is another form of code-switching or something like that. That's how I've approached it in trying to help the kids at church get ready for the ACT and their graduation exams.
But in hindsight, the pineapple didn't have a trick. It's just a fruit, no arms or legs or clothes, it can't even move like a tortoise can. It's got nothing up it's sleeves, either literally or metaphorically.
Except that this is the sort of pineapple that can talk, and issue challenges to a race. The owl isn't bringing back the other animals to an understanding of the simple reality that pineapples aren't the sort of thing that can possibly pull tricks, because in this story, pineapples clearly are the sort of thing that might possibly be able to pull a trick. This pineapple didn't, but the story wouldn't have been any sillier if it had.
I'll have to talk to my shrink about why the mistake in 63.2 made me try to understand it in terms of a reference the biblical story of Jael and Sisera instead of just autocorrect acting up.
the biblical story of Jael and Sisera
That's one seriously fucked-up story. Not that there's any shortage of those in the OT, of course, but that one is one of the worst.
If we outlaw tent pegs, only outlaws will have tents that don't fly away.
Victorian novelists really liked it. The only reason I know the story is that it comes up over and over again in nineteenth century novels when people are playing charades.
Indeed, just before Jeeves came in, I had been dreaming that some bounder was driving spikes through my head--not just ordinary spikes, as used by Jael the wife of Heber, but red-hot ones.
I suppose it would be pretty useful in the context of charades.
As I said on Facebook, I wish the test designers had gone with one of the Kevin Shapiro-gets-murdered fantasias from Young Adult Novel.
As so often happens, wikipedia sucked me in. So, I ask you , how did the Ottomans fuck up this badly (i.e. only killed two French soldiers. It's like they didn't even have guns.
It occurs to me that a major concern for testmakers must be to avoid any content that has the slightest whiff of controversy. This passage may have seemed like a good choice, because it is so transparently silly, that it coudn't offend anyone.
but it's undeniably not what the testing is predicated on.
But there's a school of thought that pretty much all tests of this kind are tautologous in that sense. The literature is full of arguments (which I'm inclined to accept) that IQ tests test your ability to do IQ tests and that g, if anything is simply that ability.
On reflection, I think the kind of question which one can more or less answer by comparing the fitness of the possibilities against each other, but not actually be confident in the answer, is extremely common in standardized tests (or was 10 years ago when I was doing them). The inexplicable nature of this reading just shows them up better.
My SAT prep book (Princeton Review, I think) outright said that you can always eliminate answers uncomplimentary to authority without bothering to read the passages.
The only way to prove one or the other is to speculate about the tentmakers
Blessed are the tentmakers, for they shall inherit the pineapple's sleeves.
This passage may have seemed like a good choice, because it is so transparently silly, that it coudn't offend anyone.
Except having chosen it they changed it, quite possibly because they thought having the moral of the story be about gambling would offend someone.
Kraab is wrong.
The important thing is that people are arguing about it.
I think the kind of question which one can more or less answer by comparing the fitness of the possibilities against each other, but not actually be confident in the answer, is extremely common in standardized tests
I dunno. I was really good at taking reading comprehension tests and this level of ambiguity is unfamiliar.
Looking at it again, I think 11 is the one I would have had the hardest time with.
you can always eliminate answers uncomplimentary to authority without bothering to read the passages
I am sure that that's 100% true, and usually it's about as subtle as a Goofus & Gallant cartoon.
71: the owl is awakening the other animals to the mass delusion that has consumed them. Soon they will fall silent, as they realize that animals can't talk, either.
By arguing about it the terrorists have already won.
I was really good at taking reading comprehension tests and this level of ambiguity is unfamiliar.
To clarify, by "confident in the answer" I mean in the sense that of actually believing the answer is true, as opposed to being pretty sure by applying test-taking skills and knowing how the writers think.
How'd ya do on the tests on misplaced modifiers, JP?
I was good at those tests too, but the following train of thought is very familiar to me: "C. Of course, it could be B, objectively, but that's not how these tests work."
One of the luckiest facts of my entire life is that I have not been subjected to this kind of idiocy since I was 13 years old. Even then it was unbearable.
There is no greater indication of the contempt in which we hold young people's humanity than the fact that we subject them to these sorts of exercises -- and increasingly force their future lives and opportunities to depend on them.
Like everything else, change will probably have to wait until it finally becomes too painful for the adults.
I don't need to channel bob; I'm plenty angry all on my own.
91, 93: Actually, I'm pretty good at finding them. But terrible at it for text that I am still in the process of authoring (or even that I have recently written--when I still have in my mind what I intended to say).
70: Code switching is a closer analogy to what I'm doing there than empathy, because I am interested in having people identify strategies and agendas, not emotions or ways of looking at the world.
Of course, teaching someone how to win in the system inevitably means pushing them to accept the legitimacy of the system, which is a problem.
In designing my own multiple choice tests I go out of the way to create problems that can't be gamed using the simple tricks I know work on other tests. One of Arthur Kaplan's original pieces of advice was that the longer answer is more likely to be right. I've seen teachers for whom that advice will work 100% of the time. But I don't let students work that trick on my tests.
I also have correct answers which are disrespectful of authority, sometimes even my own, but I didn't realize I was breaking test-writing norms there.
You can do a lot more with multiple choice testing than people give the tests credit for, but test writers usually don't put in the effort.
You can do a lot more with multiple choice testing than people give the tests credit for, but test writers usually don't put in the effort.
I actually completely agree with this.
Didn't the owl cheer for the pineapple, thinking that he was cheering for the sure winner? Even though he knew that pineapples don't have sleeves? That ain't wisdom.
The hare, on the other hand, neither says nor does anything stupid. He does demonstrate that the rest of them are fools for not believing what they knew to be true. A lot wiser than some metaphoric (and in the event useless) bullshit.
You can do a lot more with multiple choice testing than people give the tests credit for, but test writers usually don't put in the effort.
I had one Math professor in college who had sections of True/False questions which were genuinely difficult, and not in a trick question sort of way.
I never asked him, but he must have taken pride in his ability to write those, because that was literally the only time I've seen that.
The hare, on the other hand, neither says nor does anything stupid.
Well, he did run a marathon, and didn't even raise any money for charity.
I'm not rule, but I thought that the best answer to 7 was D. Amused.
My phone changed urple to "rule."
The trouble is that the owl is being literal about sleeves, but "tricks up ones sleeve" is a figure of speech. So the owl is, in fact, being an idiot.
I would have gotten this answer wrong, and refused to change it when told otherwise. Which is one reason I generally got C's in English class.
100 -- At least it wasn't crossfit.
I would love to know if this test is actually a bad test, in the sense that I suspect performance on this test probably correlates well with performance on other tests of reading comprehension, despite everything.
I wasn't really kidding about 101. I think that the story itself is so absurd--and the animals were aware of this--that they were amused and decided that the most appropriately absurd thing to do was to eat the pineapple.
I would love to know if this test is actually a bad test, in the sense that I suspect performance on this test probably correlates well with performance on other tests of reading comprehension, despite everything.
I suspect performance on most tests correlates well with performance on other tests of reading comprehension.
Maybe I'll try the questions on my kids -- they're both test freaks like the people here, generally, but I don't know if they've developed the conscious ability to identify the 'right' answer even when it's wrong yet.
Since it is my standard joke to bring up fanfic (because I find it endlessly funny) I am writing powlnapple NC-17 H/C f-preg* as we speak.
*fruit
107.2 yeah, and to be super-cynical, i bet a question about parental income would probs be a pretty safe replacement for any given question on this test.
I agree with CharleyCarp.
"pineapples don't have sleeves" has two meanings:
1) talking pineapples do not have hidden motivations and desires.
2) pineapples do not wear shirts.
The owl is originally ambiguous as to his meaning. The moose then says he agrees that meaning 2 is correct but he still thinks that the pineapple is using some trickery In response to this clarification, the owl says nothing (refusing the opportunity to make an argument with respect to meaning 1) . The owl then agrees with the other animals that they should root for the pineapple (which he shouldn't do if he believed meaning 1).
The owl doesn't come across as wise to me. He is just a dumbass who doesn't understand metaphoric language.
I think all these talmudic readings of the story would work well if they focused on the one true version of the text, from Daniel Pinkwater, which doesn't even feature an owl.
In the original version, before the test editors got ahold of it, the allegory was much more clear. In the original, the owl made a series of simple, obvious-seeming statements that completely missed the point. But the owl also had a moustache, and was therefore a Very Serious Owl.
peep nailed it in 3. I don't feel that question 10 is eliciting sufficient loathing. Sure, the "correct" answer is (D), but given the author's absolutely clear intent to satirize the tortoise and hare, the actual right answer is either (A) or (C).
As 16 and some subsequent comments point out, this test suffers from the Clever Hans problem. But that's to be expected when the testers are not very smart.
48
If I were Bob (and aren't we all Bob, sometimes?) I'd speculate that the point of teaching clever children to empathize with the people with power over them is to control them, or to train them to control themselves without realizing what they're doing.
And this is bad why?
85
I dunno. I was really good at taking reading comprehension tests and this level of ambiguity is unfamiliar
I agree. This is much dumber than anything I remember from my school days.
And I would vote for the hare as smartest.
81
But there's a school of thought that pretty much all tests of this kind are tautologous in that sense. The literature is full of arguments (which I'm inclined to accept) that IQ tests test your ability to do IQ tests and that g, if anything is simply that ability.
IQ is what IQ tests measure just like time is what clocks measure.
117: Well, the hare probably is the lightest hued of those animals.
And the question was which spoke the wisest words, not to necessarily impugn your reading comprehension.
just like time is what clocks measure.
I am going to start calling clocks "time tests".
Well, the hare probably is the lightest hued of those animals.
It's like you've never even seen an owl.
123 from the collected works of John Derbyshire.
time is what clocks measure.
Yes, but...
I was so happy when I got to the point of taking tests of reading (and writing) in which you could actually write sentences. I always had trouble with "best answer" "what was the mood of the turtle?" questions. I never started up arguing about them, but I wanted to.
IQ is what IQ tests measure just like time is what clocks measure.
You still haven't read Cosma's g post, have you? In which he demonstrates that purely random series have a general factor of intelligence.
I'm interested by Shearer's response, because I think he's about the same age cohort as me. If anything like this had emerged when I was a kid, there wold have been parental riots, questions in parliament, and the Daily Mail would have forced senior people to resign. I was supposing this was a cultural difference but if James also thinks it wouldn't have been acceptable in his day, perhaps things have changed (for the worse in this case).
116. It's not bad unless it becomes your sole objective.
I have to say that this test would have reduced me to tears in my early teens, because my reaction would have been, "This damn thing is important for my future and the people who set it are just pissing about for their own amusement."
I think Shearer is a bit younger than you, actually.
I ask you , how did the Ottomans fuck up this badly (i.e. only killed two French soldiers. It's like they didn't even have guns.
I give you the Battle of Magdala. Ethiopian fatalities on the day: 700. British fatalities: none. And, yes, the Ethiopians had guns, and had them, what's more, behind the walls of one of the finest natural fortresses in the world.
But the British were commanded by Napier, and the Ethiopians were commanded by a nutter.
http://en.wikipedia.org/wiki/Battle_of_Magdala
I think Shearer is a bit younger than you, actually.
Doesn't invalidate my point; if he was older it might.
Eating a sentient, talking pineapple alive is a pretty disturbing image. I am hearing this story being read to the kids in the voice of Anthony Hopkins.
"Question 8, Clarice. Pay attention. What do you think the pineapple said when it realised the animals were going to eat it?"
Christopher Walken would also be a good narrator for this.
Another possibility for the "Disturbing Educational Movies" franchise.
http://www.unfogged.com/archives/comments_11260.html#1307985
129
... but if James also thinks it wouldn't have been acceptable in his day, ...
Not exactly saying that just that I don't recall anything like it. It's a big country and who knows what the dumbest test was back in the day. Also apparently it isn't acceptable today as it has caused a bit of a furor.
131
I have to say that this test would have reduced me to tears in my early teens, because my reaction would have been, "This damn thing is important for my future and the people who set it are just pissing about for their own amusement."
The other thing is none of these statewide assessment tests had any effect whatsover on your future when I was a kid. At least as far as I know. I don't recall being told or particularly caring how I did.
I don't know in what sense, if any, they are "high stakes tests" today.
128
You still haven't read Cosma's g post, have you? In which he demonstrates that purely random series have a general factor of intelligence.
I think what is actually claimed is that any set of mutually positively correlated series will appear to have a general factor.
If you haven't read his previous heritability and IQ post you should read that as well, James.
I'd like to see more kids encouraged to intentionally bomb these tests, especially the ones that have far greater importance to the school than to the individual child. This is a bad system that needs to be undermined.
I haven't seen anybody mention this bit from the NY Times story:
In the world of testing, she said, it does not really matter whether an answer is right or wrong; the "right" answer is the one that field testing has shown to be the consensus answer of the "smart" kids. "It's a psychometric concept," she said.
Even very intelligent children, she said, can sometimes overthink an answer and get it wrong.
So... you're not just asking kids to infer which answer some middle-aged educational bureaucrats think is "most correct," you're asking them to infer which answer provided by middle-aged educational bureaucrats is most likely to be chosen by their 80th percentile peers. This is the context in which the correct answer to 7 is "annoyed" --- it's the answer most not-incredibly-dumb kids would arrive at, regardless of how it's supported by the text.
This maps somewhat onto The Princeton Review's "Joe Bloggs" concept, _except_ that on the SAT the "Joe Bloggs" answer is the _wrong_ answer (i.e., it's obvious and attractive, the question must have a trick).
"Even very intelligent children..." is still part of the quote, obvs.
I can still remember a test question from 9th grade history where I was too smart for my own good. The question was: "In this system, the proletariat rise up to overthrow the bourgeoisie." Knowing perfectly well that this was a statement about the beliefs of Marxists, I answered "Capitalism" because that's the system in which the proletariat would rise up (according to the Marxists). The teacher just laughed at me.
S2 Geography exam.
Q9. Why do small shopkeepers dislike large out-of-town supermarkets?
The young ajay: Because they cannot reach the high shelves.
Some days later I dropped Geography FOREVER.
144: So it's a Keynesian beauty contest?
The quote in 144 might be designed to make me sympathise with Pol Pot's attitude to teachers.
I was so happy when I got to the point of taking tests of reading (and writing) in which you could actually write sentences.
For a question asking students to discuss why college costs are so high, Mr. Perelman wrote that the No. 1 reason is excessive pay for greedy teaching assistants.
"The average teaching assistant makes six times as much money as college presidents," he wrote. "In addition, they often receive a plethora of extra benefits such as private jets, vacations in the south seas, starring roles in motion pictures."
E-Rater gave him a [top rating of] 6.
I am going to start calling clocks "time tests".
Fruit tests prefer a pineapple.
I'm sorry. It's become compulsive.
The quote in 144 might be designed to make me sympathise with Pol Pot's attitude to teachers.
Q. Why did the animals eat the pineapple?
A: It was wearing glasses.
Chris Y, Halford and Witt have it exactly right. The only way I can think about the whole thing without getting totally enraged is to entertain the possibility that the story is intended as a kōan, and New York State is setting its students on the path to enlightenment.
I gave Sally the questions over breakfast this morning -- she knew about the fuss, but hadn't literally seen them. The response was "What the hell? They didn't count these, right?" She was running late, so I didn't press her for "So, if you were looking for 'what's the answer they want' rather than what's correct, what would you guess?"
The Joe Bloggs answer is not always wrong. It's right on easy questions and wrong on hard questions. Fortunately SATs gave questions in increasing order of difficulty. See my 5.
156.2: If this is correct that is not the case for the Reading Comprehension sections (but maybe they escalate within each mini-section associated with the individual passages?).
The questions in the first third of each section are easy, those in the second third are medium and those in the last third are hard. (The only exception is the Reading Comprehension passages, which do not follow this order.)
Unless I am getting my tests wrong, don't the results of these actually matter more for the schools and (in NYC at least*) the teachers than the students?
*Per a past thread on test-based teacher evaluation.
156: You are corrrect. Having taught Princeton Review for three years in the 90s, I am ashamed of myself.
Having taught Princeton Review for three years in the 90s, I am ashamed of myself.
Plus you forgot that Joe Bloggs thing.
160: What, you didn't make $20 an hour in college teaching kids to acquiesce to the existing power structures?
What, you didn't make $20 an hour in college teaching kids to acquiesce to the existing power structures?
Dammit, I worked for a Princeton Review knock-off, and only made $19 an hour.
161: No, I made less (1970s and all that). And it was acquiescence through calculus (which served a similar role for pre-Meds). Being an old-timer and all, I am gobsmacked by the massive change in attitudes towards prepping for standardized tests* (up there with the TV/movie quality reversal). One of my kids still does a lot of SAT tutoring which feeds my antipathy.
*Although in retrospect it just seems like we were just charmingly (and stupidly) naive.
The second time I took the SATs I prepped by getting piss-drunk the night before, and raised my score by 150 points.
4: I've just now read the original Pinkwater version. It's short enough to paste the whole thing here:
The Story of the Rabbit and the Eggplant
Once there was a race between a rabbit and an eggplant. Now, the eggplant, as you know, is a member of the vegetable kingdom, and the rabbit is a very fast animal.
Everybody bet lots of money on the eggplant, thinking that if a vegetable challenges a live animal with four legs to a race, then it must be that the vegetable knows something.
People expected the eggplant to win the race by some clever trick of philosophy. The race was started, and there was a lot of cheering. The rabbit streaked out of sight.
The eggplant just sat there at the starting line. Everybody knew that in some surprising way the eggplant would wind up winning the race.
Nothing of the sort happened. Eventually, the rabbit crossed the finish line and the eggplant hadn't moved an inch.
The spectators ate the eggplant.
Moral: Never bet on an eggplant.
Wtf, test makers?
165: How are we to speculate on the mental states of these disembodied people? What that story needs is talking animals.
An eggplant makes more sense than a pineapple.
An eggplant makes more sense than a pineapple.
Why? Are they known for wearing dress shirts?
More ambiguous test questions!
Joey, on his homework, was asked to write "the opposite of few." The correct answer was "many." Joey answered "none."
168: I'll leave it as an exercise for the reader.
169: Iris was just going over antonyms, and of course as she goes through the list I"m noting that many are not true opposites. More like "contrasting pairs".
171: Problems like that bug the fuck out of me. "Father" is NOT the opposite of "mother."
172 ironically, it's Tom Selleck.
143
140: Yes. Yes. See the problem?
Not really no. As I understand it, despite the claims of some enthusiasts, g isn't really uniquely specified. But so what? What practical difference does it make? You can still usefully say the stock market was up yesterday even though the market isn't precisely defined and I think you can talk about g in similar ways although the concept is a little fuzzy when looked at too closely.
141
If you haven't read his previous heritability and IQ post you should read that as well, James.
I believe I have read it. It seemed to me that Cosma was considerably more comfortable pointing out what was wrong with IQ research than what was right with it. While this can be useful it doesn't give a balanced picture.
Showed the test to my own 13 year old just now. Had to interrupt a game of 'guess the logo' on the ipod, so I'm not sure I had her full attention. She suggested "kindly?" For how they were feeling towards the pineapple, but got everything else 'right', without even a pause before "annoyed", and saying that the wisest must be the one who said the moral before checking who that was.
You can still usefully say the stock market was up yesterday even though the market isn't precisely defined
Shearer, when you say things like that you make the SEC a very sad bear. The stock market is incredibly precisely defined. How on earth could you think otherwise?
Also, analogy ban.
What's the opposite of a mustache?
A Brazilian. Opposite sex, opposite end, no hair.
178
Shearer, when you say things like that you make the SEC a very sad bear. The stock market is incredibly precisely defined. How on earth could you think otherwise?
You have many different indices defined in various ways, S&P 500, Dow Jones Industrial, Wilshire 5000 etc. Usually they move up or down a similar amount but they aren't precisely the same.
Also, analogy ban.
I think analogies are often useful. Almost all pairs of stocks have positively correlated prices which is the topic at hand.
But anyone who said that the performance of the stock market was driven by u, a general factor of upness, would be rightly told they were committing a tautology (as well as being an idiot). The stocks in an index are positively correlated by definition with the index - that doesn't mean there is some underlying metaphysical force driving it, just that the index summarises the behaviour of the stocks in it.
I've just realised that this is actually very much like the Ontological Argument for the existence of God. We can name all this stuff, but there is surely at least one thing we cannot, and that's got to be God. Similarly, we haz a factor residual, and that's got to be...if not God, then exactly whatever suits my prior prejudices.
179 is funny. You should write exam questions.
You can break down stock movements into beta, the movement of the market as a whole, and alpha, the degree to which one particular stock outdoes beta. But that's slightly different.
Big Owl has got to be one of the more successful pr outfits, from the time of Athena to now.
181
But anyone who said that the performance of the stock market was driven by u, a general factor of upness, would be rightly told they were committing a tautology (as well as being an idiot). The stocks in an index are positively correlated by definition with the index - that doesn't mean there is some underlying metaphysical force driving it, just that the index summarises the behaviour of the stocks in it. The stocks in an index are positively correlated by definition with the index - that doesn't mean there is some underlying metaphysical force driving it, just that the index summarises the behaviour of the stocks in it.
This isn't correct. The fact that stocks tend to move up and down together is an empirical fact. It isn't by definition or a tautology. And people do talk about a general factor which affects all stocks.
