I've usually preferred those classes where the prof talked about five likely questions with everyone and then chose two or three (maybe with slight tweaking) for the test. Is it too late for that?
I also have a fairly good conscience about that kind of exam; I've used them for the past couple of years. The students have to study a fair amount of stuff and don't wind up with any nasty surprises.
Whatever you do, don't write exams like the legal ethics one I took this morning that was fucking impossible to finish. I hate that shit. I'm a fast worker, and if I can barely finish, the professor's just being an asshole.
The nice thing about options is that the more there are, the less non-sucktastic each individual question can be. The bad thing is that you have to think of more questions.
Well, I was thinking A and B are fairly dry, "did you study" questions, one from the first half of the semester, one from the second. C is the clever, "can you think on your feet" question that Labs wants to ask, so he's already got that one ready.
On one, I had two essay sections: One in which students answered one of a few, clearly defined topical questions. Another in which students had to answer a single, very general and open-ended question -- like the "train" question, only one that was very clearly rooted in the lectures. They had to write both.
On the other, I had one, highly general essay question, written to invite a variety of interpretations and drawing on the reading and the lectures alike.
Just now, it looks like the former exam produced the better answers.
As an undergraduate, the best professor I had extracted good work from us by the following method.
1. Distribute review sheets on which you found, say, 5 essay questions. Three of those would appear on the exam, you would have to answer one.
2. BUT the exam would also feature a second essay section, in which again you would have a choice, but none of the essays had appeared on the review sheet.
Brutal, but elegant and effective. I still know that stuff cold.
The problem with A and B or C tests is that there will be some students who waste half their time with C, decide they've fucked it up, give up, move to the more straightforward questions, and then fuck those up because they've run out of time. Or, they spend fifteen minutes trying to decide what best to do.
Of course, these students might deserve to do poorly.
FYI, I use test banks-- distribute, say, 10 essay questions, I pick 5 for the test, they write on 2. At least on the essay part (2/3 of the exam) there can be surprises only for the dumb.
I TA'd a course in which we handed out 3 possible questions for the midterm ahead of time, of which 2 would appear on the midterm, and students would answer 1.
Many of the students who had tried to psychologize us and guess which two questions would appear were disappointed when I came to class with each question A, B, and C on note cards and asked for a volunteer to draw 2 cards at random. (The other TA did the same; it was the professor's idea.)
Or lazy. I've gone this route, which I find has the uncomfortable effect of making me feel righteous and smitey when an answer clearly reflects unfamiliarity with the question.
Speaking of exams, I am basically done grading for the semester. Wrote my final today; the kids take it tomorrow evening. And best of all--for the first time in my life--the T.A. is doing the grading! I could just die of pleasure.
'Course, now I'm procrastinating on entering the semester's worth of grades that I've scribbled down on random scraps of paper into the database. And on preparing for the meeting I'm supposed to run tomorrow in preparation for next semester's required freshman course, which I'm coordinating. And then when those things are done, I'll procrastinate on the article I should get working on, that's due on 1 April.
Speaking of exams, I am basically done grading for the semester. Wrote my final today; the kids take it tomorrow evening. And best of all--for the first time in my life--the T.A. is doing the grading! I could just die of pleasure.
'Course, now I'm procrastinating on entering the semester's worth of grades that I've scribbled down on random scraps of paper into the database. And on preparing for the meeting I'm supposed to run tomorrow in preparation for next semester's required freshman course, which I'm coordinating. And then when those things are done, I'll procrastinate on the article I should get working on, that's due on 1 April.
Test design - ugh! When I was a TA in a basic electrical engineering course the prof tended to "target" the best students in the class -- if they were strong in a particular area, he'd create complex problems that focused on that area to drive them to the wall. I didn't like the motivation, but sometimes the process was redeemed by less-prominent students coming out of left field with really elegant answers. A couple semesters of this and it was off to law school...
From a purely self-interested perspective, exams that demand creative answers are inevitably more difficult and time-consuming to grade. And, as you noted, they are also more sensitive to the biases (and mood/attention at the time of grading) of the professor.
However, questions that demand creative responses, and unexpected syntheses of the course material with other issues, may be the only way to test how deeply a student has considered the material, or attempted to use the material to think about other problems.
Perhaps the best solution is to avoid a false dichomoty between "creative questions" and "expected questions" by achieving a balance between the two. Keep the question reasonably grounded in the course material, so that nervous/temporarily-blanked students get some friction as they write, perhaps propelling them into creativity, but provide enough conceptual space for creativity to be used.
But, seriously, any exam, like any game, is going to include unfair factors that render the competition a little (or a lot) unfair. I wouldn't bang your head against the wall over this.
From a purely self-interested perspective, exams that demand creative answers are inevitably more difficult and time-consuming to grade. And, as you noted, they are also more sensitive to the biases (and mood/attention at the time of grading) of the professor.
It's not how long it takes to grade that's important: it's how long it feels like it takes to grade. I could spend four hours grading "easy to grade" exams but it would feel like an eternity. I could spend eight hours grading a series of intelligent responses to serious questions and it would feel like fifteen minutes. People become too fixated on the raw amount of time it takes to grade and forget that some time is better spent than others.
Which is why I don't complain when I teach literary journalism (a.k.a. how to write like John McPhee/Susan Orlean/Walt Harrington/Gay Talese/&c.) because even though that means I won't be able to establish the easy norms that get me into a grading rhythm, at least my mind won't be numbed with the 18th iteration of exactly what I fucking said in class the one time I had the balls to lecture straight.
I never liked the "Here are five questions. I'll put three on the test, and you can pick one" kind of test. I just took it as an invitation to study the three easiest questions and ignore the rest of the material (or study it just enough to take care of the multiple-choice/fill-in-the-blank/other section). My inner lazy doesn't need any encouragement, and I always felt a little bit let down when my professors gave me the opportunity to slack.
I think the creative, synthesize-and-apply exams are fine as long as you let the students know what to expect. Practice tests are great.
Scott is right about how much nicer it is to grade interesting answers. I think it may even be easier to grade mediocre answers to interesting questions, but I'm not sure...
With respect to handing out the questions first, it somewhat depends on the numbers. This term I gave them 11 questions, put 5 on the test, and had them answer 4. So there was only one free pass. As for practice tests, I hope and trust Josh meant handing out practice questions that'll be like the ones you give... this is nice in theory but when I teach a class for the first time I have enough trouble coming up with the questions I am going to ask, let alone ones I'm not going to. (Unless they get the questions in advance, in which case I feel OK asking more specific questions.)
But why give an exam at all? I've fallen into the habit of giving take-home exams when I give exams at all. That way I can try to force students to review the class as a whole without the worry that I'm just catching them on a bad day or whatever (they have a few days, at least, to mull the exam over).
It could be that I'm just bitter that my tendency to panic (very) easily led me to screw up most of the exam's I've ever written, but I'm sort of suspicious that it's a very good test of what students have learned.
So . . . is there some awesome pedagogical advantage that I'm missing here?
It's not how long it takes to grade that's important: it's how long it feels like it takes to grade. I could spend four hours grading "easy to grade" exams but it would feel like an eternity. I could spend eight hours grading a series of intelligent responses to serious questions and it would feel like fifteen minutes. People become too fixated on the raw amount of time it takes to grade and forget that some time is better spent than others.
Dr. B, I dispute your idea of "easy." Then again, I don't have kids...or responsibilities...or a life...so I'm more than happy to spend hours grading interesting answers to exam questions. I can imagine that people with kids, responsibilities and lives would prefer otherwise. Their, uh, loss.
Really? How can even the best student write as much, or as well, in two hours as they could if given time to produce a polished essay, perhaps with research?
Is it that much easier? Huh. I get my take-home exams sent to me by email. I just bang through 'em as they come in, and I so prefer reading on my computer screen to reading someone's handwriting. I sort of wondered if I was being lazy about it.
But anyway, why do you think that take-home exams are better pedagogically than sit-down exams? I'm curious if other people have better reasons than I do for my own preference.
Some of us find that being cramped for time gives rise to unwonted eloquence, as we seek to distract from the fact that we haven't addressed the question.
Sorry, I wasn't disputing that. Of course take-home exams are the best means to interesting reading--and since my students spend all quarter researching their articles and their final "exam" is the polished product, that's I do.
I meant that if I had to choose between two questions on an in-class exam, one of which would test general knowledge the other synthetic ability, I'd go with the latter every time.
Also: Either type of exam seems awfully quick and easy to grade, since I don't put comments on them. It's the comments on essays - especially multiple draft essays - that kill me.
Also: Either type of exam seems awfully quick and easy to grade, since I don't put comments on them. It's the comments on essays - especially multiple draft essays - that kill me.
Pedagogically, because of what you said: a lot of students have testing anxiety. Exams, I think, also tend to encourage cramming and memorization (I realize that very well-designed exams don't necessarily do this), whereas take-homes, essays, longer-term projects and the like encourage students to think and digest, allow for re-thinking and revision, and hopefully don't simply call on their stamina in the face of little sleep.
Of course, this all said, next semester I plan on having exams in two of my courses--the fact is that after several years of trying to assign longer projects in large lecture classes, MY stamina has reached an end, and so exams it will be.
I suppose one good case for exams is that, if halfway decent, they do hold students responsible for all the material in the course, whereas essays can allow them to get away with picking and choosing what they actually learn.
26, 28: No, no extra grading required. Sorry 'bout that. Just a sample question (or a good description of what you're expecting), so the students know what's coming. I've just had too many "gotcha" exams to let this thread go by without saying something. Not that I really even needed to. Everyone here seems to put a good amount of effort into their teaching, so I doubt it's a problem in your classes anyways.
I've always wondered how much thought my professors put into their exams and whether they wrote them with specific learning goals in mind. Too often it seemed like they didn't. Kudos to y'all.
44: Graduate students are different. If they don't do all the reading (god knows I didn't), no one cares; it'll just come back to bite them in the ass when they get the readers' remarks back for the articles they send out.
I always preferred in-class exams to take-home. No matter how hard I tried to prevent it take-home exams would become all-nighters (as did papers). Once I started a paper a couple of days early and ended up writing all night that night.
In class exams never took more than 1 hour or so for midterms, 3 hours for the final. Sure, there was preparation involved, but I got to sleep.
Well, my actual preference, all other things being equal, is for some kind of semester-long project, with at least one draft, culminating in a final paper due in the last week, before the exams even start.
I have a strong preference for exams over papers, as they limit the amount I can possibly procrastinate. What this preference should mean to professors is totally unknown to me. To explain the degree to which papers don't do this, let me note that I turned an undergraduate paper in a year and a half late, at which point the professor was living in Amsterdam, or maybe Utrecht. Consider this an apology to professors in general for taking advantage of their good nature.
This thread brings back fond memories of all those college classes where I read everything the night before the final, staggered into class with no sleep, and aced it. I prided myself on doing the least work necessary to get the A. No doubt my experience argues in favor of papers rather than exams.
By the way, I feel like I'm a lurker intruding on y'alls' turf. You're not all going to start snapping your fingers and jazz-stepping away from me now, are you?
Matt, I've been lurking here so long I'm damn near a Morlock.
Posted one but almost posted twice when y'all linked to "my morning" but chickened out...by which I mean "totally fuckin' freaked when I thought the fucker wanted to charge me with sexual harassment."
Pffft. My dog just died (no joke*) and I have four cats. That excuse won't fly with me. My cats bathe their privates like the lives that'll never spring from their loins depended on it. And I thank them for it. I love gorgonzola as much as the next guy...but not on cat ass ensconced on my pillow as I sail into the land of Nod.
*Seriously. Were typepad not down I'd link to it. Then cry some more. Thank you, typepad. Thank you.
Actually, I'm a cat person whose parents dabbled in dogs. But I love all pets: fish, ferrets, hamsters ("dear Benny Profane, you left us too soon"), dogs, cats...
...in short, I'm an animal lover. But you always love your first pet like your left arm. The rest are like the limbs you have left.
I generally don't like terribly creative exam questions, at least if the student hasn't had a chance to see them ahead of time. I don't think it's a terribly good gauge of anything except whether the student is good at thinking like the professor.
For the intro level philo classes, some combination of exams and papers seems to ensure that the students get both the breadth and depth of knowledge they're supposed to, since most of the point of ancient & medieval and modern is let you know who all the people are that wrote things.
Posted one but almost posted twice when y'all linked to "my morning" but chickened out...by which I mean "totally fuckin' freaked when I thought the fucker wanted to charge me with sexual harassment."
Of all the times I've read the word "fucker," that might be the first that it was used literally. Said fucker sounds like a real piece of work; one would think he'd just be grateful that he didn't get in trouble and leave it at that. Unless Governor Gropinator has decreed differently for the University of California system, I really don't think students have a right to fuck uninterruptedly in teachers' offices.
Dogs rule, by the way, although cats are growing on me -- but not cats from hell like bitchphd's cat. (Our dog and two cats get along fine, which would not happen if we have a BPhd-like cat.)
Exams are more difficult to cheat on. Take-homes are just as easy to procrastinate on so you aren't doing anyone any real favors by giving them. If a question requires the student synthesize enough material that it can't be answered in twenty minutes on a test it needs to be a paper given in the ordinary course of the class.
I like creative questions, except for the one I got in my Middle East survey class, which was 'If President Bush went to Iraq and got shot, what would happen in the next 1) few days 2) few weeks 3) few months?'
I cursed for about a solid week before I could even attempt to answer that question, since it was on a take-home exam. I have never hated anyone as much as I did when I read that question. Don't give questions like that.
It occurs to me that I only weigh in on questions about teaching here and only from the perspective of a student.
I always hated take-home exams. It can be hard to identify the line between good work and wasted time, and any classmates with easier schedules end up with an advantage more pronounced than they would have in an exam. Also, take-homes are too easy to dismiss as innately easy -- and without the anxiety, tests just feel like paperwork.
Or maybe I just disliked them because I can crank out prose pretty quickly, and take-homes removed that advantage.
This was in 2002, so if they had it would all be old news by now.
And the thing that irritates me about take-home exams or essays is when they don't allow you to use sources not included in the reading for the class. I can understand not letting people use Wikipedia for references but if I have legitimate sources that are easily verified it is silly not to let me use them. I suppose that makes more work for the professor that they'd like to forego.
Or maybe I just disliked them because I can crank out prose pretty quickly, and take-homes removed that advantage.
I'm the same way. I have some sort of allerigic reaction to proof-reading and revising, so a take home exam (or paper, for that matter) usually is about the same quality as an in-class essay for me. I like benefitting from the lower expectations that accompany in-class assignments. Not that the professors here should necessarily be encouraging this behavior.
Also, w/d, how on earth did you manage to turn in a paper a year and a half late? The closest I've come to that is retroactively adding a course it turned out I never officially signed up for, a year after the fact. I would think most professors would either laugh or look at you quizzically if you let that much time go by with an assignment.
I'm not that surprised at w/d's story. Fulfilling requirements at the last minute is one of the great traditions of a liberal education. My roommate Charles has a great story along these lines: his college had a PhysEd requirement, and a week or two prior to graduation he discovered that the school had lost track of the fact that he took a semester of golf. Without it, he couldn't graduate.
Fortunately, a deal was worked out. Whether or not Charles got his diploma on time came down to him and the track coach, I believe, out on the football field, seeing whether Charles could chip a golf ball over the field goal posts (he succeeded).
I can top w/d's story actually. In law school, I was taking a seminar for which the only requirement was a paper to be handed in at the end of the semester. I didn't have it done on time (of course), so I took an incomplete. I was grotesquely late handing it in. The prof got confused somehow and actually gave me an A (an "E," to be precise) before I'd done so. (I did at least finish it and hand it in, rather than taking advantage of the prof's confusedness and just not doing it.)
Oh, college. I, also, turned in a paper about a year and a half late. I had this professor (who, I loved, actually, and the one who convinced me to go to law school) who, on the first day said something like "you have to write two long papers this semester." The first question, of course, was "when?" and his answer was "whenever." As the end of the semester neared, and people hasn't still written their papers, he said that he would give people incompletes if they couldn't get them in on time. I can barely function without deadlines (and once something's already late it's even harder to do), so of course I did nothing. I finally wrote the damn papers a week before law school started, and they retroactively altered my transcript to show I'd graduated a week before that (at the end of the summer semester). So, yeah, you profs here, give your students deadlines, for god's sake. Not that I'm not a jackass for not being able to function without them, but God, it sucks.
So I called my professor and asked him for for an incomplete -- the boon and bane of every collegiate procrastinator. He agreed. I went off to study in Mexico, figuring I'd finish the paper when I got back.
eb -- nice, and thanks. I spent a little while trying to bring my joke to fruition but eventually gave up and figured I would throw it out there for the rest of you to experiment with.
And my cat is not "from hell." She's an absolute sweetheart, and is purring on my lap as we speak.
I stand corrected, B. I guess she's a sweetheart to you, but she only tolerates PK, and goes nuts if you try to bring any other pet into the house. Hope that mice thing works out.
Re incompletes: about 10 years ago, there was a partner at Sonnenschein (et al.), a major law firm here in Chicago, who got busted. The guy had one uncompleted course at the University of Michigan Law School, and so hadn't actually graduated, nor been admitted to the bar. He nonetheless practiced law for 10 years (quite capably by all accounts) before someone finally figured out that he wasn't actually a lawyer. That incomplete cost him dearly . . . .
In case it wasn't clear, my "cat from hell" description derived from this account:
Vicious cat--let's call her Daisy--will not tolerate dogs. Nor other cats. Once we had a stray kitten that hung around for a few days. Our cat would go outside and beat the crap out of him, but he kept coming back. So, okay, we took him in. We locked him in the study for ten days. The entire time, Daisy sat outside the door, growling, reaching her clawed paws under the door every time she heard a noise. A few times we brought the stray out, and Mr. B. would hold him while I held Daisy; once she scratched the crap outta me, the other times she swore words that were not fit for a kitten's ears (nor even mine).
So finally, we figured, this isn't going to work, and we took stray kitten to the no-kill shelter. We left the study door open while we were gone.
When we got back, Daisy had trashed the study. Books and papers were knocked off the desk; the computer keyboard hung by its cord; books were pulled from lower shelves; pictures were knocked off walls. Daisy is a tiny cat, but she is fierce, and she will tolerate no competition.
In case it wasn't clear, my "cat from hell" description derived from this account:
Vicious cat--let's call her Daisy--will not tolerate dogs. Nor other cats. Once we had a stray kitten that hung around for a few days. Our cat would go outside and beat the crap out of him, but he kept coming back. So, okay, we took him in. We locked him in the study for ten days. The entire time, Daisy sat outside the door, growling, reaching her clawed paws under the door every time she heard a noise. A few times we brought the stray out, and Mr. B. would hold him while I held Daisy; once she scratched the crap outta me, the other times she swore words that were not fit for a kitten's ears (nor even mine).
So finally, we figured, this isn't going to work, and we took stray kitten to the no-kill shelter. We left the study door open while we were gone.
When we got back, Daisy had trashed the study. Books and papers were knocked off the desk; the computer keyboard hung by its cord; books were pulled from lower shelves; pictures were knocked off walls. Daisy is a tiny cat, but she is fierce, and she will tolerate no competition.
I've usually preferred those classes where the prof talked about five likely questions with everyone and then chose two or three (maybe with slight tweaking) for the test. Is it too late for that?
Posted by Jackmormon | Link to this comment | 12-15-05 3:18 PM
I also have a fairly good conscience about that kind of exam; I've used them for the past couple of years. The students have to study a fair amount of stuff and don't wind up with any nasty surprises.
Posted by Matt Weiner | Link to this comment | 12-15-05 3:21 PM
Whatever you do, don't write exams like the legal ethics one I took this morning that was fucking impossible to finish. I hate that shit. I'm a fast worker, and if I can barely finish, the professor's just being an asshole.
Not that I'm bitter or anything.
Posted by silvana | Link to this comment | 12-15-05 3:31 PM
How about presenting options? "Either answer A and B, or answer C."
Posted by bitchphd | Link to this comment | 12-15-05 3:32 PM
The nice thing about options is that the more there are, the less non-sucktastic each individual question can be. The bad thing is that you have to think of more questions.
Posted by Matt Weiner | Link to this comment | 12-15-05 3:33 PM
Well, I was thinking A and B are fairly dry, "did you study" questions, one from the first half of the semester, one from the second. C is the clever, "can you think on your feet" question that Labs wants to ask, so he's already got that one ready.
Posted by bitchphd | Link to this comment | 12-15-05 3:37 PM
I am now grading two exams.
On one, I had two essay sections: One in which students answered one of a few, clearly defined topical questions. Another in which students had to answer a single, very general and open-ended question -- like the "train" question, only one that was very clearly rooted in the lectures. They had to write both.
On the other, I had one, highly general essay question, written to invite a variety of interpretations and drawing on the reading and the lectures alike.
Just now, it looks like the former exam produced the better answers.
As an undergraduate, the best professor I had extracted good work from us by the following method.
1. Distribute review sheets on which you found, say, 5 essay questions. Three of those would appear on the exam, you would have to answer one.
2. BUT the exam would also feature a second essay section, in which again you would have a choice, but none of the essays had appeared on the review sheet.
Brutal, but elegant and effective. I still know that stuff cold.
Posted by slolernr | Link to this comment | 12-15-05 3:39 PM
The problem with A and B or C tests is that there will be some students who waste half their time with C, decide they've fucked it up, give up, move to the more straightforward questions, and then fuck those up because they've run out of time. Or, they spend fifteen minutes trying to decide what best to do.
Of course, these students might deserve to do poorly.
Posted by Jackmormon | Link to this comment | 12-15-05 3:40 PM
these students might deserve to do poorly
This raises for me a perennial, nagging question: are exams actually any good at achieving a just distribution of good?
Posted by slolernr | Link to this comment | 12-15-05 3:41 PM
Of course, these students might deserve to do poorly.
These students are me.
Posted by Sam K | Link to this comment | 12-15-05 3:44 PM
Whether or not they deserve to do poorly, half-answered questions are WAY easy to grade.
Posted by bitchphd | Link to this comment | 12-15-05 3:46 PM
half-answered questions are WAY easy to grade.
Yeah, but fully and poorly answered questions are WAY harder to mark than well answered question.
Posted by slolernr | Link to this comment | 12-15-05 3:48 PM
FYI, I use test banks-- distribute, say, 10 essay questions, I pick 5 for the test, they write on 2. At least on the essay part (2/3 of the exam) there can be surprises only for the dumb.
Posted by FL | Link to this comment | 12-15-05 3:48 PM
I TA'd a course in which we handed out 3 possible questions for the midterm ahead of time, of which 2 would appear on the midterm, and students would answer 1.
Many of the students who had tried to psychologize us and guess which two questions would appear were disappointed when I came to class with each question A, B, and C on note cards and asked for a volunteer to draw 2 cards at random. (The other TA did the same; it was the professor's idea.)
Posted by eb | Link to this comment | 12-15-05 3:49 PM
there can be surprises only for the dumb.
Or lazy. I've gone this route, which I find has the uncomfortable effect of making me feel righteous and smitey when an answer clearly reflects unfamiliarity with the question.
Posted by slolernr | Link to this comment | 12-15-05 3:50 PM
"Discuss the impact and signficance of trains on European development."
Like this?
Posted by eb | Link to this comment | 12-15-05 3:56 PM
Most of eb's article seems to be an excuse to use that opening anecdote, which is fantastic.
Posted by Jackmormon | Link to this comment | 12-15-05 4:00 PM
Speaking of exams, I am basically done grading for the semester. Wrote my final today; the kids take it tomorrow evening. And best of all--for the first time in my life--the T.A. is doing the grading! I could just die of pleasure.
'Course, now I'm procrastinating on entering the semester's worth of grades that I've scribbled down on random scraps of paper into the database. And on preparing for the meeting I'm supposed to run tomorrow in preparation for next semester's required freshman course, which I'm coordinating. And then when those things are done, I'll procrastinate on the article I should get working on, that's due on 1 April.
Sigh. No wonder no one wants to hire me.
Posted by bitchphd | Link to this comment | 12-15-05 4:06 PM
Speaking of exams, I am basically done grading for the semester. Wrote my final today; the kids take it tomorrow evening. And best of all--for the first time in my life--the T.A. is doing the grading! I could just die of pleasure.
'Course, now I'm procrastinating on entering the semester's worth of grades that I've scribbled down on random scraps of paper into the database. And on preparing for the meeting I'm supposed to run tomorrow in preparation for next semester's required freshman course, which I'm coordinating. And then when those things are done, I'll procrastinate on the article I should get working on, that's due on 1 April.
Sigh. No wonder no one wants to hire me.
Posted by bitchphd | Link to this comment | 12-15-05 4:08 PM
No wonder no one wants to hire me
This sounds to me much like the way hired people behave, i.e., scarcely a reason not to hire.
Posted by slolernr | Link to this comment | 12-15-05 4:10 PM
Oops.
Posted by bitchphd | Link to this comment | 12-15-05 4:10 PM
Test design - ugh! When I was a TA in a basic electrical engineering course the prof tended to "target" the best students in the class -- if they were strong in a particular area, he'd create complex problems that focused on that area to drive them to the wall. I didn't like the motivation, but sometimes the process was redeemed by less-prominent students coming out of left field with really elegant answers. A couple semesters of this and it was off to law school...
Posted by bill | Link to this comment | 12-15-05 4:34 PM
From a purely self-interested perspective, exams that demand creative answers are inevitably more difficult and time-consuming to grade. And, as you noted, they are also more sensitive to the biases (and mood/attention at the time of grading) of the professor.
However, questions that demand creative responses, and unexpected syntheses of the course material with other issues, may be the only way to test how deeply a student has considered the material, or attempted to use the material to think about other problems.
Perhaps the best solution is to avoid a false dichomoty between "creative questions" and "expected questions" by achieving a balance between the two. Keep the question reasonably grounded in the course material, so that nervous/temporarily-blanked students get some friction as they write, perhaps propelling them into creativity, but provide enough conceptual space for creativity to be used.
But, seriously, any exam, like any game, is going to include unfair factors that render the competition a little (or a lot) unfair. I wouldn't bang your head against the wall over this.
;) Just be creative.
Posted by Andrew | Link to this comment | 12-15-05 5:04 PM
From a purely self-interested perspective, exams that demand creative answers are inevitably more difficult and time-consuming to grade. And, as you noted, they are also more sensitive to the biases (and mood/attention at the time of grading) of the professor.
It's not how long it takes to grade that's important: it's how long it feels like it takes to grade. I could spend four hours grading "easy to grade" exams but it would feel like an eternity. I could spend eight hours grading a series of intelligent responses to serious questions and it would feel like fifteen minutes. People become too fixated on the raw amount of time it takes to grade and forget that some time is better spent than others.
Which is why I don't complain when I teach literary journalism (a.k.a. how to write like John McPhee/Susan Orlean/Walt Harrington/Gay Talese/&c.) because even though that means I won't be able to establish the easy norms that get me into a grading rhythm, at least my mind won't be numbed with the 18th iteration of exactly what I fucking said in class the one time I had the balls to lecture straight.
Posted by Scott Eric Kaufman | Link to this comment | 12-15-05 7:07 PM
A lurker's two cents:
I never liked the "Here are five questions. I'll put three on the test, and you can pick one" kind of test. I just took it as an invitation to study the three easiest questions and ignore the rest of the material (or study it just enough to take care of the multiple-choice/fill-in-the-blank/other section). My inner lazy doesn't need any encouragement, and I always felt a little bit let down when my professors gave me the opportunity to slack.
I think the creative, synthesize-and-apply exams are fine as long as you let the students know what to expect. Practice tests are great.
Posted by Josh | Link to this comment | 12-15-05 7:10 PM
Practice tests?? You mean *meaningless grading*? Are you on crack?
Posted by bitchphd | Link to this comment | 12-15-05 7:14 PM
practice test + peer review = normed incompetence.
Brilliant!
Posted by Scott Eric Kaufman | Link to this comment | 12-15-05 7:26 PM
Scott is right about how much nicer it is to grade interesting answers. I think it may even be easier to grade mediocre answers to interesting questions, but I'm not sure...
With respect to handing out the questions first, it somewhat depends on the numbers. This term I gave them 11 questions, put 5 on the test, and had them answer 4. So there was only one free pass. As for practice tests, I hope and trust Josh meant handing out practice questions that'll be like the ones you give... this is nice in theory but when I teach a class for the first time I have enough trouble coming up with the questions I am going to ask, let alone ones I'm not going to. (Unless they get the questions in advance, in which case I feel OK asking more specific questions.)
Posted by Matt Weiner | Link to this comment | 12-15-05 7:29 PM
But why give an exam at all? I've fallen into the habit of giving take-home exams when I give exams at all. That way I can try to force students to review the class as a whole without the worry that I'm just catching them on a bad day or whatever (they have a few days, at least, to mull the exam over).
It could be that I'm just bitter that my tendency to panic (very) easily led me to screw up most of the exam's I've ever written, but I'm sort of suspicious that it's a very good test of what students have learned.
So . . . is there some awesome pedagogical advantage that I'm missing here?
Posted by Chris | Link to this comment | 12-15-05 7:44 PM
Pedagogically, essays or take-homes are better than exams. But exams are *way* easier to grade, especially if you have a big class.
Posted by bitchphd | Link to this comment | 12-15-05 7:46 PM
Pedagogically, essays or take-homes are better than exams. But exams are *way* easier to grade, especially if you have a big class.
Posted by bitchphd | Link to this comment | 12-15-05 7:46 PM
It's not how long it takes to grade that's important: it's how long it feels like it takes to grade. I could spend four hours grading "easy to grade" exams but it would feel like an eternity. I could spend eight hours grading a series of intelligent responses to serious questions and it would feel like fifteen minutes. People become too fixated on the raw amount of time it takes to grade and forget that some time is better spent than others.
Great point.
Posted by Anonymous | Link to this comment | 12-15-05 7:49 PM
Dr. B, I dispute your idea of "easy." Then again, I don't have kids...or responsibilities...or a life...so I'm more than happy to spend hours grading interesting answers to exam questions. I can imagine that people with kids, responsibilities and lives would prefer otherwise. Their, uh, loss.
Posted by Scott Eric Kaufman | Link to this comment | 12-15-05 7:54 PM
Really? How can even the best student write as much, or as well, in two hours as they could if given time to produce a polished essay, perhaps with research?
Posted by bitchphd | Link to this comment | 12-15-05 8:02 PM
Is it that much easier? Huh. I get my take-home exams sent to me by email. I just bang through 'em as they come in, and I so prefer reading on my computer screen to reading someone's handwriting. I sort of wondered if I was being lazy about it.
But anyway, why do you think that take-home exams are better pedagogically than sit-down exams? I'm curious if other people have better reasons than I do for my own preference.
Posted by Chris | Link to this comment | 12-15-05 8:06 PM
Some of us find that being cramped for time gives rise to unwonted eloquence, as we seek to distract from the fact that we haven't addressed the question.
Posted by ben wolfson | Link to this comment | 12-15-05 8:07 PM
Sorry, 35 was to 30 (and 31!) in case that wasn't clear.
Posted by Chris | Link to this comment | 12-15-05 8:07 PM
Sorry, I wasn't disputing that. Of course take-home exams are the best means to interesting reading--and since my students spend all quarter researching their articles and their final "exam" is the polished product, that's I do.
I meant that if I had to choose between two questions on an in-class exam, one of which would test general knowledge the other synthetic ability, I'd go with the latter every time.
Posted by Scott Eric Kaufman | Link to this comment | 12-15-05 8:08 PM
Also: Either type of exam seems awfully quick and easy to grade, since I don't put comments on them. It's the comments on essays - especially multiple draft essays - that kill me.
Posted by Chris | Link to this comment | 12-15-05 8:09 PM
Also: Either type of exam seems awfully quick and easy to grade, since I don't put comments on them. It's the comments on essays - especially multiple draft essays - that kill me.
Posted by Chris | Link to this comment | 12-15-05 8:10 PM
38 in reply to 33. Man, you people respond panther-like.
Posted by Scott Eric Kaufman | Link to this comment | 12-15-05 8:11 PM
Pedagogically, because of what you said: a lot of students have testing anxiety. Exams, I think, also tend to encourage cramming and memorization (I realize that very well-designed exams don't necessarily do this), whereas take-homes, essays, longer-term projects and the like encourage students to think and digest, allow for re-thinking and revision, and hopefully don't simply call on their stamina in the face of little sleep.
Of course, this all said, next semester I plan on having exams in two of my courses--the fact is that after several years of trying to assign longer projects in large lecture classes, MY stamina has reached an end, and so exams it will be.
I suppose one good case for exams is that, if halfway decent, they do hold students responsible for all the material in the course, whereas essays can allow them to get away with picking and choosing what they actually learn.
Posted by bitchphd | Link to this comment | 12-15-05 8:13 PM
42 to 35.
Posted by bitchphd | Link to this comment | 12-15-05 8:14 PM
whereas essays can allow them to get away with picking and choosing what they actually learn.
Lord love 'em.
Posted by ben wolfson | Link to this comment | 12-15-05 8:14 PM
26, 28: No, no extra grading required. Sorry 'bout that. Just a sample question (or a good description of what you're expecting), so the students know what's coming. I've just had too many "gotcha" exams to let this thread go by without saying something. Not that I really even needed to. Everyone here seems to put a good amount of effort into their teaching, so I doubt it's a problem in your classes anyways.
I've always wondered how much thought my professors put into their exams and whether they wrote them with specific learning goals in mind. Too often it seemed like they didn't. Kudos to y'all.
Posted by Josh | Link to this comment | 12-15-05 8:17 PM
44: Graduate students are different. If they don't do all the reading (god knows I didn't), no one cares; it'll just come back to bite them in the ass when they get the readers' remarks back for the articles they send out.
Posted by bitchphd | Link to this comment | 12-15-05 8:25 PM
Shit.
Posted by ben wolfson | Link to this comment | 12-15-05 8:35 PM
Shit is right. I think I need a drink.
Posted by Chris | Link to this comment | 12-15-05 8:45 PM
I always preferred in-class exams to take-home. No matter how hard I tried to prevent it take-home exams would become all-nighters (as did papers). Once I started a paper a couple of days early and ended up writing all night that night.
In class exams never took more than 1 hour or so for midterms, 3 hours for the final. Sure, there was preparation involved, but I got to sleep.
Posted by eb | Link to this comment | 12-15-05 8:58 PM
Well, my actual preference, all other things being equal, is for some kind of semester-long project, with at least one draft, culminating in a final paper due in the last week, before the exams even start.
Yes, I am that mean.
Posted by bitchphd | Link to this comment | 12-15-05 9:07 PM
I have a strong preference for exams over papers, as they limit the amount I can possibly procrastinate. What this preference should mean to professors is totally unknown to me. To explain the degree to which papers don't do this, let me note that I turned an undergraduate paper in a year and a half late, at which point the professor was living in Amsterdam, or maybe Utrecht. Consider this an apology to professors in general for taking advantage of their good nature.
Posted by washerdreyer | Link to this comment | 12-15-05 9:30 PM
This thread brings back fond memories of all those college classes where I read everything the night before the final, staggered into class with no sleep, and aced it. I prided myself on doing the least work necessary to get the A. No doubt my experience argues in favor of papers rather than exams.
Posted by Frederick | Link to this comment | 12-15-05 10:55 PM
Frederick, I think you're a victim of some instructor's low expectations.
Posted by Scott Eric Kaufman | Link to this comment | 12-15-05 11:34 PM
By the way, I feel like I'm a lurker intruding on y'alls' turf. You're not all going to start snapping your fingers and jazz-stepping away from me now, are you?
Posted by Scott Eric Kaufman | Link to this comment | 12-15-05 11:38 PM
We like n00bz! Hang around until you're getting about half the in-jokes. Then seek help.
Posted by Matt Weiner | Link to this comment | 12-15-05 11:40 PM
Matt, I've been lurking here so long I'm damn near a Morlock.
Posted one but almost posted twice when y'all linked to "my morning" but chickened out...by which I mean "totally fuckin' freaked when I thought the fucker wanted to charge me with sexual harassment."
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 12:02 AM
Pffft. "Posted once but..." Figures I'd fuck that up.
"Welcome n00bz," u sez n ur hedz, "Ur nt 3133+ cuz u cant spellz!"
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 12:05 AM
Ah, who hasn't experienced the lurking fear?
Posted by eb | Link to this comment | 12-16-05 12:06 AM
Fuck me gently with a chainsaw! No way I parody Lovecraft with this many under my belt. This is some kind of sick initiation, isn't it?
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 12:09 AM
Just so you know, I'm not going into any basements.
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 12:10 AM
This is some kind of sick initiation, isn't it?
Nope, it's a fruit basket.
Posted by washerdreyer | Link to this comment | 12-16-05 12:36 AM
I refuse to click on all those links on principle.
And because dogs love balls indiscriminately. Who are you to judge them.
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 12:47 AM
A cat person, that's who.
Posted by ben wolfson | Link to this comment | 12-16-05 12:48 AM
Pffft. My dog just died (no joke*) and I have four cats. That excuse won't fly with me. My cats bathe their privates like the lives that'll never spring from their loins depended on it. And I thank them for it. I love gorgonzola as much as the next guy...but not on cat ass ensconced on my pillow as I sail into the land of Nod.
*Seriously. Were typepad not down I'd link to it. Then cry some more. Thank you, typepad. Thank you.
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 12:55 AM
Yeah, I read the post. What I'm hearing here is that you're a dog person who dabbles in cat.
Posted by ben wolfson | Link to this comment | 12-16-05 1:01 AM
That came off a bit insensitive, didn't it? Sorry.
Posted by ben wolfson | Link to this comment | 12-16-05 1:03 AM
Actually, I'm a cat person whose parents dabbled in dogs. But I love all pets: fish, ferrets, hamsters ("dear Benny Profane, you left us too soon"), dogs, cats...
...in short, I'm an animal lover. But you always love your first pet like your left arm. The rest are like the limbs you have left.
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 1:39 AM
I. Am trying. To break. Your heart.
(But honestly, not all doped up and the like.)
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 1:42 AM
No way Wilco kills this thread. I'm just sayin'...
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 1:58 AM
But I love all pets: fish, ferrets, hamsters ("dear Benny Profane, you left us too soon"), dogs, cats...
...in short, I'm an animal lover.
There's a fine line between animal lover and animal hoarder.
Posted by eb | Link to this comment | 12-16-05 2:00 AM
I've taught that essay since 2002. You're preaching to the converted.
Posted by Scott Eric Kaufman | Link to this comment | 12-16-05 2:19 AM
No possum hoarders were mentioned in the article. (Beaver hoarders, yes). There's an opportunity there for an enterprising person.
Posted by John Emerson | Link to this comment | 12-16-05 6:27 AM
I generally don't like terribly creative exam questions, at least if the student hasn't had a chance to see them ahead of time. I don't think it's a terribly good gauge of anything except whether the student is good at thinking like the professor.
For the intro level philo classes, some combination of exams and papers seems to ensure that the students get both the breadth and depth of knowledge they're supposed to, since most of the point of ancient & medieval and modern is let you know who all the people are that wrote things.
Exams are more difficult to cheat on.
Posted by Cala | Link to this comment | 12-16-05 6:38 AM
Posted one but almost posted twice when y'all linked to "my morning" but chickened out...by which I mean "totally fuckin' freaked when I thought the fucker wanted to charge me with sexual harassment."
Of all the times I've read the word "fucker," that might be the first that it was used literally. Said fucker sounds like a real piece of work; one would think he'd just be grateful that he didn't get in trouble and leave it at that. Unless Governor Gropinator has decreed differently for the University of California system, I really don't think students have a right to fuck uninterruptedly in teachers' offices.
Dogs rule, by the way, although cats are growing on me -- but not cats from hell like bitchphd's cat. (Our dog and two cats get along fine, which would not happen if we have a BPhd-like cat.)
Posted by Frederick | Link to this comment | 12-16-05 6:54 AM
Exams are more difficult to cheat on. Take-homes are just as easy to procrastinate on so you aren't doing anyone any real favors by giving them. If a question requires the student synthesize enough material that it can't be answered in twenty minutes on a test it needs to be a paper given in the ordinary course of the class.
I like creative questions, except for the one I got in my Middle East survey class, which was 'If President Bush went to Iraq and got shot, what would happen in the next 1) few days 2) few weeks 3) few months?'
I cursed for about a solid week before I could even attempt to answer that question, since it was on a take-home exam. I have never hated anyone as much as I did when I read that question. Don't give questions like that.
It occurs to me that I only weigh in on questions about teaching here and only from the perspective of a student.
Posted by winna | Link to this comment | 12-16-05 7:04 AM
I wonder if your professor will be getting a visit from the Secret Service.
Posted by bza | Link to this comment | 12-16-05 7:18 AM
Totally off-topic, but you all should be checking out this LGM thread. Two comments long, and already two great sex- and teaching-related anecdotes.
Posted by bza | Link to this comment | 12-16-05 7:23 AM
I always hated take-home exams. It can be hard to identify the line between good work and wasted time, and any classmates with easier schedules end up with an advantage more pronounced than they would have in an exam. Also, take-homes are too easy to dismiss as innately easy -- and without the anxiety, tests just feel like paperwork.
Or maybe I just disliked them because I can crank out prose pretty quickly, and take-homes removed that advantage.
Posted by tom | Link to this comment | 12-16-05 7:35 AM
This was in 2002, so if they had it would all be old news by now.
And the thing that irritates me about take-home exams or essays is when they don't allow you to use sources not included in the reading for the class. I can understand not letting people use Wikipedia for references but if I have legitimate sources that are easily verified it is silly not to let me use them. I suppose that makes more work for the professor that they'd like to forego.
Posted by winna | Link to this comment | 12-16-05 7:42 AM
Or maybe I just disliked them because I can crank out prose pretty quickly, and take-homes removed that advantage.
I'm the same way. I have some sort of allerigic reaction to proof-reading and revising, so a take home exam (or paper, for that matter) usually is about the same quality as an in-class essay for me. I like benefitting from the lower expectations that accompany in-class assignments. Not that the professors here should necessarily be encouraging this behavior.
Posted by Matt F | Link to this comment | 12-16-05 8:03 AM
I like benefitting from the lower expectations that accompany in-class assignments.
One more for that bench. Although I've learned to rewrite and proof since I've left school, so it might be different if I went back now.
Posted by LizardBreath | Link to this comment | 12-16-05 8:05 AM
Also, w/d, how on earth did you manage to turn in a paper a year and a half late? The closest I've come to that is retroactively adding a course it turned out I never officially signed up for, a year after the fact. I would think most professors would either laugh or look at you quizzically if you let that much time go by with an assignment.
Posted by Matt F | Link to this comment | 12-16-05 8:07 AM
I'm not that surprised at w/d's story. Fulfilling requirements at the last minute is one of the great traditions of a liberal education. My roommate Charles has a great story along these lines: his college had a PhysEd requirement, and a week or two prior to graduation he discovered that the school had lost track of the fact that he took a semester of golf. Without it, he couldn't graduate.
Fortunately, a deal was worked out. Whether or not Charles got his diploma on time came down to him and the track coach, I believe, out on the football field, seeing whether Charles could chip a golf ball over the field goal posts (he succeeded).
Posted by tom | Link to this comment | 12-16-05 8:16 AM
I can top w/d's story actually. In law school, I was taking a seminar for which the only requirement was a paper to be handed in at the end of the semester. I didn't have it done on time (of course), so I took an incomplete. I was grotesquely late handing it in. The prof got confused somehow and actually gave me an A (an "E," to be precise) before I'd done so. (I did at least finish it and hand it in, rather than taking advantage of the prof's confusedness and just not doing it.)
Posted by Frederick | Link to this comment | 12-16-05 8:47 AM
I'm glad to see I'm not the only one who has lawsuit bullshit hanging over her head.
And my cat is not "from hell." She's an absolute sweetheart, and is purring on my lap as we speak.
Posted by bitchphd | Link to this comment | 12-16-05 8:52 AM
Oh, college. I, also, turned in a paper about a year and a half late. I had this professor (who, I loved, actually, and the one who convinced me to go to law school) who, on the first day said something like "you have to write two long papers this semester." The first question, of course, was "when?" and his answer was "whenever." As the end of the semester neared, and people hasn't still written their papers, he said that he would give people incompletes if they couldn't get them in on time. I can barely function without deadlines (and once something's already late it's even harder to do), so of course I did nothing. I finally wrote the damn papers a week before law school started, and they retroactively altered my transcript to show I'd graduated a week before that (at the end of the summer semester). So, yeah, you profs here, give your students deadlines, for god's sake. Not that I'm not a jackass for not being able to function without them, but God, it sucks.
Posted by silvana | Link to this comment | 12-16-05 9:24 AM
Posted by eb | Link to this comment | 12-16-05 11:10 AM
Jeez it's like you guys are living in the dark ages. Essay questions? Come on! The future belongs to SCAN-tron.
Please remember to bring 2 #2 pencils and an eraser.
Posted by Jeremy Osner | Link to this comment | 12-16-05 11:26 AM
Which essay best answers the question?
(a) I
(b) II
(c) III
(d) None of the above
(e) The one I just squeezed into the bubble
Posted by eb | Link to this comment | 12-16-05 11:32 AM
eb -- nice, and thanks. I spent a little while trying to bring my joke to fruition but eventually gave up and figured I would throw it out there for the rest of you to experiment with.
Posted by Jeremy Osner | Link to this comment | 12-16-05 11:38 AM
And my cat is not "from hell." She's an absolute sweetheart, and is purring on my lap as we speak.
I stand corrected, B. I guess she's a sweetheart to you, but she only tolerates PK, and goes nuts if you try to bring any other pet into the house. Hope that mice thing works out.
Re incompletes: about 10 years ago, there was a partner at Sonnenschein (et al.), a major law firm here in Chicago, who got busted. The guy had one uncompleted course at the University of Michigan Law School, and so hadn't actually graduated, nor been admitted to the bar. He nonetheless practiced law for 10 years (quite capably by all accounts) before someone finally figured out that he wasn't actually a lawyer. That incomplete cost him dearly . . . .
Posted by Frederick | Link to this comment | 12-17-05 5:24 PM
In case it wasn't clear, my "cat from hell" description derived from this account:
Vicious cat--let's call her Daisy--will not tolerate dogs. Nor other cats. Once we had a stray kitten that hung around for a few days. Our cat would go outside and beat the crap out of him, but he kept coming back. So, okay, we took him in. We locked him in the study for ten days. The entire time, Daisy sat outside the door, growling, reaching her clawed paws under the door every time she heard a noise. A few times we brought the stray out, and Mr. B. would hold him while I held Daisy; once she scratched the crap outta me, the other times she swore words that were not fit for a kitten's ears (nor even mine).
So finally, we figured, this isn't going to work, and we took stray kitten to the no-kill shelter. We left the study door open while we were gone.
When we got back, Daisy had trashed the study. Books and papers were knocked off the desk; the computer keyboard hung by its cord; books were pulled from lower shelves; pictures were knocked off walls. Daisy is a tiny cat, but she is fierce, and she will tolerate no competition.
Posted by Frederick | Link to this comment | 12-18-05 7:09 AM
In case it wasn't clear, my "cat from hell" description derived from this account:
Vicious cat--let's call her Daisy--will not tolerate dogs. Nor other cats. Once we had a stray kitten that hung around for a few days. Our cat would go outside and beat the crap out of him, but he kept coming back. So, okay, we took him in. We locked him in the study for ten days. The entire time, Daisy sat outside the door, growling, reaching her clawed paws under the door every time she heard a noise. A few times we brought the stray out, and Mr. B. would hold him while I held Daisy; once she scratched the crap outta me, the other times she swore words that were not fit for a kitten's ears (nor even mine).
So finally, we figured, this isn't going to work, and we took stray kitten to the no-kill shelter. We left the study door open while we were gone.
When we got back, Daisy had trashed the study. Books and papers were knocked off the desk; the computer keyboard hung by its cord; books were pulled from lower shelves; pictures were knocked off walls. Daisy is a tiny cat, but she is fierce, and she will tolerate no competition.
Posted by Frederick | Link to this comment | 12-18-05 7:10 AM
How'd I do that? Oops.
Posted by Frederick | Link to this comment | 12-18-05 7:15 AM