No doubt if this comes to pass, Silver will find a way to work it into his model.
Damn, you -- you stole my comment!
Well, his on-the-nosity seems to me partly a function of how lopsided the polls consistently were. Armed with just as much data, how well or with how much certainty could he have predicted the 2000 election?
Silver is limited by the data he has, and I don't know of any pollster who regularly samples the preferences of Supreme Court justices.
Yesterday I read that Pollster actually ended up a bit closer than 538, albeit with a later call (ie, Nate was righter earlier, but wronger later). I haven't read into it, but it suggests that Silver may prove to be more of a one-trick pony than it appears.
Not to badmouth him or his work at all - I was there everyday, just like the rest of you. And good on him for basically creating himself ex nihilo in short order.
He'll be one of an increasingly numerous tribe of precogs. The precogs will be managed by a media elite which will pursue the same "simplify then exaggerate" tactic that has worked so well in the past. Eventually, a minority report will leak out from a nonconsensus precog. This report will cause Tom Cruise to abandon his platforms and also to recognize that he'll never, ever, be one with Xenu.
On an unrelated note, DJ Ronaldo's Jaguar is up at palms out sounds.
5: Did you read me saying something like that in comments somewhere? Because I have been saying it, repeatedly, basically because I'm annoyed that no one who really knows what they're doing has yet posted a comparison of the accuracy of various projection systems. I have a spreadsheet that compares a couple of them (538, electoral-vote, and pollster) just on the basis of how far off their projection of the margin in each state was from the actual one, but surely someone should be doing a more rigorous review.
The reason I saw "something like that" is because if it was me, you're misremembering the "Nate was righter earlier, but wronger later." I actually said I hadn't looked into that issue, or other possible ways of comparing different systems.
7: It was as likely you as anyone - I can't picture the stylesheet for context, which argues for comments here rather than a post elsewhere.
I know a shorter way to say comment 2.
Bah. Silver's use of statistics suvked, adn he came to roughly the same result as the other meta-analyses. he kept adjusting his model along the way, which statisticians can't do. Most spectacularly, he found a four point drop in Obama's projected winning percentage after the Democratic convention because there wasn't much of an upward bump at that time. Then he retroactively erased that adjustment a few days later. Don't try this in a clinical drug study.
His most spectacular claim was that he was measuring the ultimate outcome at all times in the campaign. He wasn't. Most of the time, his results were a long distance from the ultimate outcome. Obviously they got closer at the end, but so did the competing sites.
Silver also thought that Obama woudl out-perform the results of the polls because of his superior get-out-the-vote effort. It didn't happen.
A much more accurate measure thorughout the campaign was Sam Wang at election.princeton.edu, using much simpler statistical metods. That site found that Obama's odds of winning were outside the margin of error throughout the entire campaign except for a few days around the Republican convention. They were right. Silver thought it was much closer, and he was wrong.
A
Silver thought it was much closer, and he was wrong.
How do you know what Obama's odds of winning were?
The woe which Becks discerns in the post is essentially the same thing I was complaining about in my post about Saiselgy's post about someone else's post about the media some time back. Read yer Moran—he hasn't got the final account or anything but his exposition of one of the ways one can be self-alienated is good and apropos.
How do you know what Obama's odds of winning were?
Since I live in the future, I am confident that Obama's odds of winning are 100%.
the election.princeton.edu historical chart is here. Silver doesn't have a comparable chart that I could find. I wonder why not?
How good is Nate Silver? Petey was extolling his virtues on the Yglesias blog for chrissakes. What more do you need to know?
Seriously, 538 was a fun blog to follow, but it's very unclear how much value-added there was in the various statistical adjustments. Unimaginative is right that Sam Wang's site was very good.
Not sure why Silver would affect election results more than, you know, the existence of polls.
lw,
Yeah. Yeah. You know, you could make a movie out of that.
But, ben - can you translate (I mean 'dumb down') 11 for me cause I'm hoping to learn one thing this week and it might as well be what you are saying.
My instinct is to give Wang points for things like his median electoral vote estimate bottoming out at a higher point (and above 270 for Obama) than Nate's (around 250 for Obama) did during the post-Republican National Convention McCain polling bounce. But I think that might be ex post reasoning.
Count me in the Sam Wang campr.
Not sure why Silver would affect election results more than, you know, the existence of polls.
Right. Polls, and prediction markets, and so on, all of which have a fairly good track record of predicting the outcome. The difference in Silver's case is his "secret sauce" of poll weightings and fudge factors, which gives him an aura of "genius."
Silver doesn't have a comparable chart that I could find.
That's interesting. If his methodology works, such a graph should show a fairly flat, smooth prediction curve which slowly converges on the correct outcome. What does his real curve look like?
Since I live in the future, I am confident that Obama's odds of winning are 100%.
That's hilarious and all, but also not an answer. If I say the odds of a one-time thing happening are 2:1 and you say they're 3:1, and it happens, how are you going to adjudicate who was right? Your claim is that because Wang had Obama's victory being more likely, and Obama won, Wang's odds were more correct. Huh?
My instinct is to give Wang points for things like his median electoral vote estimate bottoming out at a higher point ...
The results aren't really comparable: Wang's estimate is always "if the election was held today" whereas Silver's is supposed to be a projection of the actual outcome. I.e., one should expect fairly large swings in Wang's numbers, whereas Silver's should be relatively stable (e.g., the convention bounce is supposedly already accounted for).
The difference in Silver's case is his "secret sauce" of poll weightings and fudge factors, which gives him an aura of "genius."
Don't agree with this part. I think the difference was in the well-written and well-reasoned daily updates explaining what that day's polling actually meant and which polls might be off in which ways for what reasons.
I think the difference was in the well-written and well-reasoned daily updates explaining what that day's polling actually meant and which polls might be off in which ways for what reasons.
I agree with this: his explanations were good. But he was basically doing the same thing that Ruy Texeira did in 2004, without the wishful thinking.
The difference in Silver's case...
What I meant here was the difference between Silver and election markets as a conventional-wisdom-distorting oracle...
9: I would've had to say the same thing 1 said, then say "pwned" in a second comment, right? How is that shorter?
16 - That's not actually true for something like an election where people's preferences change with time. Real events (the economic crash for example) changes the underlying distribution of voter preference, so adding additional poll information won't simply converge to some predetermined "true" election result.
It's also misleading to say Nate's model is better or worse given that someone else more closely predicted the true results. Nate's model creates a full probability distribution for the election, and reports the most likely outcome. Given the width of the distribution, the likelihood of the most-likely-outcome compared to all nearly as likely outcomes is actually rather small. Given the large number of people predicting election outcomes, you are guaranteed to find someone else whose predictions are closer to the final result, whether or not those predictions are based on real information, better modeling, or random chance.
Re complacency: weren't there e-mails going out from the Obama campaign that said (basically) "Our exit polls are closer than expected. We need people!!!"?
One imagines that an e-mail/leak like that would spur lots of people to non-complacency.
and reports the most likely outcome.
This is wrong, it reports the median outcome of the 10,000 simulations, that's why it wasn't a whole number. He also, towards the end, started reporting the single most likely outcome in his simulations, but that wasn't the number people were usually looking at.
That's not actually true for something like an election where people's preferences change with time.
If I understand correctly, Silver's model is intended to incorporate this kind of "preference evolution" by, for example, making corrections for convention bounces, assuming that races tighten in October, etc. Obviously, he can't predict events like the financial crisis, but he does claim to be making a high-probability prediction of the actual outcome (as opposed to polls or the Meta-Analysis, which only claim to be representing a snapshot of voter preferences).
any blog that allows a simpering little vichy-priest like Klopsko, or his palsie benji free rein should be de=blogged
stay the F offline, chester
Oops, mean outcome. The median would of course be a whole number.
The ToS is going into his final tailspin.
My guess is days, not weeks.
We'll find out about that, emerson, you cheap biblethumping fraud. even got yr churchie located now, perp. yr weblog galpals won't help ya
Unfogged-- the zionist version
he was basically doing the same thing that Ruy Texeira did in 2004, without the wishful thinking
That's not even especially true -- there was definitely a bit of wishful thinking on the 538 commentary. Remember the possibility that the devastating Obama ground game would make all the polling irrelevant? This was especially prominent when McCain was actually winning in the polls and Obama supporters were feeling demoralized.
I think ToS is refining his work -- concise, brilliant little nuggets where almost every word is some original form of abuse.
Statistics question: I have data consisting of the % of votes Obama got in every state, % of votes McCain got in every state, and what three different systems projected those percentages would be. Is there some agreed upon best way of assessing the accuracy of the systems? I don't think I've done it the proper way.
You are correct. You've done it wrong. Any further questions?
Okay, I've thought about the presenting question more. The number of people directly influenced by Nate Silver has got to be pretty small. What's more, they're political junkies. If his site produces complacency, then, it will be of a completely virtual kind -- i.e., his readers will begin to worry that other people are getting complacent and attempt to make up for that hypothetical complacency. The net effect will be less complacency, which he will, of course, incorporate into his system.
17: For what it's worth, I still fully stand behind this prediction that the odds of a McCain landslide were better than the odds of an Obama win.
the differnce is Wang didn't include any tracking polls or national polls, so during the conventions when no state polls were coming out, his numbers where basically just carryover. anyway, in 2004 he made assumption about 'undecideds break for the chajlenger' and got burned.
The main value of the Nate & sean site wasn't the "final prediction" - thats just one post and whatever. but for months it was giving pretty good updates that showed how one poll was affecting things overall. the 'right before election prediction' post might not have any extra value, but eh average post during this election was unusually good.
22: The usual term is "pre-pwnd", though it's not as common as "pwnd". So 38 would be "post-pwnd".
Anyone just wasting time who wants to troll the Republicans, go to the Coleman recount site here.
[Note: everyone here is by definition wasting time.]
What's with the stopping of the commenting all of a sudden here?
Statistics question: I have data consisting of the % of votes Obama got in every state, % of votes McCain got in every state, and what three different systems projected those percentages would be. Is there some agreed upon best way of assessing the accuracy of the systems? I don't think I've done it the proper way.
There isn't a single way of calculating the difference between two series of state results. Some metrics off the top of my head:
1) average absolute difference
2) median absolute difference
3) 90th-percentile absolute difference
For simplicity's sake, you may just want to focus on one candidate's vote share, unless the different systems had meaningfully different predictions of third-party vote shares.
43: People are busy wanking to Kim Jong Il before he becomes Kim Jong Dead.
So, for people who found Silver's results a cause for complacency, was it because a 3-in-100 chance that McCain would be president struck them as okay odds? Because man that was too much of a chance for my blood.
What I really like that Silver brought was (a) a reasonably clear and thorough explanation of his methodology, (b) a pretty good writing style, and (c) by far the most important, that histogram where he presented the results of all his model runs. The topline win percentage numbers I didn't really pay attention to, but assuming you bought his model, that histogram was gold.
Also, to the second question in Becks's post, even if predictions do become more influential, if they're also becoming more accurate then then they will accurately track their own influence on people's thinking, which should erase any complacency effects. If they aren't tracking people's complacency due to polls, they aren't accurate.
36: The number of people directly influenced by Nate Silver has got to be pretty small. What's more, they're political junkies. If his site produces complacency, then, it will be of a completely virtual kind
Yes, this.
48: But if the accuracy of a prediction and the influence thereof are related in any way, how could they account for influence and thus improve accuracy without further affecting influence?
||
I found this picture of Bush to be, uh, shocking.
|>
51: Damn, I don't know what that hand-symbol means.
50: I dunno, but if they couldn't Becks's question would be moot, because they wouldn't be increasingly accurate.
54: Ah. Now see, I couldn't think how to go about a search for a hand gesture. But I figured it was something like that, and there's a variant I am aware of.
it's not that though, cause the first and second finger are split in the Bush picture and together in the wikipedia version.
It means "seven" in ASL...
Yes! Though I hadn't thought of that.
Man, if kids are doing the Shocker all over the place all the time (I don't encounter them much), we really need to bring back some of the good ol' gestures: the Italian (I believe) gesture of slapping your hand into the crook of the opposite arm and quickly raising the forearm and fist of that arm. It means Up Yours, I've always thought.
Hrm. Looking for a link.
59: I MEANT THE VARIANT PARSIMON MENTIONED JEEZ.
39: As I said earlier, I agree that the posts were good about discussing what each days polling meant. But it's not clear to me how one even judges if he was doing a good job of assessing what each days polling meant.
What's the one where you flick your chin?
If you're counting on your fingers in binary, it's 22!
64: Looks roughly right. There doesn't seem to be an official name for this Italian elbow gesture besides "elbow gesture", means "up yours" or "fuck you", the French do it too, may be accompanied by the finger, may be vulgar ... or may be a gesture signalling exuberance! Just like The Shocker! But no pictures of this elbow gesture.
Chin flicking thing? Dunno; a sign of disrespect. It's difficult to look up gestures on the interwebs.
I think the Bras d'honneur is actually the gesture parsimon is talking about.
What the hell is going on in that picture?
About.com comes through with a demonstration of the gesture.
68: begin by holding your hand vertically underneath your chin, with your palm facing your wattle. Sweep your hand forwards in a flicking motion while smirking like an asshole. Finally, ignore decades of precedent in an ongoing attempt to make the United States a more heartless, kleptocratic place.
Wow, this is distracting! The Wikipedia entry on Gestures: Types of Gestures is fascinating. There's a listing for flicking your chin.
I'm seriously saddened that we don't use more of these things: biting one's hand? And I wonder what the Clinton Thumb is.
Oh, and yes, the About.com picture; bras d'honneur is what I was originally looking for.
You all just watch out if I interject a parenthetical biting of my hand in one of these comment threads, for "In Sicily, biting one's flat hand or fist is a powerful threat."
As predictions get more accurate, what are the impacts they might have on the electoral process?
Read Isaac Asimov's "Franchise" to find out.
75: see, now that's the kind of thing you're going to want to disclose before applying.
That you made your own Star Wars movie for $60,000?
77: that you're on Facebook.
I'd actually missed the Star Wars part. I suppose it's good he didn't build a death star and annihilate the whole city.
Sifu is right. The frequency distribution was the best thing on the site; it converged on the eventual result quite a long way out, too.
These graphs show that Missouri is by far the least racist state with a large black population, and has a much larger black population than any of the other less-racist states. I might have guessed.
Senator McCaskill came out for Obama very early. She deserves even more praise than she's been getting.
Goddamn those illegible charts. It's Maryland.
I still liek McCaskill, though.
Could an outcome be affected by people being complacent about a predicted win and not donating or volunteering?
Indeed it could.
34: With Silver's system, you could (if you had access to his simulations) calculate the likelihood his model assigned to the final result, and I think the same thing is true of Wang's. Just talking the average-in-some-sense difference between actual votes and predicted votes isn't really a good assessment of error, unless you can assume that state outcomes are (1) uncorrelated with each other, and (2) you should be able to make equally precise predictions about each state. I could be persuaded to believe (1), but not (2).
Actually, if you wouldn't mind sharing the data, this might make a good take-home problem for my data-mining class.
(I have not looked at how either Silver or Wang were doing their projections, so I have no opinion on their relative merits.)
10: he kept adjusting his model along the way, which statisticians can't do.
O sancta simplicitas!
51: maybe they were going for scout's honor.
Apparently the picture in 51 is supposed to be GWB with the ASU track team, making a gesture reminiscent of a trident. Like the one wielded by the mascot.
I don't buy it.
I enjoyed 538 during the election season... but I have my doubts now about the predictive capacity. Also, the model from PEC and 538 seemed to give awfully similar final distributions--I kept writing the proprietors to make bigger figures and only depict the regions with non-zero bin values. Alas and alack, they did not.
One thing about 538 was his concise but excellent reporting on field observations, which should be routine in campaign coverage but isn't.
You have dueling polls in this kind of thing. One group on non-participant polls, plus at least two participant polls. The R and D teams were going against 538 and the others, using their data to locate weak spots and areas of opportunity. The purpose of one group of polls was to predict the result, whereas the purpose of the second group was to figure out how to change the result.
