I thought he phoned it in with the last one, though (the article that used pit bulls as a lead-in to a discussion about the usefulness of stereotypes). It seemed like a throwaway that didn't make the cut for Blink.
I liked this article too, although I had problems with the one or two places where he seemed to be saying "public policy isn't the answer; adhering to the policy is." Obviously, it's both: there has to be a policy to adhere to (e.g., w/r/t police brutality issues). I'm a little wary of arguments that X imperfect solution needs to be completely replaced with Y solution; it seems to me that the more useful argument is "soup kitchens and shelters work for some people, but for the hardcore homeless we need to provide these other things instead (or as well)."
I don't know how revolutionary it is -- it's certainly a concept I've seen before. Even given that, thaough, it's fairly powerful to recognize that there's a real class of social policy problems where addressing the bulk of the population affected by the problem hasn't got anything to do with addressing the bulk of the problem, which is covered by a small fraction of the population.
The core anecdote is about a homeless guy who, among other things, ran up a million dollar medical bill in various emergency rooms over a couple of years. Think about it - if there are a thousand guys like that, that's a billion dollars in available health care savings if we can find some way to get them medically under control.
You see the same sort of distribution, for example in health care costs. Two percent of the population is responsible for 50% of the health care costs; I seem to remember that figure being thrown around ten years ago. The problem is that you don't know who falls into that 2% until that person starts his run.
What was interesting to me about that article was not the possible savings from better targeted services (e.g., hospital bills) but the way it might force people to re-think hard problems that seem to admit no answer. I represent poor people on housing court in new york, and the article's thesis is correct, in my view: there are a handful of people who have very hard problems, but the vast majority can get by with minimal social service/legal intervention. The thesis -- which is not really earthshattering -- could nonetheless mean alot for debates about "welfare reform" and the like: we're not looking at a hopeless problem, but actually a very solvable one for all but a handful of people. And for those, lets do what Denver does --- just hand them a minimal income and an apartment, no questions asked.
That latter is the really radical part, in the context of social welfare policy. The idea of welfare reform is that you have a mostly recalcitrant population who needs to be punished before they do what is right. In fact the situation could be very different: what you have is a more or less capable population who don't necessarily need punishment to 'behave' or solve their problems, and a small segment that will remain totally impervious to any punishment you mete out (no eviction, no cut in welfare benefits will make them 'see the light,' and you are still left at the end of the day with someone you need to feed and give medical care).
The idea of welfare reform is that you have a mostly recalcitrant population who needs to be punished before they do what is right.
Or, to cut welfare reform advocates some slack, that they need to be provided with strong enough negative incentives to straighten up and fly right. While that's an awfully superficially appealing idea, it really seems to be the case that for a small population that consumes a great deal of social services, that that isn't an effective approach.
Two percent of the population is responsible for 50% of the health care costs; I seem to remember that figure being thrown around ten years ago. The problem is that you don't know who falls into that 2% until that person starts his run.
Well, you know that they aren't a generally healthy person walking into an internists office with a little rash, and so that making it more difficult and expensive for people to get minor ailments looked at is unlikely to do a lot to reduce health care costs. Which indicates that all the HSA, higher deductible, make people more price-conscious consuers of healthcare, etc., ideas are pointless and doomed as a means of controlling costs.
To nitpick, I'm not sure I see the difference between punishing people to make them behave and giving them negative incentives so they'll straighten up and fly right. And what I see as appealing is not so much the idea of negative incentives as the idea that there might possibly be an approach that works.
Excellent point about the healthy person with the rash.
all the HSA, higher deductible, make people more price-conscious consumers of healthcare, etc., ideas are pointless and doomed as a means of controlling costsall the HSA, higher deductible, make people more price-conscious consumers of healthcare, etc., ideas are pointless and doomed as a means of controlling costs
Yeah, of course, because it's when you're dying that the money gets spent, at which point you're unlikely to comparison shop. Give me my triple bypass, now!!
Sorry. Kidding aside, that really was an interesting article. As I read it I kept thinking, "This makes a lot of sense." Then, unfortuately, I came to this part (which also made sense, somewhat*):
Power-law solutions have little appeal to the right, because they involve special treatment for people who do not deserve special treatment; and they have little appeal to the left, because their emphasis on efficiency over fairness suggests the cold number-crunching of Chicago-school cost-benefit analysis.
* Gladwell's explanation for why the right wouldn't buy into this approach seems spot-on to me, but what about his explanation for the left? Is "fairness" really the sticking point?
It must be difficult in some cases to tell the difference between the shaft and the blade though. I'm pretty sympathetic to LB's formulation of the welfare reform position from my albeit anecdotal observation of the younger generation of the welfare-dependent population in my Central Valley town pre-TANF. Little girls would actually say "I'm going to go on welfare when I grow up." So if we grant that negative incentives are good for a shaft, how do you provide a fail-safe for the blade without undoing the negative incentive in the first place? The homelessness "solution" is that shelters and intractable homelessness are so awful that no one who can avoid the situation will fail to, but welfare, getting government money to care for your kids in your home, is a lot more appealing, especially when working doesn't mean a huge jump, or any jump up, in quality of life.
I'm not so much saying that there is a difference, more that using the softer language acknowledges that not all advocates of the element in welfare reform that we're talking about are motivated by cold-hearted malice, as 'punishment' suggests. They may instead believe that the negative incentives will mend up making everyone's life esier and happier.
Despite that nod to the motives of welfare reform advocates, I still think, partially for the reasons laid out in the article, that they're wrong, and that consumers of social services are mostly composed of (1) a large class of people whose lives are quite bad enough already, thank you, and who don't need any more negative incentives to make their lives better -- they need fairly small amounts of practical help, rather than an attitude adjustment, and (2) a small class of people who are in so much trouble that negative incentives are irrelevant, and who need lots of help to keep from being an endless problem.
13: eh. I don't know if I can speak for the left, but it's not unappealing to me. If not addressing the root problems of the blade costs more money than addressing them, it's no more "unfair" to pour resources into their apartments and private nurses than it is into emergency rooms. Saving money means you have more money to spend on programs for the shaft. I'm pretty dubious that this is so much of an issue for "the left" in general, since I'd think one of the left's fundamental principles is that some people need more help than others, it's right that resources are expended on these few at the expense of the many, and this expense will probably ususally not turn out to be at the expense of the many after all, because these few, when not cared for, are a drain on the many in other ways.
Little girls would actually say "I'm going to go on welfare when I grow up." So if we grant that negative incentives are good for a shaft, how do you provide a fail-safe for the blade without undoing the negative incentive in the first place?
I don't know that I do grant that, I just wanted to tag it as not obviously insane or malicious. Wouldn't you think that positive incentives (free day-care if you're enrolled in college, heavily subsidized if you work... you know the sort of thing) are as likely to work for someone who's capable of functioning at all as negative incentives, and really haven't been tried in any solid way?
I think the positive incentives help, and I definitely support them, but I don't know that they would have been sufficient in and of themselves to convince people that were pretty comfortably ensconced in their homes that all of a sudden they should go out and work at McDonalds. I'm not saying they were living high on the hog, or they were welfare queens. But I lived in an apartment complex where rent was 250 dollars a month on a two bedroom apartment off a highway, and I had neighbors who were on AFDC (or something), and I interacted with them and helped some of their kids with their homework, and while they certainly didn't have much money, they spent time with their family and were probably happier than they would have been doing hard, stupid work (I hate doing hard stupid work, and if the government paid me to not be a secretary, I would take the money). I don't find it crazy to suppose that what some AFDC-recipients really wanted was to stay home and take care of their kids (that was the impression I got), and I pretty much sympathize with that, but if it's more expensive than subsidized child care, etc., I don't think it should happen at taxpayer expense. And even if it isn't, I think it built up a culture where kids didn't even have any professional ambition, reducing approximately to zero the chance that they would ever find out that there might be work available to them that wouldn't be stupid.
When Gladwell says "we can't do both" he seems to feel that social policy rationale will eventually do these little initiatives in. Gladwell always wants to be a provocateur, so you can be sure his "truth' needs to be, at least rhetorically, separate from and hostile to the "left" and "right," as he has constructed them. I don't buy it; I think these approaches are much closer to liberalism, and most of us here are delighted at the notion of being so rational and doing so much good. As has been observed above, the shelters and soup kitchens are also doing good, and even if the typical person spends only one night, I want them to be there for that night.
And he has lumped together problems with different moral dimensions as if they all equally illustrated his left-and-right-are-equally-obtuse thesis. The car emissions thing particularly bothered me. Do we really want to be equally punished because we have collectively contributed to polution? I'm a car guy, I'm the only person I know who is looking forward to watching the Daytona 500, and I always test my 21-year-old, 150,000 mi. Honda Civic "hot." Would I resent Stedman's vans taking over enforcement? Would you? Of course not.
I thought Gladwell missed the point: homelessness is not one category, it's (at least) two. Being two categories, it needs two responses.
His 'hockey stick' image is wrong. It's not a hockey stick, it's a bi-modal distribution. Whenever you get a bi-modal distribution you should be thinking 'we don't have one population here, we have two.'
Homelessness is a symptom. It's a result. It's like slurred speech: maybe a stroke, and maybe too many beers. Some homelessness results from a run of unfortunate events (job loss, etc.) and sometimes it's something else (long term severe mental problems).
The scary thing about Gladwell's analysis is that it justifies throwing out the remedies that do work in many cases because they don't work in all cases. This has been a very popular political argument: let's end the war on poverty, because it hasn't cured poverty.
I'm echoing what B and LB have already said. I know. But it's a very important point. And I think this place needs a chorus.
25: I thought that *was* Gladwell's point. What's the difference between a bimodal and a power law distribution, and why isn't homelessness a power law distribution?
I don't think his analysis justifies what you say it does. He isn't advocating getting rid of soup kitchens, or emmisions standards for car makers, he's advocating adding additional measures to take care of the top of the distribution.
I don't think his analysis justifies what you say it does. He isn't advocating getting rid of soup kitchens,...
From Gladwell, quotoing (approvingly, I think) Philip Mangano
Our intent is to take homeless policy from the old idea of funding programs that serve homeless people endlessly and invest in results that actually end homelessness
I don't have a net cite. It's page 102, bottom penultimate paragraph of column 1, in the print version.
I may be misreading, or it may be a deliberate obfuscation, but those are two problems. First, the endless stream of new, temporarily homeless people. Second, the endless homelessness of certain homeless individuals. Those two things should be kept quite distinct, and they are muddled in that quote.
I think you can play games with axes to turn a normal distribution with a long tail into a power law/hockey stick. If there are two populations, the relevant question is whether you can distinguish the two. The second concern is the one that Tia mentions and Gladwell gestures at - will your attempt to do so create incentives for people to move from one population to the other?
Not exactly on topic, but: I do budgets for government-subsidized housing and I notice that none of the Chinese tenants report their medical expenses to me (or, through me, to HUD), even though it would significantly reduce their rent. I thought it was a language or miscommunication issue, so I organized a meeting where a translator explained the issue--emphasizing that Chinese herbs, which they spend often spend hundreds of dollars on, would count. They're still all reporting 0 in medical expenses. I don't know if it's a cultural thing, not to take advantage of everything going, or what.
What's the difference between a bimodal and a power law distribution, and why isn't homelessness a power law distribution?
AFAIK, a bimodal distribution looks like a snake that swallowed two pigs. A power law distribution looks like a hyperbola, starts of rising slowly, then rises more rapidly, then rises even more rapidly - with the rate of increase steadily increasing towards infinite. The classic example is population increase. A hockey stick is straight, then turns a corner and again goes straight, but at a greater slope. The now classic example is, of course, global climate change.
here's Gladwell, page 98, third column:
... 80% of homeless were in and out really quickly ... The next ten percent were ... episodic users. They would come in for three weeks at a time, and return periodically ....the last ten percent ... were the chronically homeless, who lived in the shelters, sometimes for years at a time.
If we plot the amount of resources used on the horizontal, and number of individual cases on the vertical, we get the snake and pigs. Big, big lump at the left side (lots of cases, few resources used per case). Then the episodics. Then the second (smaller) pig, the cluster of cases that consume a lot of resources.
Okay, maybe it's a trimodal distribution.
I see that SCMT has already addressed the question. Fortunately I like being part of the chorus.
will your attempt to do so create incentives for people to move from one population to the other?
I think this worry is overstated -- the kind of help that, say, the guy in the article needed, entailing constant supervision, really isn't all that attractive to most ordinarily lazy people.
really isn't all that attractive to most ordinarily lazy people.
Probably not attractive for the people who need it, either. There are claims that a lot of the persistent homeless problem is explained by the release of a large number of people from mental institutions under Reagan. The number of people who think that they're the fuckups who need their lives run for them is, I'm guessing, pretty small. So we need some way to distinguish them from the population. That's hard, I think.
I've often been told I needed a keeper, but that's a different issue.
Talk of incentives presupposes rational actors. I don't really think it's the rational ones that are the problem. The rational ones are probably the ones who out and stable quickly.
Some people are crazily self-destructive. Hauling them to the ER doesn't change that. That's the lesson I drew from Gladwell's example of Murray Barr.
If the long term chronic homeless people are crazy, shouldn't we look at mental health services? We do sheltered living for the developmentally disabled, and for some chronically mentally ill. Putting a crazy person in an apartment won't help if they're too crazy to handle it.
Some also have long term medical issues - cirhossis, diabetes, etc. We have a medical system that's primarily oriented to emergencies. Hauling someone to the ER is a lousy way to deal with these problems, too.
I guess I'm saying that if we had a broader and deeper support system, rather than the patchwork of gaps and holes and expensive emergency interventions we have now, it might work better.
But SCMT is absolutely right, we need a diagnostic to distinguish among the population, to fit the solution to the problem.
Re. Tia's comment, this is a tough one. My sense, having also grown up in the valley, is that the "stupid work" problem is the real kicker: where the stigma against an embarrassing job is greater than the stigma against public assistance, you've got a real problem. Of course, one solution is the provision of non-lame jobs; but another, frankly, is making higher ed, vocational training, community college, etc. *possible* and accessible for folks who have no family background and for whom therefore the choices really do seem to be "stupid job" or "welfare."
Huh, this thread seems to have somewhat converged with the other one.
You mean into total bullshit? I don't mean bphd, but Michael and SCMT are swapping misinformation & misinterpretations and then patting each other on the back.
Two percent of the population is responsible for 50% of the health care costs; I seem to remember that figure being thrown around ten years ago.
OK, let's start by not pulling bogus statistics out of our asses. Save that for the mineshaft. I googled around for this data, and didn't turn it up. I did find people citing this sort of data (20/80 is the most common formulation on the web), but in the cases where they linked a source, that source had made it up a lot like SCMT.
I think you can play games with axes to turn a normal distribution with a long tail into a power law/hockey stick.
Mmm, no. Back to math class, Tim. May I call you Tim?
These are different distributions. They are statistically distinguishable. The log normal looks a lot like a power law, but even this is distinguishable with the right test and enough data.
What these distributions mean is open to interpretation, and in fact considerable misinterpretation. (skim down to where Shalizi begins 'There have been two problems'...)
It's not a hockey stick, it's a bi-modal distribution.
Michael, two populations don't necessarily imply a bimodal distribution. If you have one population with very low incident rates, as in the average cops in LAPD violence case, and one population with very high incidence rates, then you will only have one mode, which is the value that occurs most frequently in the data set. Because the low violence population does not account for many incidents for each person.
Gladwell's article is pretty confusing about what is being graphed, and it's pretty different stuff in the different examples of LAPD, homelessness, and polluting. Keep in mind that these 'hockey sticks' are generally frequency tables with specific individuals represented on the X axis - like you graphed a scatter plot of the height of all the people in your college class or office and then arranged them left to right from shortest to tallest.
I'm a little wary of arguments that X imperfect solution needs to be completely replaced with Y solution
Damn straight, b. In his quest for 'the middle way' Gladwell completely ignores that these are the distributions that result after liberal policy solutions have been applied. Would the emissions pattern look like this without regulation? I don't think so. Our hockey stick polluters might have been the extreme tail of a normal distribution before the policy was applied. We're just facing the reality that the extreme groups don't respond to the same treatments that those at the middle do.
I run into this power law stuff in some of the areas where I work. Some of it has real utility, but as Shalizi describes above a lot of it is really half-assed stuff from physicists and complexity theorists who are being really sloppy about distinguishing between the mechanisms that lead to the outcomes. It drives me kind of crazy - I had to listen to someone bang the table and go on about Pareto distributions and infinite variance just last weekend. I guess its pent up frustration that's led me to this Farberesque post.
If we plot the amount of resources used on the horizontal, and number of individual cases on the vertical, we get the snake and pigs. Big, big lump at the left side (lots of cases, few resources used per case). Then the episodics. Then the second (smaller) pig, the cluster of cases that consume a lot of resources.
Please, Michael, tell me this has something to do with FL's superkoranic fellatio powers. Please.
A hockey stick is straight, then turns a corner and again goes straight, but at a greater slope. The now classic example is, of course, global climate change.
No, no, no! That was a completely different issue. That 'hockey stick' is about the time trend in atmospheric temperature. If there is hockey stick in the temperature data then the rate of increase in temperature has accelerate significantly, most likely because of human carbon emissions.
The classic example is population increase. A hockey stick is straight, then turns a corner and again goes straight, but at a greater slope. The now classic example is, of course, global climate change.
Global climate change and population increases are not distributions, they are over-time trends.
What's the difference between a bimodal and a power law distribution, and why isn't homelessness a power law distribution?
A bimodal distribution has two peaks, or humps. (Insert joke here.) In the simplest cases one might just be two normal distributions observed together -- imagine you measured the average heights of 100 7th-graders and 100 10th graders, for instance, and plotted the distribution. It would be bimodal.
Loosely speaking, in the kind of distributions Gladwell is talking about (power law distributions), all the events/items of interest in the population are concentrated in a tiny number of observations, and the size/intensity/frequency of the events is related to the frequency of their occurence. (Insert Mineshaft joke here, at the Mineshaft.) So, 1 percent of homeless consume 50 percent of the homeless budget; 5 percent consume 75 percent; 10 percent consume 90 percent, etc. Often power law distributions are really just log-linear distributions, which are easier to understand but much less exciting-sounding.
Zipf's law is a nice example --- it says that the nth most common frequency will occur 1/n as often as the first. This is roughly true for things like city size, weblog readership, earthquake strength and so on. For unhappy academics, there's a version for productivity, too. Of the universe of people with PhDs, maybe 50 percent never publish a single article. (This is quite true.) Of the 50 percent who do publish at least one article, 50 percent of them only publish a single one. Of the 25 percent (of the total) who have published at least 2 articles, half only ever publish two, and so on down the line.
One other thought - a bimodal distribution is impossible in the sort of distribution chart Gladwell talks about because the individuals are ranked by frequency. Thus the two modes would be right next to each other at the left end of the chart and would just look like a thick peak.
OK. You have actual knowledge, and I bow in your general direction. FWIW, I believe the 2%-50% figure (if I'm remembering it correctly) was given to me by an economist at HHS; it's entirely possible I'm am misremembering the numbers, but the source had reason to know. I'll try comprehending Shalizi's bits, but there's a very good chance it's beyond my ken.
Sorry I was harsh, SCMT. The underlying work bugs me, and this discussion just dredged it up.
I do believe that health costs are skewed towards a small group of people every year. Who those people are and how they move into (and possibly/hopefully out of) that group will have as much to do with how we solve the problem as the distribution of people. If we stop with the distribution, the easy conclusion is - stop wasting money on these useless end of life treatments. But that may not be what's driving the data if we look at movement in & out of categories over time.
Gladwell's article is pretty confusing about what is being graphed, ...
Apparently I was well and truly confused. Thank you for the clarification. And yes, I did call population and climate change distributions, which they clearly aren't. Apologies. I was trying to illustrate shapes without drawing, just imagining shapes, and forgot what I was trying to illustrate.
Keep in mind that these 'hockey sticks' are generally frequency tables with specific individuals represented on the X axis - like you graphed a scatter plot of the height of all the people in your college class or office and then arranged them left to right from shortest to tallest.
Aha! Thank you. That's not at all what I thought Gladwell was talking about. I was picturing a data set where he had a list of individuals and the shelter stay for each. E.g. Fred, 1 day; Sam, 1 day. Murray, 287 days; Guillame, 2 days; Harry, 1 day; George, 2 days; Bill, 312 days. etc.
I imagined him making a bar graph (if that's the right term; histogram, mebbe?). Pile Fred, Sam and Harry on top of the spot on the axis for 1 day. Pile George and Guillame atop the 2 day spot. Then a lot of spots with nobody. Then Murray and Bill in the spot for 250 or more days days. That gave me the two hump distribution, which I sloppily called bimodal.
I get the idea of lining them up in order of duration of stay - Fred, Sam, Harry all showing one day, George and Guillame showing two days (the shaft of the hockey stick) and then Murray 287 and Bill 312 standing way proud and tall, the blade.
I'm repeating all this so that someone can tell me I'm still an idiot.
I tried Shalizi and it's beyond my grasp. However, I got the strong impression he was saying that people leap upon power law distributions when they're not necessary to explain the data.
That takes me back to Gladwell. He claims that Culhane discovered that homelessness has a power law distribution (or that the resource usage by the homeless has such a distribution - he does say "[homelessness] has a power law distribution"). My from-the-ass belief is that it is much more like the height distribution of 80 1st graders, 10 4th graders, and 10 sixth graders treated as a single population. Triple humped.
Is he probably right, or is this another patented Gladwell whiz-bang New Idea that unifies everything?
I think Michael S has a point here. If the facts on the ground are "there is a small population that accounts for a disproportionate amount of the costs", then why not just say that, and think about how to solve that problem, rather than bringing in all this waffle about power laws? For one thing, there are several distributions with big tails and it doesn't really matter whether the data is described by a power law or not, and for another, talking about power laws means that you stop thinking about whether we're talking about "the three drunkest people in Spokane cause 50% of the trouble" or "Jim, Bob and Fatty cause 50% of the trouble", when the two cases would clearly warrant different solutions.
I think these silver-bullet solutions always tend to disappoint (and indeed, that the small tail of them which work really well are probably a power law phenomenon themselves). For one thing, as cw notes, they are usually quite parasitic on a whole nother set of solutions keeping things in line for the majority population. For a second, they can create perverse incentive effects, and Gladwell just seems to mention this problem, say how serious it is, and then trail off into a discussion of how wonderful his policy which has this problem is (which is at the heart of what I find most annoying about Gladwell). And for a third, I suspect that these problems are mostly of the "three drunkest people in Spokane" variety rather than the "Jim, Bob and Fatty". You sort out the problems of the worst 1% and guess what? There is a whole new worst 1% and they are still causing a disproportionate amount of harm.
I think the actual solution to these "power law" problems is to learn to live with them. Murray's drinking wasn't a problem for Murray; he seemed quite happy with it. And if there is a $1m of pain out there, there is probably a case for having one guy with a $1m of problems rather than 200 guys with $5k of problems, particularly if there is diminishing marginal disutility as there very well might be.
You sort out the problems of the worst 1% and guess what? There is a whole new worst 1% and they are still causing a disproportionate amount of harm.
This is a v. important point, because half the attraction of the power-law thing is that the distributions are scale-invariant -- which is cool, but it cuts both ways.
If the facts on the ground are "there is a small population that accounts for a disproportionate amount of the costs", then why not just say that, and think about how to solve that problem, rather than bringing in all this waffle about power laws?
This is a good point -- I figure Gladwell was doing some handwaving to make the article sound excitingly technical.
And if there is a $1m of pain out there, there is probably a case for having one guy with a $1m of problems rather than 200 guys with $5k of problems, particularly if there is diminishing marginal disutility as there very well might be.
This, on the other hand, I don't get. If you spend, say, $50,000 on services for Murray to keep him indoors and as healthy as possible (even if he remains alcoholic, which presumably he would), that saves the remaining $950,000 of his emergency room bills. I don't see how it would increase the costs incurred by the next drunkest homeless guys out there; they seem unconnected.
On the subject of "perverse incentive effects" and 37, I'm just not convinced that positive incentives in the realm of the realistic would have done the trick by themselves, especially in the face of an entrenched culture (though maybe that's not true; perhaps a guaranteed annual income of 15K above what welfare gave them would have been enough). It takes a long time and a lot of investment to create a whole new economic structure in an area, and there's a limit to how many jobs you can create that are really more rewarding than spending the day with your kids or the rest of your family. In the meantime, by leaving the situation alone, the culture got even more entrenched. But then you're left with the possibility of cutting off assistance to people who really can't find work, either because they really are unemployable, or because there just isn't the work to be had, and hurting their children even more. Teh difficult.
We could always start another war, and just draft 'em. I had a high school teacher who thought that wars for empire were mostly about (a) giving excess and potentially troublesome males something to do, and (b) thining out the population that would return.
I did read once about a guy who had a $2 000 000 disease -- he had to be in a bubble for 6 months, IIRC -- and almost bankrupted his provider. As the story was told, it was a clearcut case: he recovered completely, but would have died without the 2 mil.
The demographic explanation of war makes more sense than people give it credit for. It doesn't limit population much, though, unless you put women on the front lines. The Mongols practiced polygamy and replenished their troops with captives.
I've long been of the opinion that a military which would all be either gay or female or both would be a good thing. My motives for this are bad ones, however.
Aren't a huge percentasge of healthcare costs tied up in end-of-life and near-end-of-life interventions? (Chemo, emergency surgery, etc.) The 1% is not just the folks who go the emergency room for a rash, but the people who are either dying or who will die without immediate, expensive treatment (e.g. heart valve replacement).
Well, exactly (note that this is also an area where patients are really unlikely to have the knowledge to be cost-sensitive in any kind of useful way.)
1. Per cw, the number and things realted to it are open to some doubt.
2. I believe the 2% is end of life primarily, but it's not straightforward. It is often something like someone goes into the hospital for a hip replacement, there are complications, and then more complication, as expense, death, and hilarity ensues. It's not clear how many months we are adding to that person's life, but (IIRC) not a lot.
3. IIRC, most people are not part of the 2%; most people die relatively peacefully.
4. My recollection is that there was a rough consensus among health economists that innovation drove costs, and that the only real limiter was de facto rationing. Not politically something one can openly say.
I think that there was a small immediate saving when ICU doctors stopped trying to squeeze as many days as possible out of dying patients. When I worked with patients I saw one patient twice a month apart, and she'd apparently been in ICU the whole time, with about 5 IVs, 2 catheters, and four-point restraints. She died a week or two later.
I think that the idea that there's a lot of unnecessary care is somewhat a myth, though, as Tim says or implies.
49: If the facts on the ground are "there is a small population that accounts for a disproportionate amount of the costs", then why not just say that, and think about how to solve that problem, rather than bringing in all this waffle about power laws?
Right. Thank you.
We're trying to understand what's going on so we can most efficiently change it for the better. Knowing how costs are distributed across the population can help - but only if it leads to better approaches.
I claim that understanding the underlying mechanisms that produce homelessness will help us improve the efficiency of our approach to homelessness. I have no evidence for this assertion. I have argument, which I sketch below.
Gladwell's invocation of the power-law doesn't help. I think it's another instance of what CW described in 39 as
...being really sloppy about distinguishing between the mechanisms that lead to the outcomes.
That is partly because characterizing it as a power-law distribution says, in effect, 'look over here, just ignore those cases behind the curtain.' As PW says in 41:
... [in (power law distributions] all the events/items of interest in the population are concentrated in a tiny number of observations ...
That's just what Gladwell was saying (p.101, col2):
The homelessness problem is like the LAPD's bad-cop problem. It's a matter of a few hard cases, and that's good news, because when a problem is that concentrated you can wrap your arms around it and think about solving it."
Treating is as a power-law distribution not only misdirects our attention away from the cases behind the curtain, it also leads one to expect the perverse incentive problem and the scale-invariant problem.
The alternative is to treat it as a two-hump (which I call bimodal) distribution. This makes sense to me because I think LB is correct in 16 in saying we've got two classes, two populations.
Treating it as bimodal leads one to look for approaches that work for humpA, and approaches for humpB, without being discarding humpA approaches because they don't work for humpB cases.
Treating it as bimodal leads to the question of how to tell whether a particular case is humpA or humpB. To look for the point where shaft becomes blade, to identify the distinguishing characteristics, so to speak. This helps avoid wasting money on humpA approaches to humpB cases, and vice versa.
Treating it as bimodal may also make the perverse incentive problem less worrisome. We're less likely to worry about a person trying to pass for humpB if we end up saying (for example) "yes, we'll give you a free apartment for you and your kids, as soon as you show that you are a dysfunctional psychotic or chronic alcoholic with untreated life threatening medical conditions."
Thus, I think Gladwell was doing more than mere technical handwaving. I think that defining a whole bunch of problems as power-law distribution problems is a way to confuse and befuddle people of ordinary intelligence (e.g. me), so we can say 'not our problem.' It's a way to say 'don't look at whether ordinary cops have a problem with using excessive force, it's just a few bad apples'; 'don't worry about paying for shelters and soup kitchens, it's only a few hard cases'; 'don't bother getting your car's emissions checked every year, it's not you it's them.' And, of course, it leads to a whole lot of rather irrelevant discussions of what constitute power-law distributions.
Purely personal opinion, but I wouldn't recommend that; though, to be sure, it wasn't remotely as bad as the later books; like many OMG-I'm-making-so-much-money contract extensions, the series, IMO, had a qualitive decline with each book (though I gave up reading any, other than taking glancing looks, after the above; Dune: excellent; Children of Dune: adequate; God-Emperor of Dune: getting bad; thereafter: please, God, no).
Purely, as I said, personal opinion; I'm great with other people adoring all the others, and everything Brian Herbert ever did/does, and so forth.
As regards Gladwell, I don't do the math, alas, but I would keep in mind that he's writing for a popular audience, and much of the terminology being tossed about in this thread is not stuff one should, IMO, be using for a piece in the New Yorker; not unless you're given a lot more space to explain it, and why it's important, at least. And I didn't read him as being remotely as dismissive of various points as seems to be what's being suggested at spots above. But I've certainly read the above discussion with great interest, and learned some things from it, even if I have nothing useful to contribute.
69: God knows I do not want to hijack this thread to get into an argument about science fiction, let alone with Gary of all people, but I'll just say I didn't think, for instance, Chapterhouse: Dune was all that bad.
I just read this week's New Yorker and New York Magazine on the plane. New York's article on blogs also prominently features the concept of power-law distributions, only as a justification for why it's so hard for new blogs to crack the "A List" and become the next Gawker or Kos. Interesting that both would have articles on that concept in the same week to such different ends.
I only skimmed the online version of the New York article, but Clay Shirky said a lot of the stuff I saw in it, about blog popularity and power laws, in early 2003.
I like Clay Shirky's idea of the long tail - that successful internet businesses are exploiting the fat tail of the power law. That is, ebay and Amazon make their money on all the weird little shit that nobody else could sell (like Amazon - no discount on academic books) not on the center of the distribution.
But that early stuff he did on powerlaws and blog popularity was stupid. The point of the power law is that there is much more action in the tails of the distribution (hence the long tail or fat tail). If blog attention was normally distributed it would be _harder_ for a new blog to break in because there would be fewer subcommunities to build an audience in & the traffic would be more focused on the popular sites. So this bit about power laws and popularity seems to get it backwards.
"New York's article on blogs also prominently features the concept of power-law distributions, only as a justification for why it's so hard for new blogs to crack the "A List"....
Only by coughing up Clay Shirkey's thesis from what is more or less the Jurassic Period of the blogging era, at this point. I mean, it's correct (I think), but everyone acknowledged that umpty years ago (okay, three), back when he wrote it, and it was a hot topic for a few weeks.
"Interesting that both would have articles on that concept in the same week to such different ends."
Isn't that what happens with hackeyned old ideas? It would be only slightly less surprising, I think, if someone in both magazines used the concepts of "evolution" and "mutation" to different ends. Though not if they're doing a quick hack job on the Exciting! New! Topic! of Blogs!
"Clay Shirky said a lot of the stuff I saw in it, about blog popularity and power laws, in early 2003."
And some proto-notions on panix.talk back in the Nineties. But I only first saw his writing in 1995, so I don't know what he was writing about before that.
A good blog on the long tail is Chris Anderson's The Long Tail. Somewhere (possibly on that site) I read a clever bit about how the long tail effects of the internet had rewritten Andy Warhol's prediction: it's not that everybody will be famous for fifteen minutes, but that everybody will be famous for fifteen people.
This rejoinder is entirely too late, but I wonder if some of the above discussion might miss what is useful about the Gladwell article. I respect the earnesty with which people familiar with the technical side of the argument approach the article, but let's not think that "homelessness policy" or "welfare policy" is about solving problems --- at least not always. These policies usually embody a set of moral judgments ("the kind of society we want to live in") rather than a set of actual solutions. We can go back and forth about the different methods of describing a problem and its distribution, but that's different from making policy. Even in health care (I'm shocked to hear) some economists argue that offering less coverage is "efficient" because most people can live with minor illnessess just fine --- giving more people coverage just means they are going to run off to the doctor for every little problem and run up costs. That sounds like a technical argument about efficiency, but even that is a moral argument about the sort of society that economist is willing to live in.
So, I think the Gladwell article says this: when it comes to homelessness, stop thinking about it as an unsolvable problem comprised of tens of thousands of people who cannot be absorbed into society --- an "underclass" --- and realize that the overwhelming majority of homeless are amenable to help, want it, and, if given it, will make the best use possible.
If you can use this to convince people to make a different moral judgment of the homeless or people on welfare, you can get to different, smarter policy. For example, over and over again the "underclass" thesis is disproved, or at least proven to describe a very small percentage of the population; recall that welfare reform was passed at a time when many believed every homeless person, every person on welfare, was part of an class of people totally disconnected from, well, the "mainstream." I think you can use the kind of argument Gladwell's making to make a distinction between the handful of hard problems and the great majority of less-hard problems solvable by targeted help.
And some proto-notions on panix.talk back in the Nineties. But I only first saw his writing in 1995, so I don't know what he was writing about before that.
These here intertrons existed before the run-up to the 2004 election?
After hearing Gladwell interviewed today on Morning Edition, I can officially declare that I find him despicable. I was trying towake up so my recollection is a bit dim and I haven't looked for it on npr.org, but I remember I few points.
He described Mangano, the Bush appointee who's so keen on calling this a power-law distribution problem "wonderful." I take that as support for my earlier assertion that Gladwell was quoting this guy approvingly.
The interview had to ask twice "but what about the single mom with three kids?" before getting an answer. I think finally Gladwell came down as saying 'I hope, really hope, that person isn't forgotten as we solve the homelssness problem, which is really about the hard cases.'
Bah, humbug.
This is the rebranding of homelessness. First, we develop a new image: it's no longer mom & kids, our poster child isnow to be Million Dollar Murray. Then, since homeless people are clearly unsympathetic chronic drunks, we cut 'em off.
They did this in reverse with AIDS (remember Ryan White? I hope that's the right name). They did this with AFDC (remember the caddie driving welfare queen?). Now they're going to do it with homelessness. Now, homelessness isn't going to mean the young family down on its luck, it's going to mean Murray. This really sucks.
Incidentally, I suggest that all you New Guy bloggers here at Unfogged better unload your vast financial stakes in Unfogged, Inc., because the blog market is about to crash.
Michael: "They did this in reverse with AIDS (remember Ryan White? I hope that's the right name)."
I'm not following you terribly well: it's a bad thing that there's been more and more money poured into AIDS research and treatment over the years?
"Then, since homeless people are clearly unsympathetic chronic drunks, we cut 'em off."
This would be a new argument someone would be making?
The point I was trying to make was that if you change the brand, the image, the face of a problem, you change the what's politic. With Ryan White, by changing it from queers to innocent kids you made funding more possible. It could have worked perfectly well in reverse.
By calling homelessness a single problem with a power-law distribution, rather than two (or more) problems, you can make Million Dollar Murray the brand, the face, the issue. That will allow everyone to say "homelessness is not a problem of ordinary people, not a problem of single moms with three kids" and that will eventually allow funding to be cut off.
What I heard Gladwell assume, without actually saying it, is that the problem with homelessness is that we're spending too much tax money. If the goal is to reduce how much tax money you spend, making it an issue of Murrays is a great approach.
"That will allow everyone to say 'homelessness is not a problem of ordinary people, not a problem of single moms with three kids' and that will eventually allow funding to be cut off."
I'm afraid that my perspective is that that's what they've always said since FDR's day ended (or at least since Nixon's began, if you prefer), and that there's never been much funding in the first place.
Just different perspectives, not different facts.
"What I heard Gladwell assume, without actually saying it, is that the problem with homelessness is that we're spending too much tax money."
I certainly can't speak to what he said in a radio interview I didn't hear, so you may be perfectly correct about what he said in it, but what I read in that article was more him saying that the money being currently spent could be spent in much better ways. It didn't come off to me as any sort of we-should-spend-less-on-the-homeless argument, but, to be sure, we're describing subjective summaries and reactions. My reading, though, was that it was as much an argument for spending a lot more money on relieving homelessness as it was arguing elsewise. Perhaps I was projecting my own views into my interpretation, however.
I may be flashing my freudian slip, revealing my paranoia for a few cheap beads. It's entirely possible that evil does not lurk in the hidden agenda of every single Bush appointee. Maybe they really are saying "we must focus on one end of this power-law distribution, but it's also very important that we don't forget the rest of the curve."
But you're right, I've been talking about the vasty undifferentiated evil "they", and that's poor technique.
"It's entirely possible that evil does not lurk in the hidden agenda of every single Bush appointee."
Malcom Gladwell is a political appointee to something?
I thought he was still a Canadian citizen, though I might easily be wrong about that, and I don't know that there's any law against appointing furriners to Senior Executive Service positions.
Though I'm reasonably sure that New Yorker columnist isn't such a position.
"But you're right, I've been talking about the vasty undifferentiated evil 'they', and that's poor technique."
Not Gladwell, Mangano. From the print article, p 101:
"The leading exponent for the power-law theory of homelessness is Philip Mangano, who, since he was appointed by President Bush in 2002, has been the executive director of the US Interagency Council on Homelessness, a group that oversees the programs of twenty federal agencies"
It's entirely possible that evil does not lurk in the hidden agenda of every single Bush appointee.
Why bother with very distant possibilities? It's also possible that the physical world we see is a fantasy put into our minds by a lying God, but that's not relevant to policy-making.
Yeah, I loved that article.
I thought he phoned it in with the last one, though (the article that used pit bulls as a lead-in to a discussion about the usefulness of stereotypes). It seemed like a throwaway that didn't make the cut for Blink.
Posted by Joe Drymala | Link to this comment | 02-16-06 9:32 AM
I liked this article too, although I had problems with the one or two places where he seemed to be saying "public policy isn't the answer; adhering to the policy is." Obviously, it's both: there has to be a policy to adhere to (e.g., w/r/t police brutality issues). I'm a little wary of arguments that X imperfect solution needs to be completely replaced with Y solution; it seems to me that the more useful argument is "soup kitchens and shelters work for some people, but for the hardcore homeless we need to provide these other things instead (or as well)."
Posted by bitchphd | Link to this comment | 02-16-06 9:32 AM
Right -- I think that's the core. That you need one policy solution for the shaft of the hockey stick, and another for the blade.
Posted by LizardBreath | Link to this comment | 02-16-06 9:39 AM
I am increasingly disenchanted with Gladwell. It sounds like a distribution that occurs fairly often. But I haven't read the article yet.
Posted by SomeCallMeTim | Link to this comment | 02-16-06 9:45 AM
I don't know how revolutionary it is -- it's certainly a concept I've seen before. Even given that, thaough, it's fairly powerful to recognize that there's a real class of social policy problems where addressing the bulk of the population affected by the problem hasn't got anything to do with addressing the bulk of the problem, which is covered by a small fraction of the population.
The core anecdote is about a homeless guy who, among other things, ran up a million dollar medical bill in various emergency rooms over a couple of years. Think about it - if there are a thousand guys like that, that's a billion dollars in available health care savings if we can find some way to get them medically under control.
Posted by LizardBreath | Link to this comment | 02-16-06 9:51 AM
"...but it's definitely worth reading."
Second that. I got around to finishing it yesterday, and it's in my next-five-or-so things-to-blog.
Posted by Gary Farber | Link to this comment | 02-16-06 9:59 AM
You see the same sort of distribution, for example in health care costs. Two percent of the population is responsible for 50% of the health care costs; I seem to remember that figure being thrown around ten years ago. The problem is that you don't know who falls into that 2% until that person starts his run.
Posted by SomeCallMeTim | Link to this comment | 02-16-06 10:02 AM
What was interesting to me about that article was not the possible savings from better targeted services (e.g., hospital bills) but the way it might force people to re-think hard problems that seem to admit no answer. I represent poor people on housing court in new york, and the article's thesis is correct, in my view: there are a handful of people who have very hard problems, but the vast majority can get by with minimal social service/legal intervention. The thesis -- which is not really earthshattering -- could nonetheless mean alot for debates about "welfare reform" and the like: we're not looking at a hopeless problem, but actually a very solvable one for all but a handful of people. And for those, lets do what Denver does --- just hand them a minimal income and an apartment, no questions asked.
That latter is the really radical part, in the context of social welfare policy. The idea of welfare reform is that you have a mostly recalcitrant population who needs to be punished before they do what is right. In fact the situation could be very different: what you have is a more or less capable population who don't necessarily need punishment to 'behave' or solve their problems, and a small segment that will remain totally impervious to any punishment you mete out (no eviction, no cut in welfare benefits will make them 'see the light,' and you are still left at the end of the day with someone you need to feed and give medical care).
Posted by sparacando | Link to this comment | 02-16-06 10:05 AM
The idea of welfare reform is that you have a mostly recalcitrant population who needs to be punished before they do what is right.
Or, to cut welfare reform advocates some slack, that they need to be provided with strong enough negative incentives to straighten up and fly right. While that's an awfully superficially appealing idea, it really seems to be the case that for a small population that consumes a great deal of social services, that that isn't an effective approach.
Two percent of the population is responsible for 50% of the health care costs; I seem to remember that figure being thrown around ten years ago. The problem is that you don't know who falls into that 2% until that person starts his run.
Well, you know that they aren't a generally healthy person walking into an internists office with a little rash, and so that making it more difficult and expensive for people to get minor ailments looked at is unlikely to do a lot to reduce health care costs. Which indicates that all the HSA, higher deductible, make people more price-conscious consuers of healthcare, etc., ideas are pointless and doomed as a means of controlling costs.
Posted by LizardBreath | Link to this comment | 02-16-06 10:39 AM
To nitpick, I'm not sure I see the difference between punishing people to make them behave and giving them negative incentives so they'll straighten up and fly right. And what I see as appealing is not so much the idea of negative incentives as the idea that there might possibly be an approach that works.
Excellent point about the healthy person with the rash.
Posted by Matt Weiner | Link to this comment | 02-16-06 11:42 AM
all the HSA, higher deductible, make people more price-conscious consumers of healthcare, etc., ideas are pointless and doomed as a means of controlling costsall the HSA, higher deductible, make people more price-conscious consumers of healthcare, etc., ideas are pointless and doomed as a means of controlling costs
Yeah, of course, because it's when you're dying that the money gets spent, at which point you're unlikely to comparison shop. Give me my triple bypass, now!!
Posted by Anonymous | Link to this comment | 02-16-06 11:47 AM
11 = me.
Posted by mcmc | Link to this comment | 02-16-06 11:48 AM
So where do I go to get my free hockey stick?
Sorry. Kidding aside, that really was an interesting article. As I read it I kept thinking, "This makes a lot of sense." Then, unfortuately, I came to this part (which also made sense, somewhat*):
Power-law solutions have little appeal to the right, because they involve special treatment for people who do not deserve special treatment; and they have little appeal to the left, because their emphasis on efficiency over fairness suggests the cold number-crunching of Chicago-school cost-benefit analysis.
* Gladwell's explanation for why the right wouldn't buy into this approach seems spot-on to me, but what about his explanation for the left? Is "fairness" really the sticking point?
Posted by My Alter Ego | Link to this comment | 02-16-06 11:50 AM
It must be difficult in some cases to tell the difference between the shaft and the blade though. I'm pretty sympathetic to LB's formulation of the welfare reform position from my albeit anecdotal observation of the younger generation of the welfare-dependent population in my Central Valley town pre-TANF. Little girls would actually say "I'm going to go on welfare when I grow up." So if we grant that negative incentives are good for a shaft, how do you provide a fail-safe for the blade without undoing the negative incentive in the first place? The homelessness "solution" is that shelters and intractable homelessness are so awful that no one who can avoid the situation will fail to, but welfare, getting government money to care for your kids in your home, is a lot more appealing, especially when working doesn't mean a huge jump, or any jump up, in quality of life.
Posted by Tia | Link to this comment | 02-16-06 11:51 AM
I'm not so much saying that there is a difference, more that using the softer language acknowledges that not all advocates of the element in welfare reform that we're talking about are motivated by cold-hearted malice, as 'punishment' suggests. They may instead believe that the negative incentives will mend up making everyone's life esier and happier.
Despite that nod to the motives of welfare reform advocates, I still think, partially for the reasons laid out in the article, that they're wrong, and that consumers of social services are mostly composed of (1) a large class of people whose lives are quite bad enough already, thank you, and who don't need any more negative incentives to make their lives better -- they need fairly small amounts of practical help, rather than an attitude adjustment, and (2) a small class of people who are in so much trouble that negative incentives are irrelevant, and who need lots of help to keep from being an endless problem.
Posted by LizardBreath | Link to this comment | 02-16-06 11:51 AM
TIA TO SHAFTED MOMS: DROP DEAD
Posted by Joe Drymala | Link to this comment | 02-16-06 11:53 AM
13: eh. I don't know if I can speak for the left, but it's not unappealing to me. If not addressing the root problems of the blade costs more money than addressing them, it's no more "unfair" to pour resources into their apartments and private nurses than it is into emergency rooms. Saving money means you have more money to spend on programs for the shaft. I'm pretty dubious that this is so much of an issue for "the left" in general, since I'd think one of the left's fundamental principles is that some people need more help than others, it's right that resources are expended on these few at the expense of the many, and this expense will probably ususally not turn out to be at the expense of the many after all, because these few, when not cared for, are a drain on the many in other ways.
Posted by Tia | Link to this comment | 02-16-06 11:59 AM
Tia -- next time you have trouble distinguishing between the shaft and the blade, just give me a holler -- I'll be glad to come help out.
Posted by Jeremy Osner | Link to this comment | 02-16-06 12:00 PM
a hockey stick once distributed one of my teeth across the rink. care to explain that, gladwell?
also, i thought the tipping point could have been so much better.
Posted by matty | Link to this comment | 02-16-06 12:03 PM
And 13 -- I could see the proposed solution not appealing to certain straw leftists for fairness reasons.
Posted by Jeremy Osner | Link to this comment | 02-16-06 12:04 PM
As one more left-leaning voter, I'm inclined to cast my vote for "17 gets it exactly right."
Posted by My Alter Ego | Link to this comment | 02-16-06 12:06 PM
Little girls would actually say "I'm going to go on welfare when I grow up." So if we grant that negative incentives are good for a shaft, how do you provide a fail-safe for the blade without undoing the negative incentive in the first place?
I don't know that I do grant that, I just wanted to tag it as not obviously insane or malicious. Wouldn't you think that positive incentives (free day-care if you're enrolled in college, heavily subsidized if you work... you know the sort of thing) are as likely to work for someone who's capable of functioning at all as negative incentives, and really haven't been tried in any solid way?
Posted by LizardBreath | Link to this comment | 02-16-06 12:11 PM
I think the positive incentives help, and I definitely support them, but I don't know that they would have been sufficient in and of themselves to convince people that were pretty comfortably ensconced in their homes that all of a sudden they should go out and work at McDonalds. I'm not saying they were living high on the hog, or they were welfare queens. But I lived in an apartment complex where rent was 250 dollars a month on a two bedroom apartment off a highway, and I had neighbors who were on AFDC (or something), and I interacted with them and helped some of their kids with their homework, and while they certainly didn't have much money, they spent time with their family and were probably happier than they would have been doing hard, stupid work (I hate doing hard stupid work, and if the government paid me to not be a secretary, I would take the money). I don't find it crazy to suppose that what some AFDC-recipients really wanted was to stay home and take care of their kids (that was the impression I got), and I pretty much sympathize with that, but if it's more expensive than subsidized child care, etc., I don't think it should happen at taxpayer expense. And even if it isn't, I think it built up a culture where kids didn't even have any professional ambition, reducing approximately to zero the chance that they would ever find out that there might be work available to them that wouldn't be stupid.
Posted by Tia | Link to this comment | 02-16-06 12:26 PM
When Gladwell says "we can't do both" he seems to feel that social policy rationale will eventually do these little initiatives in. Gladwell always wants to be a provocateur, so you can be sure his "truth' needs to be, at least rhetorically, separate from and hostile to the "left" and "right," as he has constructed them. I don't buy it; I think these approaches are much closer to liberalism, and most of us here are delighted at the notion of being so rational and doing so much good. As has been observed above, the shelters and soup kitchens are also doing good, and even if the typical person spends only one night, I want them to be there for that night.
And he has lumped together problems with different moral dimensions as if they all equally illustrated his left-and-right-are-equally-obtuse thesis. The car emissions thing particularly bothered me. Do we really want to be equally punished because we have collectively contributed to polution? I'm a car guy, I'm the only person I know who is looking forward to watching the Daytona 500, and I always test my 21-year-old, 150,000 mi. Honda Civic "hot." Would I resent Stedman's vans taking over enforcement? Would you? Of course not.
Posted by John Tingley | Link to this comment | 02-16-06 12:29 PM
I thought Gladwell missed the point: homelessness is not one category, it's (at least) two. Being two categories, it needs two responses.
His 'hockey stick' image is wrong. It's not a hockey stick, it's a bi-modal distribution. Whenever you get a bi-modal distribution you should be thinking 'we don't have one population here, we have two.'
Homelessness is a symptom. It's a result. It's like slurred speech: maybe a stroke, and maybe too many beers. Some homelessness results from a run of unfortunate events (job loss, etc.) and sometimes it's something else (long term severe mental problems).
The scary thing about Gladwell's analysis is that it justifies throwing out the remedies that do work in many cases because they don't work in all cases. This has been a very popular political argument: let's end the war on poverty, because it hasn't cured poverty.
I'm echoing what B and LB have already said. I know. But it's a very important point. And I think this place needs a chorus.
Posted by Michael H Schneider | Link to this comment | 02-16-06 12:32 PM
#25
I'd like to sing first tenor in that chorus.
Posted by John Tingley | Link to this comment | 02-16-06 12:42 PM
25: I thought that *was* Gladwell's point. What's the difference between a bimodal and a power law distribution, and why isn't homelessness a power law distribution?
I don't think his analysis justifies what you say it does. He isn't advocating getting rid of soup kitchens, or emmisions standards for car makers, he's advocating adding additional measures to take care of the top of the distribution.
Posted by pdf23ds | Link to this comment | 02-16-06 4:05 PM
26: I'll get back to your first question.
I don't think his analysis justifies what you say it does. He isn't advocating getting rid of soup kitchens,...
From Gladwell, quotoing (approvingly, I think) Philip Mangano
Our intent is to take homeless policy from the old idea of funding programs that serve homeless people endlessly and invest in results that actually end homelessness
I don't have a net cite. It's page 102, bottom penultimate paragraph of column 1, in the print version.
I may be misreading, or it may be a deliberate obfuscation, but those are two problems. First, the endless stream of new, temporarily homeless people. Second, the endless homelessness of certain homeless individuals. Those two things should be kept quite distinct, and they are muddled in that quote.
Posted by Michael H Schneider | Link to this comment | 02-16-06 4:24 PM
I think you can play games with axes to turn a normal distribution with a long tail into a power law/hockey stick. If there are two populations, the relevant question is whether you can distinguish the two. The second concern is the one that Tia mentions and Gladwell gestures at - will your attempt to do so create incentives for people to move from one population to the other?
Posted by SomeCallMeTim | Link to this comment | 02-16-06 4:32 PM
Not exactly on topic, but: I do budgets for government-subsidized housing and I notice that none of the Chinese tenants report their medical expenses to me (or, through me, to HUD), even though it would significantly reduce their rent. I thought it was a language or miscommunication issue, so I organized a meeting where a translator explained the issue--emphasizing that Chinese herbs, which they spend often spend hundreds of dollars on, would count. They're still all reporting 0 in medical expenses. I don't know if it's a cultural thing, not to take advantage of everything going, or what.
Posted by ac | Link to this comment | 02-16-06 4:48 PM
What's the difference between a bimodal and a power law distribution, and why isn't homelessness a power law distribution?
AFAIK, a bimodal distribution looks like a snake that swallowed two pigs. A power law distribution looks like a hyperbola, starts of rising slowly, then rises more rapidly, then rises even more rapidly - with the rate of increase steadily increasing towards infinite. The classic example is population increase. A hockey stick is straight, then turns a corner and again goes straight, but at a greater slope. The now classic example is, of course, global climate change.
here's Gladwell, page 98, third column:
... 80% of homeless were in and out really quickly ... The next ten percent were ... episodic users. They would come in for three weeks at a time, and return periodically ....the last ten percent ... were the chronically homeless, who lived in the shelters, sometimes for years at a time.
If we plot the amount of resources used on the horizontal, and number of individual cases on the vertical, we get the snake and pigs. Big, big lump at the left side (lots of cases, few resources used per case). Then the episodics. Then the second (smaller) pig, the cluster of cases that consume a lot of resources.
Okay, maybe it's a trimodal distribution.
I see that SCMT has already addressed the question. Fortunately I like being part of the chorus.
Posted by Michael H Schneider | Link to this comment | 02-16-06 4:52 PM
will your attempt to do so create incentives for people to move from one population to the other?
I think this worry is overstated -- the kind of help that, say, the guy in the article needed, entailing constant supervision, really isn't all that attractive to most ordinarily lazy people.
Posted by LizardBreath | Link to this comment | 02-16-06 4:55 PM
I don't know; I've often thought I'd like a nanny.
Posted by text | Link to this comment | 02-16-06 5:02 PM
really isn't all that attractive to most ordinarily lazy people.
Probably not attractive for the people who need it, either. There are claims that a lot of the persistent homeless problem is explained by the release of a large number of people from mental institutions under Reagan. The number of people who think that they're the fuckups who need their lives run for them is, I'm guessing, pretty small. So we need some way to distinguish them from the population. That's hard, I think.
Posted by SomeCallMeTim | Link to this comment | 02-16-06 5:09 PM
I've often been told I needed a keeper, but that's a different issue.
Talk of incentives presupposes rational actors. I don't really think it's the rational ones that are the problem. The rational ones are probably the ones who out and stable quickly.
Some people are crazily self-destructive. Hauling them to the ER doesn't change that. That's the lesson I drew from Gladwell's example of Murray Barr.
If the long term chronic homeless people are crazy, shouldn't we look at mental health services? We do sheltered living for the developmentally disabled, and for some chronically mentally ill. Putting a crazy person in an apartment won't help if they're too crazy to handle it.
Some also have long term medical issues - cirhossis, diabetes, etc. We have a medical system that's primarily oriented to emergencies. Hauling someone to the ER is a lousy way to deal with these problems, too.
I guess I'm saying that if we had a broader and deeper support system, rather than the patchwork of gaps and holes and expensive emergency interventions we have now, it might work better.
But SCMT is absolutely right, we need a diagnostic to distinguish among the population, to fit the solution to the problem.
Posted by Michael H Schneider | Link to this comment | 02-16-06 5:17 PM
We also have to decide whether we really want to give self destructive people the freedom to self destruct.
Posted by Michael H Schneider | Link to this comment | 02-16-06 5:25 PM
Re. Tia's comment, this is a tough one. My sense, having also grown up in the valley, is that the "stupid work" problem is the real kicker: where the stigma against an embarrassing job is greater than the stigma against public assistance, you've got a real problem. Of course, one solution is the provision of non-lame jobs; but another, frankly, is making higher ed, vocational training, community college, etc. *possible* and accessible for folks who have no family background and for whom therefore the choices really do seem to be "stupid job" or "welfare."
Posted by bitchphd | Link to this comment | 02-16-06 5:28 PM
Huh, this thread seems to have somewhat converged with the other one.
Posted by Matt Weiner | Link to this comment | 02-16-06 5:56 PM
Huh, this thread seems to have somewhat converged with the other one.
You mean into total bullshit? I don't mean bphd, but Michael and SCMT are swapping misinformation & misinterpretations and then patting each other on the back.
Two percent of the population is responsible for 50% of the health care costs; I seem to remember that figure being thrown around ten years ago.
OK, let's start by not pulling bogus statistics out of our asses. Save that for the mineshaft. I googled around for this data, and didn't turn it up. I did find people citing this sort of data (20/80 is the most common formulation on the web), but in the cases where they linked a source, that source had made it up a lot like SCMT.
I think you can play games with axes to turn a normal distribution with a long tail into a power law/hockey stick.
Mmm, no. Back to math class, Tim. May I call you Tim?
These are different distributions. They are statistically distinguishable. The log normal looks a lot like a power law, but even this is distinguishable with the right test and enough data.
What these distributions mean is open to interpretation, and in fact considerable misinterpretation. (skim down to where Shalizi begins 'There have been two problems'...)
It's not a hockey stick, it's a bi-modal distribution.
Michael, two populations don't necessarily imply a bimodal distribution. If you have one population with very low incident rates, as in the average cops in LAPD violence case, and one population with very high incidence rates, then you will only have one mode, which is the value that occurs most frequently in the data set. Because the low violence population does not account for many incidents for each person.
Gladwell's article is pretty confusing about what is being graphed, and it's pretty different stuff in the different examples of LAPD, homelessness, and polluting. Keep in mind that these 'hockey sticks' are generally frequency tables with specific individuals represented on the X axis - like you graphed a scatter plot of the height of all the people in your college class or office and then arranged them left to right from shortest to tallest.
I'm a little wary of arguments that X imperfect solution needs to be completely replaced with Y solution
Damn straight, b. In his quest for 'the middle way' Gladwell completely ignores that these are the distributions that result after liberal policy solutions have been applied. Would the emissions pattern look like this without regulation? I don't think so. Our hockey stick polluters might have been the extreme tail of a normal distribution before the policy was applied. We're just facing the reality that the extreme groups don't respond to the same treatments that those at the middle do.
I run into this power law stuff in some of the areas where I work. Some of it has real utility, but as Shalizi describes above a lot of it is really half-assed stuff from physicists and complexity theorists who are being really sloppy about distinguishing between the mechanisms that lead to the outcomes. It drives me kind of crazy - I had to listen to someone bang the table and go on about Pareto distributions and infinite variance just last weekend. I guess its pent up frustration that's led me to this Farberesque post.
If we plot the amount of resources used on the horizontal, and number of individual cases on the vertical, we get the snake and pigs. Big, big lump at the left side (lots of cases, few resources used per case). Then the episodics. Then the second (smaller) pig, the cluster of cases that consume a lot of resources.
Please, Michael, tell me this has something to do with FL's superkoranic fellatio powers. Please.
Posted by cw | Link to this comment | 02-16-06 8:14 PM
Oh my god, I forgot about this:
A hockey stick is straight, then turns a corner and again goes straight, but at a greater slope. The now classic example is, of course, global climate change.
No, no, no! That was a completely different issue. That 'hockey stick' is about the time trend in atmospheric temperature. If there is hockey stick in the temperature data then the rate of increase in temperature has accelerate significantly, most likely because of human carbon emissions.
Posted by cw | Link to this comment | 02-16-06 8:18 PM
The classic example is population increase. A hockey stick is straight, then turns a corner and again goes straight, but at a greater slope. The now classic example is, of course, global climate change.
Global climate change and population increases are not distributions, they are over-time trends.
What's the difference between a bimodal and a power law distribution, and why isn't homelessness a power law distribution?
A bimodal distribution has two peaks, or humps. (Insert joke here.) In the simplest cases one might just be two normal distributions observed together -- imagine you measured the average heights of 100 7th-graders and 100 10th graders, for instance, and plotted the distribution. It would be bimodal.
Loosely speaking, in the kind of distributions Gladwell is talking about (power law distributions), all the events/items of interest in the population are concentrated in a tiny number of observations, and the size/intensity/frequency of the events is related to the frequency of their occurence. (Insert Mineshaft joke here, at the Mineshaft.) So, 1 percent of homeless consume 50 percent of the homeless budget; 5 percent consume 75 percent; 10 percent consume 90 percent, etc. Often power law distributions are really just log-linear distributions, which are easier to understand but much less exciting-sounding.
Zipf's law is a nice example --- it says that the nth most common frequency will occur 1/n as often as the first. This is roughly true for things like city size, weblog readership, earthquake strength and so on. For unhappy academics, there's a version for productivity, too. Of the universe of people with PhDs, maybe 50 percent never publish a single article. (This is quite true.) Of the 50 percent who do publish at least one article, 50 percent of them only publish a single one. Of the 25 percent (of the total) who have published at least 2 articles, half only ever publish two, and so on down the line.
Posted by ProbablyWasted | Link to this comment | 02-16-06 8:36 PM
I was right: cw pre-empted me in #39 and #40. Oh well.
Posted by WasWasted | Link to this comment | 02-16-06 8:38 PM
One other thought - a bimodal distribution is impossible in the sort of distribution chart Gladwell talks about because the individuals are ranked by frequency. Thus the two modes would be right next to each other at the left end of the chart and would just look like a thick peak.
Posted by cw | Link to this comment | 02-16-06 8:53 PM
cw:
OK. You have actual knowledge, and I bow in your general direction. FWIW, I believe the 2%-50% figure (if I'm remembering it correctly) was given to me by an economist at HHS; it's entirely possible I'm am misremembering the numbers, but the source had reason to know. I'll try comprehending Shalizi's bits, but there's a very good chance it's beyond my ken.
Posted by SomeCallMeTim | Link to this comment | 02-16-06 9:25 PM
Sorry I was harsh, SCMT. The underlying work bugs me, and this discussion just dredged it up.
I do believe that health costs are skewed towards a small group of people every year. Who those people are and how they move into (and possibly/hopefully out of) that group will have as much to do with how we solve the problem as the distribution of people. If we stop with the distribution, the easy conclusion is - stop wasting money on these useless end of life treatments. But that may not be what's driving the data if we look at movement in & out of categories over time.
Posted by cw | Link to this comment | 02-16-06 9:45 PM
No worries, cw. I looked at the Shalizi stuff - beyond my ken, I'm afraid.
Posted by SomeCallMeTim | Link to this comment | 02-16-06 10:08 PM
Gladwell's article is pretty confusing about what is being graphed, ...
Apparently I was well and truly confused. Thank you for the clarification. And yes, I did call population and climate change distributions, which they clearly aren't. Apologies. I was trying to illustrate shapes without drawing, just imagining shapes, and forgot what I was trying to illustrate.
Keep in mind that these 'hockey sticks' are generally frequency tables with specific individuals represented on the X axis - like you graphed a scatter plot of the height of all the people in your college class or office and then arranged them left to right from shortest to tallest.
Aha! Thank you. That's not at all what I thought Gladwell was talking about. I was picturing a data set where he had a list of individuals and the shelter stay for each. E.g. Fred, 1 day; Sam, 1 day. Murray, 287 days; Guillame, 2 days; Harry, 1 day; George, 2 days; Bill, 312 days. etc.
I imagined him making a bar graph (if that's the right term; histogram, mebbe?). Pile Fred, Sam and Harry on top of the spot on the axis for 1 day. Pile George and Guillame atop the 2 day spot. Then a lot of spots with nobody. Then Murray and Bill in the spot for 250 or more days days. That gave me the two hump distribution, which I sloppily called bimodal.
I get the idea of lining them up in order of duration of stay - Fred, Sam, Harry all showing one day, George and Guillame showing two days (the shaft of the hockey stick) and then Murray 287 and Bill 312 standing way proud and tall, the blade.
I'm repeating all this so that someone can tell me I'm still an idiot.
I tried Shalizi and it's beyond my grasp. However, I got the strong impression he was saying that people leap upon power law distributions when they're not necessary to explain the data.
That takes me back to Gladwell. He claims that Culhane discovered that homelessness has a power law distribution (or that the resource usage by the homeless has such a distribution - he does say "[homelessness] has a power law distribution"). My from-the-ass belief is that it is much more like the height distribution of 80 1st graders, 10 4th graders, and 10 sixth graders treated as a single population. Triple humped.
Is he probably right, or is this another patented Gladwell whiz-bang New Idea that unifies everything?
Posted by Michael H Schneider | Link to this comment | 02-16-06 11:32 PM
Have we really gotten through 47 comments in this thead without a link to Malcolm Gladwell Is Not Gay?
Posted by washerdreyer | Link to this comment | 02-17-06 12:12 AM
I think Michael S has a point here. If the facts on the ground are "there is a small population that accounts for a disproportionate amount of the costs", then why not just say that, and think about how to solve that problem, rather than bringing in all this waffle about power laws? For one thing, there are several distributions with big tails and it doesn't really matter whether the data is described by a power law or not, and for another, talking about power laws means that you stop thinking about whether we're talking about "the three drunkest people in Spokane cause 50% of the trouble" or "Jim, Bob and Fatty cause 50% of the trouble", when the two cases would clearly warrant different solutions.
I think these silver-bullet solutions always tend to disappoint (and indeed, that the small tail of them which work really well are probably a power law phenomenon themselves). For one thing, as cw notes, they are usually quite parasitic on a whole nother set of solutions keeping things in line for the majority population. For a second, they can create perverse incentive effects, and Gladwell just seems to mention this problem, say how serious it is, and then trail off into a discussion of how wonderful his policy which has this problem is (which is at the heart of what I find most annoying about Gladwell). And for a third, I suspect that these problems are mostly of the "three drunkest people in Spokane" variety rather than the "Jim, Bob and Fatty". You sort out the problems of the worst 1% and guess what? There is a whole new worst 1% and they are still causing a disproportionate amount of harm.
I think the actual solution to these "power law" problems is to learn to live with them. Murray's drinking wasn't a problem for Murray; he seemed quite happy with it. And if there is a $1m of pain out there, there is probably a case for having one guy with a $1m of problems rather than 200 guys with $5k of problems, particularly if there is diminishing marginal disutility as there very well might be.
Posted by dsquared | Link to this comment | 02-17-06 7:39 AM
You sort out the problems of the worst 1% and guess what? There is a whole new worst 1% and they are still causing a disproportionate amount of harm.
This is a v. important point, because half the attraction of the power-law thing is that the distributions are scale-invariant -- which is cool, but it cuts both ways.
Posted by argle | Link to this comment | 02-17-06 7:52 AM
If the facts on the ground are "there is a small population that accounts for a disproportionate amount of the costs", then why not just say that, and think about how to solve that problem, rather than bringing in all this waffle about power laws?
This is a good point -- I figure Gladwell was doing some handwaving to make the article sound excitingly technical.
And if there is a $1m of pain out there, there is probably a case for having one guy with a $1m of problems rather than 200 guys with $5k of problems, particularly if there is diminishing marginal disutility as there very well might be.
This, on the other hand, I don't get. If you spend, say, $50,000 on services for Murray to keep him indoors and as healthy as possible (even if he remains alcoholic, which presumably he would), that saves the remaining $950,000 of his emergency room bills. I don't see how it would increase the costs incurred by the next drunkest homeless guys out there; they seem unconnected.
Posted by LizardBreath | Link to this comment | 02-17-06 8:33 AM
And at the end of the tail there's one guy who costs $500,000,000 all by himself.
Posted by John Emerson | Link to this comment | 02-17-06 8:42 AM
And his name ... John Emerson.
Posted by SomeCallMeTim | Link to this comment | 02-17-06 8:44 AM
On the subject of "perverse incentive effects" and 37, I'm just not convinced that positive incentives in the realm of the realistic would have done the trick by themselves, especially in the face of an entrenched culture (though maybe that's not true; perhaps a guaranteed annual income of 15K above what welfare gave them would have been enough). It takes a long time and a lot of investment to create a whole new economic structure in an area, and there's a limit to how many jobs you can create that are really more rewarding than spending the day with your kids or the rest of your family. In the meantime, by leaving the situation alone, the culture got even more entrenched. But then you're left with the possibility of cutting off assistance to people who really can't find work, either because they really are unemployable, or because there just isn't the work to be had, and hurting their children even more. Teh difficult.
Posted by Tia | Link to this comment | 02-17-06 8:54 AM
We could always start another war, and just draft 'em. I had a high school teacher who thought that wars for empire were mostly about (a) giving excess and potentially troublesome males something to do, and (b) thining out the population that would return.
Posted by SomeCallMeTim | Link to this comment | 02-17-06 8:58 AM
I aspire to that.
I did read once about a guy who had a $2 000 000 disease -- he had to be in a bubble for 6 months, IIRC -- and almost bankrupted his provider. As the story was told, it was a clearcut case: he recovered completely, but would have died without the 2 mil.
Posted by John Emerson | Link to this comment | 02-17-06 8:59 AM
The demographic explanation of war makes more sense than people give it credit for. It doesn't limit population much, though, unless you put women on the front lines. The Mongols practiced polygamy and replenished their troops with captives.
Posted by John Emerson | Link to this comment | 02-17-06 9:01 AM
As the women own the means of baby production, you'd have to draft them. More than that, we should be drafting them first.
Posted by SomeCallMeTim | Link to this comment | 02-17-06 9:05 AM
I've long been of the opinion that a military which would all be either gay or female or both would be a good thing. My motives for this are bad ones, however.
I feel the same way about marriage.
Posted by John Emerson | Link to this comment | 02-17-06 9:16 AM
"I'm sorry, miss, but you can only marry another woman. It's the law. Why would you want to marry a man?"
......
"So he could stick his WHAT? in your WHAT? Eeeeew. ....And your church tells you to do this?"
Posted by John Emerson | Link to this comment | 02-17-06 9:18 AM
Aren't a huge percentasge of healthcare costs tied up in end-of-life and near-end-of-life interventions? (Chemo, emergency surgery, etc.) The 1% is not just the folks who go the emergency room for a rash, but the people who are either dying or who will die without immediate, expensive treatment (e.g. heart valve replacement).
Posted by Chopper | Link to this comment | 02-17-06 10:00 AM
Well, exactly (note that this is also an area where patients are really unlikely to have the knowledge to be cost-sensitive in any kind of useful way.)
Posted by LizardBreath | Link to this comment | 02-17-06 10:04 AM
Is that 2% of patients over the whole lifespan, or 2% in any given year? Will most of us end up in the 2% eventually?
Posted by John Emerson | Link to this comment | 02-17-06 11:00 AM
"I've long been of the opinion that a military which would all be either gay or female or both would be a good thing."
Ever read Joe Haldeman's The Forever War?
Posted by Gary Farber | Link to this comment | 02-17-06 12:40 PM
Ever read Joe Haldeman's The Forever War?
Or Frank Herbert's God Emperor of Dune?
Posted by Matthew Harvey | Link to this comment | 02-17-06 12:52 PM
1. Per cw, the number and things realted to it are open to some doubt.
2. I believe the 2% is end of life primarily, but it's not straightforward. It is often something like someone goes into the hospital for a hip replacement, there are complications, and then more complication, as expense, death, and hilarity ensues. It's not clear how many months we are adding to that person's life, but (IIRC) not a lot.
3. IIRC, most people are not part of the 2%; most people die relatively peacefully.
4. My recollection is that there was a rough consensus among health economists that innovation drove costs, and that the only real limiter was de facto rationing. Not politically something one can openly say.
Posted by SomeCallMeTim | Link to this comment | 02-17-06 1:04 PM
I think that there was a small immediate saving when ICU doctors stopped trying to squeeze as many days as possible out of dying patients. When I worked with patients I saw one patient twice a month apart, and she'd apparently been in ICU the whole time, with about 5 IVs, 2 catheters, and four-point restraints. She died a week or two later.
I think that the idea that there's a lot of unnecessary care is somewhat a myth, though, as Tim says or implies.
Posted by John Emerson | Link to this comment | 02-17-06 1:29 PM
49: If the facts on the ground are "there is a small population that accounts for a disproportionate amount of the costs", then why not just say that, and think about how to solve that problem, rather than bringing in all this waffle about power laws?
Right. Thank you.
We're trying to understand what's going on so we can most efficiently change it for the better. Knowing how costs are distributed across the population can help - but only if it leads to better approaches.
I claim that understanding the underlying mechanisms that produce homelessness will help us improve the efficiency of our approach to homelessness. I have no evidence for this assertion. I have argument, which I sketch below.
Gladwell's invocation of the power-law doesn't help. I think it's another instance of what CW described in 39 as
...being really sloppy about distinguishing between the mechanisms that lead to the outcomes.
That is partly because characterizing it as a power-law distribution says, in effect, 'look over here, just ignore those cases behind the curtain.' As PW says in 41:
... [in (power law distributions] all the events/items of interest in the population are concentrated in a tiny number of observations ...
That's just what Gladwell was saying (p.101, col2):
The homelessness problem is like the LAPD's bad-cop problem. It's a matter of a few hard cases, and that's good news, because when a problem is that concentrated you can wrap your arms around it and think about solving it."
Treating is as a power-law distribution not only misdirects our attention away from the cases behind the curtain, it also leads one to expect the perverse incentive problem and the scale-invariant problem.
The alternative is to treat it as a two-hump (which I call bimodal) distribution. This makes sense to me because I think LB is correct in 16 in saying we've got two classes, two populations.
Treating it as bimodal leads one to look for approaches that work for humpA, and approaches for humpB, without being discarding humpA approaches because they don't work for humpB cases.
Treating it as bimodal leads to the question of how to tell whether a particular case is humpA or humpB. To look for the point where shaft becomes blade, to identify the distinguishing characteristics, so to speak. This helps avoid wasting money on humpA approaches to humpB cases, and vice versa.
Treating it as bimodal may also make the perverse incentive problem less worrisome. We're less likely to worry about a person trying to pass for humpB if we end up saying (for example) "yes, we'll give you a free apartment for you and your kids, as soon as you show that you are a dysfunctional psychotic or chronic alcoholic with untreated life threatening medical conditions."
Thus, I think Gladwell was doing more than mere technical handwaving. I think that defining a whole bunch of problems as power-law distribution problems is a way to confuse and befuddle people of ordinary intelligence (e.g. me), so we can say 'not our problem.' It's a way to say 'don't look at whether ordinary cops have a problem with using excessive force, it's just a few bad apples'; 'don't worry about paying for shelters and soup kitchens, it's only a few hard cases'; 'don't bother getting your car's emissions checked every year, it's not you it's them.' And, of course, it leads to a whole lot of rather irrelevant discussions of what constitute power-law distributions.
Posted by Michael H Schneider | Link to this comment | 02-17-06 1:47 PM
"Or Frank Herbert's God Emperor of Dune?"
Purely personal opinion, but I wouldn't recommend that; though, to be sure, it wasn't remotely as bad as the later books; like many OMG-I'm-making-so-much-money contract extensions, the series, IMO, had a qualitive decline with each book (though I gave up reading any, other than taking glancing looks, after the above; Dune: excellent; Children of Dune: adequate; God-Emperor of Dune: getting bad; thereafter: please, God, no).
Purely, as I said, personal opinion; I'm great with other people adoring all the others, and everything Brian Herbert ever did/does, and so forth.
As regards Gladwell, I don't do the math, alas, but I would keep in mind that he's writing for a popular audience, and much of the terminology being tossed about in this thread is not stuff one should, IMO, be using for a piece in the New Yorker; not unless you're given a lot more space to explain it, and why it's important, at least. And I didn't read him as being remotely as dismissive of various points as seems to be what's being suggested at spots above. But I've certainly read the above discussion with great interest, and learned some things from it, even if I have nothing useful to contribute.
Posted by Gary Farber | Link to this comment | 02-17-06 2:02 PM
I thought that hockey stick distribution was what the Edmonton police do in the tough neighborhoods.
Posted by John Emerson | Link to this comment | 02-17-06 2:07 PM
69: God knows I do not want to hijack this thread to get into an argument about science fiction, let alone with Gary of all people, but I'll just say I didn't think, for instance, Chapterhouse: Dune was all that bad.
Posted by Matthew Harvey | Link to this comment | 02-17-06 2:23 PM
To be fair, while women own the means of baby production, they still need the raw material, half of which is provided by men.
Posted by bitchphd | Link to this comment | 02-17-06 2:41 PM
half of which is provided by men
Not by weight.
Posted by apostropher | Link to this comment | 02-17-06 2:48 PM
I just read this week's New Yorker and New York Magazine on the plane. New York's article on blogs also prominently features the concept of power-law distributions, only as a justification for why it's so hard for new blogs to crack the "A List" and become the next Gawker or Kos. Interesting that both would have articles on that concept in the same week to such different ends.
Posted by Becks | Link to this comment | 02-17-06 5:03 PM
I only skimmed the online version of the New York article, but Clay Shirky said a lot of the stuff I saw in it, about blog popularity and power laws, in early 2003.
Posted by washerdreyer | Link to this comment | 02-17-06 5:39 PM
OT: if anyone else would like to have an aneurysm, you can read this article...
Posted by silvana | Link to this comment | 02-17-06 5:51 PM
I think that the power-law theory of blogs may be valid, the same as the power-law theory of city size.
Posted by John Emerson | Link to this comment | 02-17-06 6:37 PM
I like Clay Shirky's idea of the long tail - that successful internet businesses are exploiting the fat tail of the power law. That is, ebay and Amazon make their money on all the weird little shit that nobody else could sell (like Amazon - no discount on academic books) not on the center of the distribution.
But that early stuff he did on powerlaws and blog popularity was stupid. The point of the power law is that there is much more action in the tails of the distribution (hence the long tail or fat tail). If blog attention was normally distributed it would be _harder_ for a new blog to break in because there would be fewer subcommunities to build an audience in & the traffic would be more focused on the popular sites. So this bit about power laws and popularity seems to get it backwards.
Posted by cw | Link to this comment | 02-17-06 8:56 PM
"New York's article on blogs also prominently features the concept of power-law distributions, only as a justification for why it's so hard for new blogs to crack the "A List"....
Only by coughing up Clay Shirkey's thesis from what is more or less the Jurassic Period of the blogging era, at this point. I mean, it's correct (I think), but everyone acknowledged that umpty years ago (okay, three), back when he wrote it, and it was a hot topic for a few weeks.
"Interesting that both would have articles on that concept in the same week to such different ends."
Isn't that what happens with hackeyned old ideas? It would be only slightly less surprising, I think, if someone in both magazines used the concepts of "evolution" and "mutation" to different ends. Though not if they're doing a quick hack job on the Exciting! New! Topic! of Blogs!
"Clay Shirky said a lot of the stuff I saw in it, about blog popularity and power laws, in early 2003."
And some proto-notions on panix.talk back in the Nineties. But I only first saw his writing in 1995, so I don't know what he was writing about before that.
Posted by Gary Farber | Link to this comment | 02-17-06 10:47 PM
"hackeyned"
Or "hackneyed," even.
Posted by Gary Farber | Link to this comment | 02-17-06 10:48 PM
A good blog on the long tail is Chris Anderson's The Long Tail. Somewhere (possibly on that site) I read a clever bit about how the long tail effects of the internet had rewritten Andy Warhol's prediction: it's not that everybody will be famous for fifteen minutes, but that everybody will be famous for fifteen people.
Posted by My Alter Ego | Link to this comment | 02-17-06 10:52 PM
This rejoinder is entirely too late, but I wonder if some of the above discussion might miss what is useful about the Gladwell article. I respect the earnesty with which people familiar with the technical side of the argument approach the article, but let's not think that "homelessness policy" or "welfare policy" is about solving problems --- at least not always. These policies usually embody a set of moral judgments ("the kind of society we want to live in") rather than a set of actual solutions. We can go back and forth about the different methods of describing a problem and its distribution, but that's different from making policy. Even in health care (I'm shocked to hear) some economists argue that offering less coverage is "efficient" because most people can live with minor illnessess just fine --- giving more people coverage just means they are going to run off to the doctor for every little problem and run up costs. That sounds like a technical argument about efficiency, but even that is a moral argument about the sort of society that economist is willing to live in.
So, I think the Gladwell article says this: when it comes to homelessness, stop thinking about it as an unsolvable problem comprised of tens of thousands of people who cannot be absorbed into society --- an "underclass" --- and realize that the overwhelming majority of homeless are amenable to help, want it, and, if given it, will make the best use possible.
If you can use this to convince people to make a different moral judgment of the homeless or people on welfare, you can get to different, smarter policy. For example, over and over again the "underclass" thesis is disproved, or at least proven to describe a very small percentage of the population; recall that welfare reform was passed at a time when many believed every homeless person, every person on welfare, was part of an class of people totally disconnected from, well, the "mainstream." I think you can use the kind of argument Gladwell's making to make a distinction between the handful of hard problems and the great majority of less-hard problems solvable by targeted help.
Posted by sparacando | Link to this comment | 02-18-06 9:53 AM
What sparacando said.
Posted by Gary Farber | Link to this comment | 02-18-06 3:19 PM
And some proto-notions on panix.talk back in the Nineties. But I only first saw his writing in 1995, so I don't know what he was writing about before that.
These here intertrons existed before the run-up to the 2004 election?
Posted by washerdreyer | Link to this comment | 02-18-06 3:35 PM
After hearing Gladwell interviewed today on Morning Edition, I can officially declare that I find him despicable. I was trying towake up so my recollection is a bit dim and I haven't looked for it on npr.org, but I remember I few points.
He described Mangano, the Bush appointee who's so keen on calling this a power-law distribution problem "wonderful." I take that as support for my earlier assertion that Gladwell was quoting this guy approvingly.
The interview had to ask twice "but what about the single mom with three kids?" before getting an answer. I think finally Gladwell came down as saying 'I hope, really hope, that person isn't forgotten as we solve the homelssness problem, which is really about the hard cases.'
Bah, humbug.
This is the rebranding of homelessness. First, we develop a new image: it's no longer mom & kids, our poster child isnow to be Million Dollar Murray. Then, since homeless people are clearly unsympathetic chronic drunks, we cut 'em off.
They did this in reverse with AIDS (remember Ryan White? I hope that's the right name). They did this with AFDC (remember the caddie driving welfare queen?). Now they're going to do it with homelessness. Now, homelessness isn't going to mean the young family down on its luck, it's going to mean Murray. This really sucks.
In other words, what Sparacando said.
Posted by Michael H Schneider | Link to this comment | 02-18-06 6:29 PM
Some day I'll learn to try that "preview" button. Oh well.
Posted by Michael H Schneider | Link to this comment | 02-18-06 6:31 PM
Incidentally, I suggest that all you New Guy bloggers here at Unfogged better unload your vast financial stakes in Unfogged, Inc., because the blog market is about to crash.
Michael: "They did this in reverse with AIDS (remember Ryan White? I hope that's the right name)."
I'm not following you terribly well: it's a bad thing that there's been more and more money poured into AIDS research and treatment over the years?
"Then, since homeless people are clearly unsympathetic chronic drunks, we cut 'em off."
This would be a new argument someone would be making?
Posted by Gary Farber | Link to this comment | 02-18-06 8:18 PM
The point I was trying to make was that if you change the brand, the image, the face of a problem, you change the what's politic. With Ryan White, by changing it from queers to innocent kids you made funding more possible. It could have worked perfectly well in reverse.
By calling homelessness a single problem with a power-law distribution, rather than two (or more) problems, you can make Million Dollar Murray the brand, the face, the issue. That will allow everyone to say "homelessness is not a problem of ordinary people, not a problem of single moms with three kids" and that will eventually allow funding to be cut off.
What I heard Gladwell assume, without actually saying it, is that the problem with homelessness is that we're spending too much tax money. If the goal is to reduce how much tax money you spend, making it an issue of Murrays is a great approach.
Posted by Michael H Schneider | Link to this comment | 02-18-06 10:49 PM
"That will allow everyone to say 'homelessness is not a problem of ordinary people, not a problem of single moms with three kids' and that will eventually allow funding to be cut off."
I'm afraid that my perspective is that that's what they've always said since FDR's day ended (or at least since Nixon's began, if you prefer), and that there's never been much funding in the first place.
Just different perspectives, not different facts.
"What I heard Gladwell assume, without actually saying it, is that the problem with homelessness is that we're spending too much tax money."
I certainly can't speak to what he said in a radio interview I didn't hear, so you may be perfectly correct about what he said in it, but what I read in that article was more him saying that the money being currently spent could be spent in much better ways. It didn't come off to me as any sort of we-should-spend-less-on-the-homeless argument, but, to be sure, we're describing subjective summaries and reactions. My reading, though, was that it was as much an argument for spending a lot more money on relieving homelessness as it was arguing elsewise. Perhaps I was projecting my own views into my interpretation, however.
Posted by Gary Farber | Link to this comment | 02-18-06 11:11 PM
I may be flashing my freudian slip, revealing my paranoia for a few cheap beads. It's entirely possible that evil does not lurk in the hidden agenda of every single Bush appointee. Maybe they really are saying "we must focus on one end of this power-law distribution, but it's also very important that we don't forget the rest of the curve."
But you're right, I've been talking about the vasty undifferentiated evil "they", and that's poor technique.
Posted by Michael H Schneider | Link to this comment | 02-18-06 11:26 PM
"It's entirely possible that evil does not lurk in the hidden agenda of every single Bush appointee."
Malcom Gladwell is a political appointee to something?
I thought he was still a Canadian citizen, though I might easily be wrong about that, and I don't know that there's any law against appointing furriners to Senior Executive Service positions.
Though I'm reasonably sure that New Yorker columnist isn't such a position.
"But you're right, I've been talking about the vasty undifferentiated evil 'they', and that's poor technique."
They have ways of punishing you for it, you know.
Posted by Gary Farber | Link to this comment | 02-19-06 1:27 AM
Not Gladwell, Mangano. From the print article, p 101:
"The leading exponent for the power-law theory of homelessness is Philip Mangano, who, since he was appointed by President Bush in 2002, has been the executive director of the US Interagency Council on Homelessness, a group that oversees the programs of twenty federal agencies"
Posted by Michael H Schneider | Link to this comment | 02-19-06 12:09 PM
It's entirely possible that evil does not lurk in the hidden agenda of every single Bush appointee.
Why bother with very distant possibilities? It's also possible that the physical world we see is a fantasy put into our minds by a lying God, but that's not relevant to policy-making.
Posted by John Emerson | Link to this comment | 02-19-06 1:09 PM