Here's analysis by someone well-informed about the issues, whose scientific output I think is usually interesting:
http://www.michaeleisen.org/blog/?p=1480
Basically, he (IMO correctly) identifies the explanation of results as the step most in need of regulation.
I think it's good that it's being scrutinized, since they weren't even showing that the tests produced consistent results.
There's value in getting a big database of genomes to analyze over time, but it shouldn't be financed by giving people shoddy diagnoses.
A GAO report on the mishmash that can result. It's not as simple as drawing from peer-reviewed articles - or maybe it should be, but it isn't.
I think the historical lower level of regulation for laboratory tests was due to the understanding that the results usually went through doctors or other professionals with the training to interpret them - blog post on the subject here.
The FDA letter, which is a few clicks away from the OP.
I was going to use that service. It seems like a shame if they are going to regulate it away.
If they're making a claim about a particular allele being associated with a disease, of course they should have to back that up. What's the issue?
Tabarrok, quoted in the linked article:
Moreover, the FDA wants to judge not the analytic validity of the tests, whether the tests accurately read the genetic code as the firms promise (already regulated under the CLIA) but the clinical validity, whether particular identified alleles are causal for conditions or disease. The latter requirement is the death-knell for the products because of the expense and time it takes to prove specific genes are causal for diseases.
Huh? If you can't prove it, don't claim it.
Extremely interesting Epigenetics paper, showing that one epigenetic modification, DNA methylation, is directly heritable, with no question of effect coming from fetal environment.
http://www.ncbi.nlm.nih.gov/pubmed/24292232
I am unsure about the paper's main conclusion (couldn't tell how many F0 Mice there were), but this piece was new to me and definitely a really interesting result. I'd be really interested if anyone knows of similar evidence of heritability for the various histone modifications.
With proof of heritability without influence of fetal environment, IMO epigenetic modifications look much more important than I had thought, plausibly essential. As with a bunch of other regulatory mechanisms, the details for particular genes vary hugely from species to species, making animal models and thus clean genetics impossible.
Minivet's ffirst link reminds me of how much I like the GAO.
Does anyone know about the reliability of the "ancestry" claims 23&me makes?
4-5: Knowing little of this except that "Alex Tabarrok is almost certainly wrong" I tend to agree. It sounds like the FDA's position is that if you're going to do this as a test that merely states results, that's one thing. If you're going to make claims about particular alleles being causal for conditions or diseases, you are doing something that is potentially very misleading and dangerous and sure we're going to regulate you.
Minivet, I think the tests do produce consistent results in terms of appropriate DNA sequencing (assuming similar sample quality). There are acceptable false-positive and false-negative rates for any test (and lots of validation and tolerance testing is required), but the GAO report is more about the analysis and interpretation. Say you have three peer-reviewed papers. All three are about prostate cancer risk, but they look at very different populations. They might all be valid results, but they couldn't necessarily make the same conclusions about relative risk. The company then needs to decide which population or markers to use as predictive. That's kind of a crapshoot. I don't really think it's wildly irresponsible, though, since I think they are using the same data a geneticist or physician would have.
Also, the vast majority of folks who got sequencing done said they were going to discuss the results with their doctor.
At any rate, I think you're making them sound extra-sketchy, when I think they're medium-sketchy. It's not as if they're diagnosing patients with a disease and also selling treatment.
The aren't making the claim tht it's causal, right? They're just saying they're associated.
Medium sketchy is certainly more than enough to warrant regulation by a responsible agency. In any case I'm sure they want there business model to appeal to people who think that they are legitimately getting information about risk of disease.
GY in 4: If you can't prove it, don't claim it.
Eisen's post that LW linked is pretty good on this; he notes that our understanding of genetics is messy at best, and so reporting on it needs to reflect that messiness. Holding 23andme to a substantially higher standard than genetics in general seems unreasonable.
Personally, I thought 23andme did this fairly well - they gave some ratings to the strength of the correlations they reported (most of the ratings are pretty low), and had links to the current science for those who wanted to drill down more.
12 -- I think you are dead wrong. From the FDA's letter:
This product is a device within the meaning of section 201(h) of the FD&C Act, 21 U.S.C. 321(h), because it is intended for use in the diagnosis of disease or other conditions or in the cure, mitigation, treatment, or prevention of disease, or is intended to affect the structure or function of the body. For example, your company's website at www.23andme.com/health (most recently viewed on November 6, 2013) PGS for providing "health reports on 254 diseases and conditions," including categories such as "carrier status," "health risks," and "drug response," and specifically as a "first step in prevention" that enables users to "take steps toward mitigating serious diseases" such as diabetes, coronary heart disease, and breast cancer. Most of the intended uses for PGS listed on your website, a list that has grown over time, are medical device uses under section 201(h) of the FD&C Act. Most of these uses have not been classified and thus require premarket approval or de novo classification, as FDA has explained to you on numerous occasions.
Some of the uses for which PGS is intended are particularly concerning, such as assessments for BRCA-related genetic risk and drug responses (e.g., warfarin sensitivity, clopidogrel response, and 5-fluorouracil toxicity) because of the potential health consequences that could result from false positive or false negative assessments for high-risk indications such as these. For instance, if the BRCA-related risk assessment for breast or ovarian cancer reports a false positive, it could lead a patient to undergo prophylactic surgery, chemoprevention, intensive screening, or other morbidity-inducing actions, while a false negative could result in a failure to recognize an actual risk that may exist. Assessments for drug responses carry the risks that patients relying on such tests may begin to self-manage their treatments through dose changes or even abandon certain therapies depending on the outcome of the assessment. For example, false genotype results for your warfarin drug response test could have significant unreasonable risk of illness, injury, or death to the patient due to thrombosis or bleeding events that occur from treatment with a drug at a dose that does not provide the appropriately calibrated anticoagulant effect. These risks are typically mitigated by International Normalized Ratio (INR) management under a physician's care. The risk of serious injury or death is known to be high when patients are either non-compliant or not properly dosed; combined with the risk that a direct-to-consumer test result may be used by a patient to self-manage, serious concerns are raised if test results are not adequately understood by patients or if incorrect test results are reported.
Your company submitted 510(k)s for PGS on July 2, 2012 and September 4, 2012, for several of these indications for use. However, to date, your company has failed to address the issues described during previous interactions with the Agency or provide the additional information identified in our September 13, 2012 letter for (b)(4) and in our November 20, 2012 letter for (b)(4), as required under 21 CFR 807.87(1). Consequently, the 510(k)s are considered withdrawn, see 21 C.F.R. 807.87(1), as we explained in our letters to you on March 12, 2013 and May 21, 2013. To date, 23andMe has failed to provide adequate information to support a determination that the PGS is substantially equivalent to a legally marketed predicate for any of the uses for which you are marketing it; no other submission for the PGS device that you are marketing has been provided under section 510(k) of the Act, 21 U.S.C. ยง 360(k).
The Office of In Vitro Diagnostics and Radiological Health (OIR) has a long history of working with companies to help them come into compliance with the FD&C Act. Since July of 2009, we have been diligently working to help you comply with regulatory requirements regarding safety and effectiveness and obtain marketing authorization for your PGS device. FDA has spent significant time evaluating the intended uses of the PGS to determine whether certain uses might be appropriately classified into class II, thus requiring only 510(k) clearance or de novo classification and not PMA approval, and we have proposed modifications to the device's labeling that could mitigate risks and render certain intended uses appropriate for de novo classification. Further, we provided ample detailed feedback to 23andMe regarding the types of data it needs to submit for the intended uses of the PGS. As part of our interactions with you, including more than 14 face-to-face and teleconference meetings, hundreds of email exchanges, and dozens of written communications, we provided you with specific feedback on study protocols and clinical and analytical validation requirements, discussed potential classifications and regulatory pathways (including reasonable submission timelines), provided statistical advice, and discussed potential risk mitigation strategies. As discussed above, FDA is concerned about the public health consequences of inaccurate results from the PGS device; the main purpose of compliance with FDA's regulatory requirements is to ensure that the tests work.
In other words, they are marketing a product in a way that has the potential to create substantial consumer health risk, and haven't taken basic steps required by law to ensure that their marketing isn't harmful. Boo hoo hoo 23 and Me.
A conspiratorial theory that I somewhat like: One of the effects of 23andme and similar outfits is cutting doctors out of the loop. Doctors don't like that; doctors make up (and strongly influence) the FDA; hence the FDA moves to block this disrupting influence.
Eisen's post that LW linked is pretty good on this; he notes that our understanding of genetics is messy at best, and so reporting on it needs to reflect that messiness. Holding 23andme to a substantially higher standard than genetics in general seems unreasonable.
It seems perfectly reasonable to me. They're selling what are in effect medical tests, and at a very minimum implying that particular results give a certain percentage chance of getting a particular disease. We, quite rightly, strictly regulate medical claims (in theory anyway). I don't see why this should be any different than other medical claims. As soon as you step beyond a straight presentation of the science in a product you're selling to people, you're treading on very dangerous ground. If in the course of selling a medical test you don't even respond to the FDA for months on end, you're lucky not to be in jail.
Just to extract the key sentence from the letter: "These risks are typically mitigated by International Normalized Ratio (INR) management under a physician's care. The risk of serious injury or death is known to be high when patients are either non-compliant or not properly dosed; combined with the risk that a direct-to-consumer test result may be used by a patient to self-manage, serious concerns are raised if test results are not adequately understood by patients or if incorrect test results are reported."
Known risks of serious injury or death from claims about medical treatment are precisely what the FDA is designed to regulate, and in fact does regulate in many other contexts, and they are taking steps to do just that. Also 15 gets it exactly right.
The scientific underpinnings of our understanding of many human diseases are terrible. People follow hunches and test those guesses.
This has consequences for science also, here is an interesting overview by one prominent and capable human geneticist, which I read to be pretty pessimistic:
http://www.ncbi.nlm.nih.gov/pubmed/24074858
They also report likely ancestry, e.g. 20% Pacific Islander markers.
I hope these results come with little pictures next to them like a pineapple or viking warhammer or whatever.
15. These are not "in effect medical tests". They're a commodity technology (genotyping), and a summary of the results generated by computer from the scientific literature with light editing. Eisen clearly summarizes the alternate scenario-- simple technology outside US borders, clients interpreting results with Google. 23andme doesn't have anything sophisticated or secret to offer-- their advantage, such as it is, comes from having Google's experience to guide the creation of a summary document.
I am not certain of this, but I do not think that the aggregate information that 23andme will have will be useful.
I used their service too and it's one of those rare times I'm happy my results were boring. I also did the National Geographic one and was disappointed to know that my ancestors were the first to leave Africa. Translation: I'm as white as you can get! I had an 80% chance of being Eastern European. My husband's was so much more interesting with stories about nomadic tradesmen.
If you want to pile onto 23andMe, they also got an absurdly broad patent for being able to select donors for babies with reduced risks of passing on genetic diseases.
These are not "in effect medical tests". They're a commodity technology (genotyping), and a summary of the results generated by computer from the scientific literature with light editing. Eisen clearly summarizes the alternate scenario-- simple technology outside US borders, clients interpreting results with Google. 23andme doesn't have anything sophisticated or secret to offer-- their advantage, such as it is, comes from having Google's experience to guide the creation of a summary document.
I'm not saying they have anything secret or sophisticated. I'm saying they very clearly marketed the product as having medical implications. They described the results as "health reports". The point of using the product, their marketing material suggested, was to find out if you might be at risk of medical problems. Indeed, they wouldn't have needed all the disclaimers if they weren't doing that.
On the limited reading I've done about this, I am totally on the side of the regulators here. The conspiracy theory line I think is largely horseshit and on a continuum with the kind of thing that makes people believe that it's just self-interest that makes doctors et al. say that homeopathy is crap.
Homeopathy works great against dehydration.
Does anyone know about the reliability of the "ancestry" claims 23&me makes?
I have heard, from someone in a position to know, that the "X% Neanderthal" number it gives you should be taken with a grain of salt.
In other news, after about a week of feeling almost healthy, I've got a new cold, and am feverish and achey. I've been sick more or less constantly since October. Grr.
21. So if the marketers stopped talking, then the way this cheap new technology is regulated should change? Problem solved! Put big "For novelty purposes only" stickers on the package.
Then clients can find this web page with Google:
http://evolution.berkeley.edu/evolibrary/article/medicine_05
That's the alternative.
The general pattern of interaction over the year or two before the FDA finally got exasperated was as follows.
FDA: We have some concerns that your advertising is misleading. And that giving people "their test results" without any sort of doctor or genetic counselor being involved could lead to unintended consequences.
23andMe: Information wants to be free.
FDA: We have some concerns that your advertising is misleading. And that giving people "their test results" without any sort of doctor or genetic counselor being involved could lead to unintended consequences.
23andMe: Oh, that's silly.
FDA: We have some concerns that your advertising is misleading. And that giving people "their test results" without any sort of doctor or genetic counselor being involved could lead to unintended consequences.
23andMe: Be quiet! We're collecting data! We'll jump through these hoops when we have more data.
I'm on board with the conspiracy theories about doctors trying to shut this down. Giving patients a direct path to information is a huge threat to the medical establishment. The medical establishment feigns a lot of concern about preventing patient confusion or whatever, but I think its mostly about protecting their turf.
They are going to do the same thing when IBM's Watson finally becomes smart enough to start dispensing reasonable medical advice. When that does become available, doctors will be using it all the time, but if you want access you are going to have to go to a clinic and wait for your appointment.
Fuck that. I want access.
They are going to do the same thing when IBM's Watson finally becomes smart enough to start dispensing reasonable medical advice.
Oh no, never? That's really soon!
I haven't formed a clear opinion on this yet because most of what I've read on both sides pisses me off.
13: I agree that if 23andMe was as ridiculously unresponsive as it seems from the FDA's letter, they shouldn't be surprised to be shut down.
On the other hand, this seems like bullshit too:
For instance, if the BRCA-related risk assessment for breast or ovarian cancer reports a false positive, it could lead a patient to undergo prophylactic surgery, chemoprevention, intensive screening, or other morbidity-inducing actions, while a false negative could result in a failure to recognize an actual risk that may exist.
If doctors are prescribing or performing these things on the basis of a 23andMe report alone instead of independently checking their work (or having a better-valudated test done first), then it seems like it's the doctor who is doing something very wrong.
Wow, I saw a ton of outraged people on twitter when this happened, but none of it saying "This is the work of doctors, those bastards, trying to get rid of competitors." It was either "How can they do this when there's all these unregulated herbal supplements!", or the "information wants to be free" crowd.
IOW I pretty much agree with 29. When I read a rant on how ridiculous the FDA is being, I mostly agree. When I read a rant on how ridiculous 23andMe is being, I mostly agree. I don't even think that's an inconsistent attitude.
FWIW the one person I know who does policy analysis at the FDA thinks that the FDA is being ridiculous.
31: As far as I can tell it's 23andMe and the FDA both being a little bit stupid, not anyone being malicious. But if it is malice it seems like physicians have the most to gain.
OTOH some patients really are stupid and they don't deserve to die for their stupidity.
33 - You mean, aside from the investors in 23andMe, presumably.
Oh no, never? That's really soon!
It will never be perfect, but it doesn't actually have to be that smart to be a better diagnostician than your average doctor.
Its the same reason I think cars that can drive themselves could have a decent future. You've seen how shitty people are at driving, right?
I've been sick more or less constantly since October. Grr.
When did code academy end? I know that I often feel like I have a constant low-grade cold for a while after I finish something that's both mentally and emotionally demanding.
My sympathies, it's no fun (and just gets boring after a while).
I had a cold last weekend, was fine starting Monday morning, and now I can feel myself getting sick again. Why do I have to feel crappy on my own time?
If the FDA would let me have access to Watson, maybe I would know.
It's normal to respond to all of Spike's comments here with terror and dread, right?
I don't think 23andme is necessarily super-sketchy, but they're not doing much of anything to dispel the possibility.
I'm torn between being pro-regulation and anti-doctor.
When it comes down to it my sympathies are with 23andme, because I wanted to use their service and now I can't.
21. So if the marketers stopped talking, then the way this cheap new technology is regulated should change? Problem solved! Put big "For novelty purposes only" stickers on the package.
In simplistic terms, yes. In practice I think the firewalling from medical claims should be stronger than "for novelty purposes only" or equivalent, but I really don't see why this is controversial. If you sell a sugar pill as sugar, you don't get regulated as medicine. If you claim it can cure things, you do.
25: They already did this - as reported by multiple organs, though apparently gone now, their site said "for education and entertainment only and not for diagnostic use".
as reported by multiple organs
I heard their Director of Communications, Hugh Jorgen, had, like, 30 goddamn organs.
I'm getting really sick of sitting in airports waiting for delayed flights. I should institute a "no travel in winter" policy.
An Iranian friend of mine who was told by 23andme that he's part Jewish was so delighted by the news that I can't think the company is all bad.
48: it is the worst. Especially since all flights are routed through Chicago by law.
||
First time in the emergency room for years. Kids: don't be like Newt, and run while carrying glass soda bottles. Or at least don't trip and fall on them. But he's doing fine -- they bandaged him up and he is waiting for stitches with bored aplomb.
|>
51: yikes. Glad he's relatively well off.
Yep, I get to spend the night in a hotel near O'Hare. Whee.
Better than an emergency room, though. Hope he recovers quickly.
If he can get Vicodin, I'll buy it off him.
Hard to tell. I did tell him that chicks dig scars.
||
Great review of a new book...
All this for me--and I wasn't even that good. Translate that into Latin and it could be on the Palin family's coat of arms.
|>
58 was going to be my comment, but I thought hitting on Newt was a little low even for me. So I'll wait and see how his scar turns out first.
Further to 59: it gets better.
Ack! Poor Newt.
On topic, I'm with 43.
Started on Nero Wolfe. I can't decide if Archie annoys me or not.
You're the one who didn't like Bertie Wooster. Archie probably would annoy you.
Philistine.
It's always fun to play "where's my luggage?" at O'Hare.
Oh. They cleared that up: "we will send your luggage tomorrow; you can't have it tonight." That's unexpected and unpleasant.
Ah, O'Hare. My sympathies on having to deal with it.
I've dealt with the same situation enough times before. They always let me pick up the bag, though.
Hello Hilton, my old friend. I wonder how much I'll be shelling out to avoid waiting for hours for a hotel voucher.
Ooh, an hour-long line to check in at the hotel. That's a cute new feature. Are they trying to make O'Hare even worse?
Why is each person spending 10 minutes at the counter? Hotel check-in should take all of 45 seconds. Grrrrrr.
They do give out a prize for world's least interesting liveblogging, right?
Yes, and you're on track for it. Don't stop now.
Maybe 23andMe can help isolate the "boring" gene.
Everyone's home safe with lots of stitches. No important damage, but he's going to be a lefty for a week or so until the dressings come off.
Solidarity. Or maybe because she wanted a scar to attract Megan.
35: Why would 23andMe's investors benefit from the company needlessly antagonizing the FDA, or from the FDA shutting down the company?
80: If you think you might get to the gate late, you check bags.
Even the expedited security line is really long here. I think I'm going to miss my rescheduled flight.
essear's rigorous exercise in maximally anxiety and unhappiness-producing prose is a powerful, if difficult-to-read, statement on the amazingly flexibility of liveblogging as a literary form.
At least they didn't make a hashtag out of your trip, essear. What a world, what a world.