{"id":1650,"date":"2012-08-26T17:16:42","date_gmt":"2012-08-26T21:16:42","guid":{"rendered":"http:\/\/www.khpi.org\/blog\/?p=1650"},"modified":"2012-09-06T09:11:39","modified_gmt":"2012-09-06T13:11:39","slug":"lillys-new-alzheimer-drug-fails-its-major-clinical-trials-but-stock-goes-up-anyway","status":"publish","type":"post","link":"http:\/\/www.khpi.org\/blog\/lillys-new-alzheimer-drug-fails-its-major-clinical-trials-but-stock-goes-up-anyway\/","title":{"rendered":"Lilly\u2019s New Alzheimer Drug Fails Its Major Clinical Trials, But Stock Goes Up Anyway."},"content":{"rendered":"

In God we trust. All others must bring data.<\/strong><\/p>\n

Yesterday, our local paper reprinted a piece from the Associated Press with the title, \u201cAlzheimer\u2019s drug promising.\u201d\u00a0 To me, this was a typical \u00a0\u00a0example of what I call \u201cpress release medical reporting<\/a>.\u201d\u00a0 The question I asked of myself and of the article was, \u00a0\u201chow much is being promised.\u201d\u00a0 In my opinion, the answer is, \u201cnot very much,\u201d but to the investment community to which this news release was being reported, it was enough to raise the stock prices of the manufacturer, Eli Lilly.\u00a0 As a physician, I am not jumping up and down.\u00a0 As an individual whose family was savaged for over three decades by Alzheimer\u2019s disease, I am annoyed.\u00a0 Let\u2019s take a look at what is behind this commercial enthusiasm.<\/p>\n

For some weeks or months now, the financial news media have been abuzz about the impending results of drug studies by Eli Lilly of their new drug, Solanezumab, for the treatment of mild to moderate Alzheimer\u2019s disease.\u00a0 The drug is actually a man-made antibody to a protein that accumulates in the brains of people with Alzheimer\u2019s which may or may not actually cause the disease.\u00a0 An earlier version of such an antibody made Alzheimer\u2019s patients worse.\u00a0 The drug has been on the list of potential \u201cblockbusters\u201d which refers to the amount of money to be made and not necessarily to the amount of good a drug might do.\u00a0 These antibody drugs are proteins themselves and must be given intravenously. They are in the class of drugs costing multiple thousands of dollars per infusion, which generate big-time income for individual physicians, and which are playing a major role in bankrupting our healthcare system. \u00a0We need to be sure they work as claimed.<\/p>\n

The meat of the press release<\/a> actually stated that the results of the two large studies of 1000 patients each were negative: there was no evidence that the drug did what it was supposed to. When the results of the two studies were merged together, a claim is being made that there was a \u201cstatistically significant result\u201d for one of the presumably many things being measured.\u00a0\u00a0 Additionally, another \u201cstatistically significant result\u201d is claimed for a subgroup of patients with mild disease only. \u00a0\u00a0\u00a0Of course, Eli Lilly has every reason to want to make lemonade from this unsweetened fruit, and we have certainly not heard the last of this salvage effort.\u00a0 Indeed, Lilly has already received FDA approval for a diagnostic test to detect early Alzheimer\u2019s disease, so if approved, we can expect aggressive co-marketing. Allow me to suggest a few issues and concerns that are important to allow independent judgments.<\/p>\n

Secondary Analysis of Scientific Data.<\/strong>
\nScientists have long known that trying to extract more conclusions from an experiment than the experiment was designed to detect is fraught with hazard.\u00a0 The two studies mentioned above were presumably designed to have enough patients to be able to detect a clinically meaningful effect of the drug and yet they were not able do so.\u00a0 I do not have access to the details of the actual studies and do not even know if the two were identical. (If not, there is also the problem of comparing apples and oranges.) The reason that statistical science is used in the design and analysis of such studies is to make sure the primary results are not due to chance alone. If large numbers of different measurements are made on a large number of patients, there will always<\/span> be something that will give the illusion of being significant.\u00a0 The more subgroups you look for, the more likely you will find something sticking out, even if it is an artifact. For example, it might be the case that only young male vegetarians appeared to benefit from the drug.\u00a0 Unless, however, the study was designed to measure a potential difference in vegetarians, no claim of clinical effectiveness can yet be made.\u00a0 This is not to say that such secondary analysis is uninteresting or of no value, but you would not go to the bank over such a conclusion, nor should you inject it into your patient.\u00a0 This is statistics 101. It is also the basis for the statistics joke:<\/p>\n

If you torture the data long enough, it will eventually tell you what you want to hear.<\/p><\/blockquote>\n

What is being measured and does it make sense?<\/strong>
\nI have no idea at this time.\u00a0 I saw allusion in one business Internet site (where these press releases are appearing) to the \u201cMini Mental Status Exam (MMSE).\u201d\u00a0 This is a simple screening exam widely used by clinicians in everyday practice to detect a decrease in cognitive (mental) functioning. It includes some 8 questions or tasks such as \u201cwhat day is it,\u201d \u201cwhere are we,\u201d counting by 7s, naming a pencil and watch, repeating phrases, and the like.\u00a0 It is very useful for this screening purpose but I have no personal knowledge of its use in research. It is possible that blood tests or other physical studies were performed as \u201cproxies\u201d for what the real desired result of a patient and their family would be: better or independent function at home, remembering the names of your spouse or family longer, or perhaps delay of entry into a nursing home.\u00a0 Note that some traditional endpoints of other drug studies such as longer life are irrelevant here.\u00a0 Management of Alzheimer\u2019s disease is all about quality of life for the individual and for their family and family caregivers.\u00a0 These things are much harder to measure and even more difficult to attach a value to.<\/p>\n

Is the effect meaningful?<\/strong>
\nEven if the drug were shown to have a \u201cstatistically significant<\/span>\u201d effect on the entire group of patients who participated in the study, that is a far cry from having a clinically meaningful<\/span> effect.\u00a0 Even if there are no side effects (and there will be) the magnitude or duration of the effect may not be worth it. Statistics can detect differences that are invisible to the human eye!<\/p>\n

As an example for this article, I generated a hypothetical clinical study of the effect a new drug on Alzheimer\u2019s disease.\u00a0 I wanted to see if my new pill (I\u2019ll call it Wonder Drug- Extended Relief) could keep people with Alzheimer\u2019s disease out of nursing homes longer.\u00a0 Based on a study of the natural history of Alzheimer\u2019s by Jost and Grossberg, I \u201cenrolled\u201d 100 patients with random ages between 70 and 79 when most patients receive their diagnosis. In real life, the average time between diagnosis and institutionalization is 24 months with very wide variation. Therefor, as an example of how things might have turned out without treatment, I generated a random pre-institutionalization interval for each of my patients of between 14 and 34 months.\u00a0 Not surprisingly, with no treatment, my patients stayed home for an average of 23 months.\u00a0 I then assumed that Wonder Drug would add a single day<\/span> at home to each and every one of my study subjects. When I performed a statistical two-tailed paired t-Test on the hypothetical before-and-after results, I found that the one-day difference was highly statistically significant<\/span> at the P=.05 level!\u00a0 Even the most desperate or optimistic person would have to admit that this \u201csignificant\u201d improvement was absolutely worthless.\u00a0 If my Wonder Drug pill were one of the $100-Wonders, the patient and his family would better put the money to use enjoying their lives together while they could, or saving some of it for the education of the children or future Alzheimer care. Of course, any statistician worthy of the name can and will rip my example to shreds, but its conclusion, that some statistically significant scientific results are without meaningful social value, is unassailable.<\/p>\n

Why is all this important?<\/strong>
\nI once had a friend in the drug business brag to me that they could make a drug study turn out any way they wanted to by carefully selecting the patients to be studied and the things to be measured. He also claimed that the development cost of a new drug could be captured in the first few months after release. While there was a bit of hyperbole there, the core of the claims has some truth. There is tremendous pressure from all sides to get new drugs on the market, even if they do not work very well.\u00a0 Certainly the many scandals involving drug companies, withdrawn drugs, judicial penalties, examples of misleading marketing, and other kinds of pharmaceutical malfeasance give us abundant reasons to closely examine the drug development and approval process with an independent eye. Some would go so far as to say that even the Food and Drug Administration (FDA) that approves new drugs, but is funded by the drug companies, is not fully independent any more. We already know the FDA is not politically independent and therefore is vulnerable to the lobbying process to which so much money is devoted. Alas, even my beloved academic enterprise can no longer be assumed to be independent of pharmaceutical largess and influence.<\/p>\n

Evidence-based skepticism.<\/strong>
\nLet me be very clear that I have absolutely no evidence personally that Eli Lilly has done anything improper in the development of Solanezumab.\u00a0 I chose this press release to write about solely because it crossed my desk by chance. However, even this King of American pharmaceutical companies has had its days of ignobility in the development and marketing of drugs.\u00a0 In 2009 it paid a fine of $1.4 Billion for illegal off-label marketing of Zyprexia, including to people with Alzheimer\u2019s disease. (Off-label marketing means promoting a drug for a medical indication for which it has not been shown by research to be effective. That is to say, there is no statistical significance at all!) By total coincidence as I write this today, I am reminded that my own father suffered an adverse effect of receiving Zyprexia for off-label purposes, and which in my medical opinion robbed him of one of the few things in his life that still gave him pleasure.\u00a0 In 2006 Eli Lillly agreed once more to plead guilty and pay $36 Million for its illegal promotion of the drug Evista used for osteoporosis. I did not look further back, but other drug companies have recently paid even larger fines for their illegal activities.\u00a0 The drug company Abbot just paid a $1.6 Billion fine related to its marketing of the antiepileptic drug Depakote to Alzheimer\u2019s patients. \u00a0Are such fines now just part of the cost of doing business? Drug companies are desperately looking for some way to cash in on this common and terrible disease. Obviously, bringing drugs to market in the right way is vitally important to us all and demonstrably cannot be left to unregulated market forces alone.<\/p>\n

We of the public have delegated supervision of drug development to organizations such as the US Government, but also to many other public and private watchdogs. Even so, all of us have some responsibility to learn some of the language used in medical research and how the balance of adverse vs. beneficial effects of drugs is weighed.\u00a0 Otherwise we are collectively at the mercy of both legitimate and illegitimate purveyors of medical therapies.<\/p>\n

Enough of this for now.\u00a0 We will be writing more about how to protect ourselves from being over-managed by the health care system. Remember Hasselbacher\u2019s Law: It is just as bad to be over-treated as under-treated!<\/p>\n

Peter Hasselbacher, MD
\nPresident, KHPI
\nEmeritus Professor, UofL
\nAugust 26, 2012<\/p>\n

Share this:<\/h3>