Placebo effect – what good timing

A recent study suggests that the placebo effect is real.

I want to see more.

Thimerosal

So, commenter “Pat Sullivan, Jr.”:http://www.patsullivan.com (author and CEO of “Jigsaw Health”:http://www.jigsawhealth.com/? Update: Pat clarifies his role with Jigsaw Health in the comments. Thanks Pat!) has “asked me to weigh in”:http://www.randomjohn.info/wordpress/2005/09/15/ranting-and-raving-again-about-healthcare-and-the-magical-clinical-trial/#comment-60 on the thimerosol debate. For the uninitiated, thimerosal is a compound added to vaccines to increase shelf life. One of its components is mercury, a heavy metal which the FDA(Food and Drug Administration) and EPA(Environmental Protection Agency) both warn about on a regular basis in connection with pollution and fish. The debate is between the sides that believe that thimerosol increases incidence of autism and the side that insists that thimerosol is completely safe. Both sides have their salvos of studies that “confirm” one of the hypotheses, and both sides have contigents of statisticians and scientists ready to pick apart the other side’s studies in any way it can.

Personally, I have no fscking clue whether thimerosal leads to autism. I have trouble following the studies and arguments against studies, and quite frankly I don’t have the time to. As a father of a 2-year-old and father-to-be, this is damn frustrating because even the doctors don’t seem to know enough about the subject.

Even more confusing is why people are defending thimerosal so vehemently. I mean seriously, it’s just a compound. It’s not somebody’s son, or a production of literature, or prototype of the next generation flying device. It’s just thimerosal. It’s designed to increase the shelf-life of vaccines (or maybe the discovery of that use was a happy accident). Certainly with our great advances in science there are other ways that vaccine shelf-life can be enhanced, or perhaps the vaccine distribution system can be modified so that shelf-life is not as much of a factor. So, why not take it out? Our regulatory agencies are so convinced that heavy metals are hazardous, except when they are used in thimerosal? I mean, Crestor has warnings and dose restrictions on the basis of just a few cases of rhabdomyolysis; why should we treat thimerosal any more leniently?

I’m not advocating throwing away any compound that raises questions. But this particular one has been studied for years, and has two camps that don’t relent. The studies are less illuminating than confusing. It doesn’t answer any questions in my mind, and I’m not wanting to take the chance if I don’t have to. (Smallpox vaccine during a potential outbreak is a completely different story.)

While we’re on the subject, why the hell are we giving our children 3 or 4 vaccines at once? Can’t we space it out a little bit? That’s a lot to ask of our children.

If anyone can point me to data (not results, but actual datasets) on the matter, I’d love to see it. Maybe when I have a few hours to kill, I can do my own exploration.

Hornet’s nest

Thanks to commenter Pat Sullivan, Jr. in the “rant on clinical trials”:http://www.randomjohn.info/wordpress/2005/09/15/ranting-and-raving-again-about-healthcare-and-the-magical-clinical-trial/, I’ve finally found out who “Orac”:http://oracknows.blogspot.com is, and it seems I’ve stumbled into a bit of a hornet’s nest. Quite frankly, though I’ve viewed the Quackbusters for a long time as an example of the worst that science can produce (given that, in addition to science and quality of life improvements, science generates territorialism, credentialism, and a whole host of other things that are neither positive nor “scientific”), I never thought I’d be tangling with one of them. But you know what, if this continues I’m not going to back down, and here’s why.

Exposing medical fraud or any type of fraud is a noble cause. I’m glad there are people who care about people enough to spend their precious time finding those who, for the sake of only making money, will sell worthless or harmful products.

The problem comes in when “quack-busting” gets confused with “my healthcare system will kick your healthcare system’s ass.” Further confounding with a particular virulent form of “skepticism”:http://www.skepdic.com has occurred, at least in D — er, Orac’s case. (Roll your mouse over some of the links at one of “Dav…, er, Orac’s responses”:http://oracknows.blogspot.com/2005/09/little-criticism-directed-oracs-way.html to my post about clinical trials. Try the “confirmation bias” and “selective thinking” links.)

Let’s break this down a little bit further. There are some assumptions in drug development that are very necessary:

_If it isn’t documented, it isn’t true_

This is fine for the regulatory where you’re essentially trying to prove beyond a shadow of a doubt that your compound works, and that it isn’t a fluke, and that thirty scientists, doctors, and biostatisticians are going to sift through your work with a fine tooth comb to make sure you’re not out of line.

What it’s not fine for is defining what truth is. For most of us, 99.9% of life is not a regulatory environment. Truths are not documented, and are probably not documentable. Who cares, anyway, besides the skepdics? (To be distinguished from ordinary skeptics.) Aristotlean logic is not the only path to truth, and not all destinations of Aristotlean logic are truth. The results of logic are at most as good as the assumptions (and there are _always_ assumptions).

_The placebo effect has no clinical value_

Again, this is a fine assumption for a regulatory environment, where you don’t want someone to claim their $3 a pop pills work when an inactive pill work just as well. Out in the real world, (oh, and by the way, doctors use the placebo effect _all the time_ in the real world), perhaps we don’t use the placebo effect skillfully enough. The placebo effect can be thought of as simply living in a healthy way — diet, exercise, thoughts, attitudes about life, seeking out situations that fulfill and support health. The sugar pill that we’ve all heard of is a part of the placebo effect, and most “sources”:http://en.wikipedia.org/wiki/Placebo_effect “stop”:http://skepdic.com/placebo.html “there”:http://pubs.acs.org/hotartcl/mdd/99/aug/mysterious.html. (BTW, I actually found skepdic.com’s entry somewhat helpful on this, when I keep the scope and intent of the article in mind.)

And what about this “all the time in the real world.” Have you ever seen a cancer doctor treat a patient? A lot of times they’ll, upon entry, exhibit a more sanguine approach than otherwise may be warranted. They are using the principles of the placebo effect. Or the pediatrician who gives antibiotics for viral infections or recurrent ear infections, and so forth.

Every applies the principles of the placebo effect unknowingly. The same is true for the placebo’s evil twin, the “nocebo effect”:http://www.washingtonpost.com/ac2/wp-dyn/A2709-2002Apr29. (Read about the study between aspirin and another blood thinner. It’s very enlightening.) The stories of the nocebo effect warn us that these principles are not to be taken lightly, and, since we’re not in a regulatory environment most of the time, we’re free to examine our own application of the placebo effect.

There are more, but I want to move on to some assumptions that are not so necessary:

_Your experience doesn’t matter in science_

Read about it “here”:http://skepdic.com/pragmatic.html, under the fancified name “pragmatic fallacy.” The confirmation bias discussion reinforces this. While it is fallacious to assume that what works for me works for you or anyone else, it’s equally fallacious to assume that what doesn’t work for me will not work for you. Things get even murkier when we start talking about what is True. Few things are True for All People — a few principles of physics and chemistry that seem to describe how the stuff of our existence works. (And who knows, maybe they’ll be false tomorrow, the way that Newtonian mechanics became “false” when Einstein performed and published his work on relativity.) Does this mean that we can ignore everything else? Absolutely not. Things that are not on the blessed list of Truths for All People are just as real as those things that are. Put another way, just because something is not currently amenable to scientific inquiry doesn’t mean it’s untrue, ignorable, or unreal.

_If it ain’t Western medicine, it’s quackery_

This is the interpretation I get out of these quackbuster sites. Even the “Wikipedia entry”:http://en.wikipedia.org/wiki/Quackery is “disputed”:http://en.wikipedia.org/wiki/Talk:Quackery, in part because of the seeming huge overlap between the discussion of the practice of alternative medicine and the deliberate theft of money through the use of deception. Rarely will you see a quackbuster praise a study on alternative medicine unless they show the “superiority” of a conventional treatment.

_The coevolution of methodology and knowledge_

This is topic which I was reluctant to bring up, but I think it’s important. I’m reluctant to bring it up because I haven’t found a way to adequately express my misgivings on the subject. It goes something like this: our pharmaceutical process is a system, and that system includes the methodology designed to sift through and confirm the safety/efficacy of our pharmaceutical compound. Alternative medicine practices, such as homeopathy (alternative medicine really isn’t a system in itself, but consists of many such different systems), have their own theories and practices which support the system. Whether alternative medicine systems might be found lacking by the gold standard randomized controlled trial (RCT) is one matter, but the fact remains that the methodology supports one particular system the best. This isn’t to say that the RCT should be abandoned outside of pharmaceuticals (or other systems, such as modern agriculture, that have co-evolved with methodology), but that we should examine the “inherent biases”:http://www.patsullivan.com/blog/2005/09/autismmercury_c.html in systematic study[1].

And here, the inherent bias is in failing to critically examine the _system_ of pharmaceutical production (and, more generally, production of science). Just because this system has brought us antibiotics doesn’t mean that products not discovered or even refused by the system cannot assist in healing. It also does not mean that other systems cannot produce quality healing products. This may make Orac barf, but that’s probably for the best.

At any rate, we should thank those people who honestly try to protect us from scams. And we should be critical of those who take it way too far, or who use this quackbusting cause to assert their own beliefs about the world.

fn1. This article is ostensibly on the connections between mercury and autism, but it has some interesting findings on the connections between RCT(randomized controlled trial)s and funding, selection bias, and other definitely non-scientific factors.

AIDS is a complicated disease

The AIDS virus might be weakening.

I don’t know a lot about the AIDS phenomenon, so I don’t write a lot about it. I simply don’t have much to say. I do know that there are differing opinions on the matter, and I know this disease is far more complicated than any of us know. Fortunately, some of the surprises are positive.

Anecdotal evidence

Interesting timing, given that I’ve had a couple of rants (gotta stop ranting!) on this very topic. The ADHD drug Strattera, which has been approved by the FDA and other regulatory agencies, has been linked with suicidal thoughts. Warning labels have been updated on the basis of 11 cases.

bq. It was important for parents to be aware that it can occur and to discuss any unusual symptoms with their child’s doctor.

That would be Eli Lilly, the drug’s manufacturer, saying that people should take responsibility for their on healthcare and that of their children.

Now, let’s ask those 11 families. Which evidence is more important? Anecdotal or clinical trial?

Should Strattera be taken off the market? Should we use herbal remedies instead of Strattera? Let me quote Serge King:

bq. The whole point of healing is to get healthy, not to prove a method. Use what works for you.

And if this runs afoul of the “pragmatic fallacy”:http://skepdic.com/pragmatic.html, then so be it.

Great, another doctor who likes to deliver “Respectful Insolence”

I’ll skip over some of the nonsense “here”:http://oracknows.blogspot.com/2005/09/little-criticism-directed-oracs-way.html and jump straight to the meat of the matter. (And forget about the _ad hominem_ attacks in the inane comments, which numbered 4 at the time of this post. It’s best that way.)

bq. (although I would disagree with his qualification of his dislike in which he said that there’s “a lot to Kevin Trudeau that’s admirable”–I consider Trudeau to be a scam artist)

At the age we were still building towers with erector sets, the boy was building businesses. I don’t know about you, but I find that admirable. Too bad he runs scams today — like most other people, I don’t find that admirable. I don’t live in a black and white world, so I can easily make comments about admirable qualities in criminals.

Oh, and by the way, I don’t waste my time on the Usenet groups. Maybe if Orac skipped the newsgroups, especially in one he obviously hates, he’d have enough time to think a little.

And, should he skip tonight’s misc.health.alternative session, he might think a little more about these points.

bq. If you accept vague and/or poorly documented anecdotes and testimonials as sufficient evidence that an “alternative” therapy “works,” you just might be an altie.

bq. If you dismiss every well-designed randomized clinical study that failed to show a benefit for an alternative medicine or therapy over placebo control as either not proving that the therapy is ineffective or as having been manipulated by nefarious forces (conventional medicine, the pharmaceutical companies, the government, etc.) to produce a negative result, you may well be an altie.

Straw man, anyone?

Now, let’s take this even further. Say there were two candidate compounds: the pharmaceutical Fixitol and its competitor Herb Remedy. An adequate well-controlled randomized trial shows superiority of Fixitol over Herb Remedy. In addition, Ms. Jones up the road swears up and down that Herb Remedy cured her illness, and is willing to be quoted in the newspaper for saying it.

Who’s right?

Let’s examine Orac’s answer:

bq. Uh, no, John, it’s not “bias” to understand that testimonials are inherently less reliable than clinical trials in identifying which treatments work and which do not and then putting them into their correct place in the hierarchy of medical evidence. It’s just good science. And, no, John, I did not say that anecdotes are “bad.” I merely pointed out that they are far less likely to be widely applicable than the results of clinical trials.

Let’s see how less widely applicable anecdotes are: Nozitol was an approved compound for 3 years. It is blamed for 40 deaths, and on the basis of that, was removed from the market.

Not so simple, eh? Thousands of people tested in clinical trials, and 40 suspected deaths results in a wider application than all those other people. Oh yeah, let’s reveal the true name of Nozitol: Baycol.

bq. Finally, contrary to John’s characterization of what I said, I have not entirely dismissed anecdotal evidence. Actually I have said repeatedly, that such evidence, if better documented than just a testimonial, can be useful to guide further research.

At least we have some agreement.

bq. Testimonials are often highly subjective, and, of course, practitioners can and do pick which testimonials they present.

Ok, to be fair to Orac here, I am picking and choosing which statements to respond to. And, we might point out that he’s doing the same to me. I’m picking which points of his to which I’m crafting a response, and he’s, well, picking which points of mine to which he crafts a response. Hmmmm… this is starting to sound a little subjective.

I challenge Orac to prove for once and for all that, if he were able to wave a magic wand and remove testimonials from the pharmaceutical process, that it would be completely objective. In fact, I’ll make it easier. I challenge him to prove to me that an experiment with a high-energy particle accelerator is entirely objective. Whether you choose to believe testimonial evidence is entirely up to you. Quite frankly, I’m disinclined to believe testimonial evidence, but I also have to question the worth of a product if a company cannot even pay someone to speak for it.

Ok, so maybe the standard is too hard. Because, I assure you, I’m waiting with tons of examples of where subjectivity creeps in the drug development process, and, perhaps even nastier on my part, I believe that’s ok. I think it’s perfectly fine to have subjectivity in the pharmaceutical development process, and, even more, I believe that it’s impossible, despite the advances of my field, to expunge it.

Now, taking back the magic wand, will you go down to the marketing departments of Pfizer, Merck, Bristol-Meyers-Squibb, and GlaxoSmithKline and tell them to stop using testimonials? (And even their regulatory departments, if we’re talking drug development.) Please carry along a video camera, and put the results up on the internet.

Let’s get back to picking and choosing which points of Oracs I want to refute.

bq. John did have a point in mentioning that clinical trials may not adequately predict the response of any single individual to treatment. That does not invalidate my point, however, because anecdotes are considerably worse than clinical trials at such prediction.

Ask the ghosts of the 40[1] or so people who died as a result of the side effects of Baycol how well clinical trials predicted their response to treatment. The “anecdotes” of each one of those people is closer to the 39[1] other people than to the thousands in the clinical trials. I’d say they both are pretty bad. The whole point of the post to which Orac responded was how often this effect is minimized. I ought to make him go write the bolded part of the post on the chalkboard for 45 hours (approx the length of a 3 hour semester class) before he’s allowed to practice as an academic surgeon further.

bq. Note the last sentence. That’s me pointing out the proper weight that should be given to anecdotal evidence, which is not zero but is much less than the weight that should be given to a well-designed clinical trial. I suspect the problem here is that John seems to be equating testimonials and anecdotal data.

Please, oh Orac, saint of medical scientific research, tell us what proper weight should be given to anecdotal evidence and clinical trial evidence. If thou wouldst but give us thine formula, that we may follow it.

bq. In addition, it isn’t more anecdotes that will help us predict more accurately the response of any individual to any given therapy. It will be clinical trials that identify factors that might help us predict which individual patients will respond better to which treatment. Indeed, that’s the whole point of genomic medicine and molecularly targeted therapeutics like Herceptin.

Ah, a vision! Thank you, oh Orac, for this wisdom!

Let me go one up on you: it will be anecdotal evidence, perhaps taken during a clinical trial, that will identify some subpopulations that benefit from a particular therapy. It will be a clinical trial that provides the evidence that the subpopulation really does benefit. And then when the drug is marketed for that subpopulation, some people will respond in completely unexpected ways and some people will respond within the 95% confidence interval. And the process will repeat. In the meantime, people will still benefit, and people will still be harmed by, conventional and alternative therapies.

Now that we got that out of the way, let’s get back to alternative vs. conventional medicine (if there is ever such a competition). I have a list of questions:

  1. How many people has conventional medicine cured?
  2. How many people has homeopathy cured?
  3. How many people has Reiki cured?
  4. How many people have African shamans cured?
  5. How many people has osteopathy cured?
  6. How many people has witchcraft cured?
  7. How many people has stoning people cured?

I’ll give you a hint: the answers are all the same, and these are all trick questions. And in fact you can put in any healing system. It doesn’t matter. Healing systems don’t “cure” anybody. At best, they help the body heal itself, and that’s the way it should be and the only way it can be. In some miraculous cases, there’s help from parts of another person’s body (although that’s dangerous and forces us to suppress the body’s own healing capabilities).

Well, do have a lot more to say about this:

  • Assumptions of statistics, which powers the evidence part of a clinical trial
  • More about the distinctions between testimonials and anecdotal evidence
  • More on the “my health care system will kick your health care system’s ass” syndrome
  • More on not living in a black and white world
  • More on why Robert Carroll’s system of skepticism has a lot of bullshit (which is the whole point of the skeptic’s skeptic’s dictionary category)

And I might have more time in the future to say it, and hopefully in a much more coherent way than I made my points here.

fn1. Lest you get picky and jump on the number 40, I fully recognize that there are undocumented cases of Baycol deaths, documented cases where there may have been predispositions to rhabdomylosis, and so on. It’s simply the best estimate I’ve seen. And it’s small compared to the thousands who took it during Baycol’s development, and behind this number is are people who died miserably and families who grieve their deaths.

Chinks in DeLay’s armor

He’s been indicted, and has stepped aside as majority leader.

One small step for accountability.