Lying with statistics: relative risk vs. absolute risk

Apparently, the theme of the day is Mike Adams the “Health Ranger”:http://www.newstarget.com.

Pat Sullivan alerted me to “this article”:http://www.newstarget.com/019368.html by the Health Ranger which alerts people to a valid concern in the reporting of statistics. This is another case where you should really read the source before reading the rest of my comment.

My comments:

Yes, this is a valid concern. In fact, I believe that everybody possible should be alerted to the different ways statistics can be used and misused by anybody. When reporting (and by reporting I mean publishing or submitting results to the FDA, since this is my field of specialty), best practices usually require reporting of absolute risk as well as relative risk. These are two different pieces to the risk reduction puzzle, and both are required for a more complete picture.

However, Mike Adams errs in, again, painting an entire industry with an overly broad brush stroke, and, specifically, he mischaracterizes the Herceptin® buzz.

Of course the media was all abuzz when Herceptin hit the market. However, Genentech has been pretty up front about when Herceptin is effective. Go to http://www.herceptin.com. In the title of the front page, you’ll see “HER2+ metastatic breast cancer treatment.” In the middle of the page, you’ll see a link to how to get tested. As it turns out, Herceptin is not just a drug but a drug-diagnostic combination. It only is effective against so-called HER-overexpressed breast cancers, and the diagnostic tests exactly this. If the test comes back negative, then Herceptin is not recommended because of futility. (Some doctors give it anyway, but that’s just dumb and expensive.)

But, again, I think he has a point. Several recent high-profile seemingly negative CAM studies have been touted as proof that certain popular remedies (e.g. echinacaea) and supplements (e.g. Vitamin D, gluthianone) are not effective when used for their popular indications. “David Gumpert”:http://www.thecompletepatient.com and “Mark Schauss”:http://www.markschauss.com have more details on the recent supplmenet studies, and I’ve “remarked before”:http://www.randomjohn.info/wordpress/2006/04/25/defeat-snatched-out-of-the-jaws-of-victory-study-conclusions-and-media-reporting/ on the echinacaea studies.

In an email exchange with Pat Sullivan over this issue, I had the following to say:

Now, for the difference in reporting statistics. I really think it depends on the person. I know there is a growing interest in herbal medicine at the FDA. There are guidelines now for submitting herbal preparations through an approval process similar to that used for pharmaceuticals. This is, of course, a riskier proposition because you can’t patent, for example, the use of echinacaea as a remedy, but you can patent an extraction and preparation process. I think the last estimate I heard is 150 applications [are in process].

Researchers are also trying to navigate the idiosyncrasies of researching the healing and restorative effects of natural products. It’s nowhere near as refined as the pharmaceutical research process, but it’s off the ground. So sound information is starting to be reported.

So then there are a couple of questions: 1) is the information accessible, and 2) is it being reported correctly? To 1), I have to say theoretically yes/getting better, especially given the free-flow of information due to the internet and the attempts that the government has made to make this information available. However, with 2), I’d have to say that it isn’t. In some cases, it has gone as far as Mike Adams has stated, with selective statistics being reported to make one’s own research or compound look good. However, I’d also say that to paint all practitioners and advocates of Western medicine with this brush stroke is over the top.

There are subtler biases as well, such as the tendency to play up negative CAM studies (such as Orac did with the echinacaea study) even in the presence of unanswered criticisms of the study (it takes more than statistics to design a study well).

In short, I think it pays to be well-informed. Read the package insert for all unfamiliar drugs. When the pharmacist asks that question “do you have any questions for the pharmacist?” Check “yes” and sit down with her or him and go over the expected effects and side effects. If you can navigate the jargon, get a copy of the package insert that should accompany the drug, or try to find it on the internet.

For example, last year during flu season doctor wrote me a prescription for Tamiflu. I decided to go ahead and get it filled and take it. The drug made me feel just awful. I looked up the “insert”:http://www.rocheusa.com/products/tamiflu/pi.pdf on the internet and found that the measure of efficacy was a reduction of a median of 2.5 days of flu symptoms. However, the drug made me feel worse than the flu. On that basis, I decided that the drug is not a good strategy for me. (Maybe it is for someone else, or under a more severe form of the flu.) The package insert (or product information) is regulated content, and is much more objective than marketing materials.

In short, good, solid, objective information is often available for “conventional” pharmaceuticals. It is less available for natural compounds, but efforts by the FDA and NIH (and academic centers both here and across the pond) are making better information on these products available as well. The trick is to find it and apply it correctly.

And be watchful for abuse of statistics, wherever it may occur.