Residue Reasoning

Back in 2014 a study was published, titled: Diet Drink Consumption and the Risk of Cardiovascular Events.  A copy of the study can be found here.  It's nothing daunting--a mere 6 pages long.   Though once published, it's pretty clear that the media didn't bother spending any time reading, let alone analyzing the study.  Instead, they likely skipped to the last paragraph and read the following:

"In conclusion, this study suggests an association between consumption of two or more diet drinks per day and adverse CVD (Cardiovascular Disease) events, as well as increased mortality."

That was enough to produce headlines like these:

Diet Drinks Raise Heart Concern in Postmenopausal Women
Diet Drinks Linked With Heart Disease, Death
The End of Diet Soda? Huge Study Links Aspartame to These Major Health Problems
Daily Soda Intake Increases Heart Attack Risk

It's worth noting that for the 3rd link above, the study makes no mention of what type of artificial sweetener was in the 'diet drinks' consumed, whether Aspartame or otherwise.  That last link is particularly amusing, since this study was strictly among postmenopausal women and yet this is posted on Men's Fitness. Well, let's take a minute and dig into the science behind and the findings of this study, shall we?

The method of the study was, very briefly, to round up nearly 60,000 women ages 50-79 who did not have pre-existing cardiovascular disease (e.g. coronary heart disease, heart failure, myocardial infarction, etc.) and have them submit a self-reporting questionnaire annually relating to their diet soda consumption.  This was continued for several years, and the resulting data was broken down into four categories of diet soda consumption: 0-3 per month, 1-4 per week, 5-7 per week, and >2 per day.  Researchers looked at all kinds of outcomes of these women over those years, primarily focused on the prevalence of 'cardiovascular events' (i.e. the onset of heart failure) and mortality.  Before we analyze the data presented by the study, let's first identify a few problems with how the study was structured:

1) self-reported studies are notoriously unreliable and inaccurate.

2) The grouping of the diet soda consumption groups is odd.  The questionnaire had 9 consumption groupings, yet the study reports on 4 groups--were some groups combined? or ignored?  By creating a "0-3 per month" group, researchers are assuming and asserting that consuming up to 3 diet sodas a month is negligible and equivalent to never consuming diet soda.  Also, the first three groups pretty well cover the full spectrum of consumption possibilities from 0 times a month all the way through 7 times a week (i.e. one a day).  Why, then, is there such a large gap between the consumption of group 3 and 4 up to "2 or more per day"?  What about all the people who consume 7-14 per week?  The reduction in groups from the original 9 down to the 4 reported-on gives license to manipulate the output data to artificially inflate the results you're interested in.  It appears that this highest consumption group was intentionally biased away from the remaining groups to help stratify the data and produce more stark results.

3) The annual questionnaire asked participants to self-report on consumption activities (from memory) for the most recent 3 months.  What about the other 9 months of the year?

Now, as far as the data and conclusions are concerned: yes-- the "2+ a day" group did show higher rates of both CVD and mortality.  But here are some other equally valid conclusions that could be made using the very same flawed dataset:

1) Consuming 1-7 diet sodas per week reduces mortality up to 20%.
2) Consuming 1-7 diet sodas per week reduces cardiac death by 20%.
  
There you have it-- MODERATE DIET SODA CONSUMPTION SAVES LIVES.  Why didn't we see these in the headlines?

Another glaring issue with this study is the authors own observation of the following:

"There was increased prevalence of diabetes and hypertension in the women who consumed the most diet drinks, and they had a higher BMI (Body Mass Index), a greater proportion of smokers and higher calibrated energy intake."

It seems pretty scientifically irresponsible to suggest an association of CVD events and mortality with the high consumption of soda when that same consumption group has noticeably higher rates of smoking, obesity, calorie consumption, etc.  Perhaps a more appropriate conclusion for this study would have been:

Women with higher BMI and higher rates of diabetes, hypertension, and smoking are associated with adverse CVD events as well as increased mortality.  Oh, and they tend to consume more diet soda, too (maybe because they are trying to manage their weight?).

This is a prime example of what is most likely a spurious relationship:  Where higher BMI, diabetes, hypertension, smoking is present, both increased mortality and increased diet soda consumption are observed: soda consumption is the lurking variable.  To the author's credit, he does state the need for further studies to confirm or disprove the link between diet soda consumption and CVD events or death and also to evaluate whether there is a causal relationship between diet soda consumption and CVD risk.  In other words, this study of over 60,000 women didn't really prove anything and, as the favorite saying goes, correlation does not imply causation.  But the media didn't bother to report "Diet Soda Consumption: More Studies Needed."

Just for kicks, if we just want to look at data correlations within the study, here's a few more "conclusions" (for entertainment purposes):

1) Lower diet soda consumption correlates with higher college graduate rates (headline: Diet Soda Linked To College Dropouts!)
2) Diet soda consumption rates correlate with higher income (headline: Drink Diet Soda, Earn More Money!)
3) Diet soda consumption rates correlate with lower alcohol consumption (headline: Drinking Diet Soda Helps Quitting Alcohol)

This study is yet another example of borderline junk science getting gobbled up by the media and converted into unsubstantiated headlines that people base their life decisions around.

Back in 2010 the FDA sent a warning letter to Diamond Foods Inc. regarding their walnuts.  Here's the original copy.  Much of the internet, especially the 'healthy eating' community, proceeded to lose its collective mind with statements like:

FDA Ignores the Science
Walnuts are DRUGS! FDA Makes Bizarre Claim
FDA Says Walnuts are Illegal Drugs

You get the idea.  First of all, something the general public may not realize is that a warning letter from the FDA is nothing novel.  It happens all the time—Diamond Foods isn't special.  The FDA publishes all their warning letters on their website, and by the looks of it they issued approximately 600+ letters just in 2015.

Before we dive into the perfectly rational and justified reasoning by the FDA, let's explore a hypothetical:  Suppose you lived in the early 20th century and a reputable pharmaceutical company came to town selling a brand new elixir that would treat a strep throat infection.  There's only one problem—the elixir comprises an 'active ingredient' drug mixed and diluted with a solvent, which is poisonous.  By the time anyone realizes the true nature of this new drug, over 100 people are dead from ingesting it.  The sad reality is that this is no hypothetical—this is the true story of Harold Watkins and his deadly Sulfanilamide elixir of 1937.   Watkins used a substance proven to help treat a streptococcus infection, Sulfanilamide, diluted it with diethylene glycol, and topped it off with some raspberry flavor to create his new drug product.  Watkins, despite being the head pharmacist and chemist of then S.E. Massengill Company, was unaware that diethylene glycol is poisonous.  No animal or human testing was done with the new mixture before they set out marketing the new drug.  Tragedy ensued.  This event was a catalyst for the organization of the FDA as we know it today.

What does this have to do with walnuts?  A key role of the FDA is to prevent a person or company selling a drug (or device) from either A) making unsubstantiated marketing claims about the safety or efficacy of the drug (or device), or B) making unsubstantiated claims about the indications of use of the drug (or device) to treat one or more particular disease states.  On their website, Diamond Foods was in violation of both of these relating to the assertion that their walnuts contain Omega-3 fatty acids.  One example is the claim that Diamond foods made:

"Studies indicate that the omega-3 fatty acids found in walnuts may help lower cholesterol; protect against heart disease, stroke and some cancers; ease arthritis and other inflammatory diseases; and even fight depression and other mental illnesses."

Here Diamond Foods is making the claim that their walnuts can be used to help treat or prevent heart disease, stroke, cancer, arthritis, depression, and other mental illness—this is called an indication for use.  So let's be clear—if you are selling a substance which is ingested which you claim can be used to treat or prevent various diseases or illnesses, no matter what that substance may be (i.e. walnuts), you are making that substance a de facto drug.  In this case, it was not the FDA that made the claim that walnuts were drugs, it was Diamond Foods, Inc.

Diamond Foods also got dinged for making claims about the efficacy of their product by stating:

"[T]here's good evidence that omega-3s can increase HDL (good cholesterol), further reducing the risk of stroke and heart disease."

Here they are not only indicating that their product can treat a disease state, but specifically that their product would reduce the risk of stroke.  This is a description of the efficacy of their product to reduce the occurrence of a disease state.  Once again, Diamond Foods is de facto claiming that their walnuts are a drug.

Now, one primary complaint by the nay-sayers is this: "But lots of studies show that Omega-3s have been proven to help with those things!"  This is largely true.  But where are the studies that show that Diamond Food walnuts specifically contain those Omega-3's?  How do we know that there isn't another substance in walnuts that confound the positive effects of the Omega-3's?  Where are the randomized clinical trials showing that by ingesting more Diamond Food walnuts than a control population leads to lower occurrence rates or better outcomes of those disease states listed above?  For all we know, and for all the FDA knows, these studies don't exist.  This is why Diamond Foods got in trouble.

It is an unfair logical leap for Diamond Foods to assert that, since Omega-3's help treat those diseases and their walnuts presumably contain Omega-3's, therefore their walnuts must help treat those diseases.  Harold Watkins made a similar logical leap with disastrous consequences.  Obtaining approval from the FDA to market a new 'drug' or device doesn't always require a big randomized human trial, either.  If there exists a predicate approved product (i.e. one that basically already does the same thing) then all that remains is to provide sufficient bench testing data and written justification to show the equivalency of the two products in safety and efficacy, and the newer product may be approved.  This is accomplished through a 510(k) filing.  Diamond Foods may have been able to obtain approval for these types of claims through this relatively straight-forward avenue.

So what conclusions can we draw from all this?

1) The FDA doesn't decide what is and is not a drug—people and companies do by making "drug-like" claims about their products.  The FDA simply ensures that those claims are backed up by real science.

2) The FDA is not ignoring the science—if anything, Diamond Foods is ignoring the need for science.  The FDA is demanding an overwhelming burden of proof (how scientific of them!).  The FDA does this to keep us American's safe from all the snake oil salesman out there in the world.  In fact, the FDA is one of the most restrictive and conservative regulatory bodies in the world.  This is a key reason why most drug and medical-device makers do their clinical trials outside the US—it's typically easier and faster to obtain approval (and, therefore, less expensive).  As someone who works in an industry regulated by the FDA, this can be very frustrating.  As a consumer, I am grateful I can go to a local drug store and pick up some NyQuil for my colds, or Sudafed for my congestion, or Claritin for my allergies, and know that those products will be both safe and effective.

One of the prevailing "healthy eating" trends today is the shift away from processed foods toward foods that are more "organic", more "natural"--in a word: RAW.  One needs look no further than the popular Paleo diet, which purports that the ideal diet for humans is that which cavemen would have been able to eat thousands of years ago.  Part of this food fanaticism movement has included backlash on state municipalities for laws which currently limit or outlaw the sale or distribution of raw milk.  So, let this be a Public Service Announcement: raw milk is dangerous.

Let's be clear: raw milk is milk which has not undergone pasteurization-- the process which is specifically intended to eliminate pathogenic microbes in the milk.  Those pathogens include bacteria such as E. coli, Salmonella, and Listeria, as well as parasites and viruses.  Here is a recent news article detailing a deadly listeria outbreak linked to raw milk.  You may remember Listeria from other news stories in the past (here and here).  The effects of exposure to these pathogens is not always as simple as 'stomach bug' type illness.  The pathogens that pasteurization eliminates are those which can cause Typhoid Fever, Tuberculosis, and Norovirus.  Here are the facts:

Listeria alone: 1600 illnesses, 260 deaths every year

Illnesses specifically tied to raw milk: Between 1998 and 2011, 2,384 illnesses, 284 hospitalizations, and 2 deaths

How risky is raw milk, really?  Here is a fantastic research paper on the topic-- rather than summarizing, I will simply quote a portion of the paper:



Raw dairy consumption is 150 times more likely to make you sick than pasteurized dairy products.  Let's put that in perspective: if drinking raw milk made you 150 times more likely to get in a car accident, you'd get in a car accident once every month on average, and would most likely die in one within 15 years. 150 times greater is the difference between running a marathon and running from LA to New York, and then back to Denver. 

So why would anyone consume food which is clearly related to an elevated risk of illness and death?  For the purported health benefits, of course!  The only trouble is that those health benefits-- 

• higher nutritional value
• lower rate of food allergies
• reduced lactose intolerance
• preservation of probiotics

have largely been scientifically debunked.  Another reason may simply be the misunderstanding of what pasteurization is.  With a scary word like that, it must involve antibiotics, hormones, or some other non-natural chemical treatment (hint: it doesn't).  

If you're a fan of tragic irony and anecdotal evidence, here's an article about lawmakers falling ill by consuming raw milk after celebrating the legalization of raw milk.

So, I leave it to the informed consumer to decide how the risk-benefit scales are tipped: a heightened risk of illness, hospitalization, and death for anecdotal and scientifically unproven health benefits.  It's not uncommon for some among our species to choose a much more dangerous way of living without any tangible benefit.