Where is Diogenes when you need him? You remember. He was the guy who carried a lamp to shine light in people’s faces. It was supposed to show who was living truthfully.  I don’t think the authors of some “health studies” in this age of misinformation would pass Diogenes’ lamp test.

Case in Point

Dr. Greger’s video, “Is butter really back? What the science says,” analyzes how health studies can lead to wrong conclusions, not, I would add, always by accident. https://nutritionfacts.org/video/friday-favorites-is-butter-really-back-what-the-science-says/?utm_source=NutritionFacts.org&utm_campaign=684f624d61-RSS_VIDEO_WEEKLY&utm_medium=email&utm_term=0_40f9e497d1-684f624d61-29704488&mc_cid=684f624d61&mc_eid=2114502bf0

Wild guess here. Who do you think commissioned this study, and what was the corporate connection between the butter industry and the “scientists” doing the study? To be fair, good research is not easy. I recall taking a course on understanding research many years ago. I came away with tools (which I sadly have misplaced over the years) to help me study a published paper with an eye toward how it was put together.

A Few Pointers

I found a website with some excellent examples of sources of potential bias common to research.  https://www.understandinghealthresearch.org/useful-information/common-sources-of-bias-2

Here are the highlights (plus a few of my own).

• Recall Bias – relies on peoples’ memories (and I would add honesty). It’s hard to remember where I left my iPhone, let alone how often I ate, or did, X or Z over the last 6 months.

Or ask a coffee lover like me if I drink too much coffee or if it makes me jittery. I would answer “No. I don’t drink too much coffee. It doesn’t make me jittery.”  (However, I worked the night shift for years and for sure, my idea of too much coffee would not be yours. I probably numbed my caffeine receptors long ago!)  So I would be a poor subject for a coffee study, unless you were doing a study on coffee consumption of night shift workers.

       • Selection bias.  Some groups may be underrepresented or over-represented. Are most of the subjects black people living in a food desert? Are they included at all? How many Native Americans are included in health studies? The cure is randomization – subjects collected at random (ideally electronically to prevent inadvertent human errors).

• Observation bias (the Hawthorne Effect) When subjects are aware they are being studied and that, consciously or not, alters their actions or answers.

(That’s why “double-blind” studies are done. In a blind study, the participants have no idea if they are given the “real thing” or a fake. Equal numbers of participants are given the actual medication or a placebo. Because the “placebo effect” can produce the desired outcome of a drug if the participant believes he’s getting the real thing, they are not told which one they are getting.

In a “double blind” study the people administering the drug being studied have no idea if they are giving the real thing or not. So the test subjects AND people administering the drug are both “blind.” This prevents any subconscious behavior on their part that might convey to the participants what they are receiving —the real thing or a placebo. The mind and body are so closely united that even unconscious messaging could interfere with some studies.

• Confirmation bias Isn’t  it easy to read articles or reports with jaundiced eyes, looking for and seeing only data that supports our point of view?
• Publishing bias This is a common one. If I run a set of studies to prove my new medication will lower blood pressure, I will publish only the studies that prove that it does, and bury the ones that show it doesn’t. Which studies do you think are sent to the FDA?

A few of my observations

• Vested interest A food or drug company hires a team to investigate their product regarding, say, safety, and surprise! Turns out the product is perfectly safe. Or is it? Always check connections, even if twice (or more) removed.
• Quality Assurance surveys. Don’t you love those? Immediately after you interact with a company or hospital you get a questionnaire, “how are we doing?”

I’ve spent enough 24 hour vigils in the hospital advocating for my husband to know firsthand areas that desperately need improvement. In fact, if hospitals were as good as they claim, I would not have needed to do the vigils in the first place! Yet, when we receive a hospital Quality Assurance survey because they CARE so much, none of the questions relate to the egregious areas they need to address. The questions are loaded so they look good.  “Was the hospital staff polite? Did they listen to your concerns?”  I would have to answer “yes.” But you did not ask if care was timely. NO – we waited an hour before anyone answered the call light.

• Eliminating variables Go back to my comments about my coffee history. My night shift background is a variable that could make me inappropriate for some studies on caffeine. A good study eliminates as many variables as possible. Don’t want to muddy the data waters with variations.

Bottom Line

In this age of misinformation, we consumers must learn not to hang our hats on only one study, especially if that study is done by people associated with companies who have a vested interest in the outcome. Or if they happen to have a handy-dandy product to sell based on their “study” results.  That’s why I often refer to Dr. Greger in my blogs. I need a research geek to do the work for me, (or find out where on earth I buried my textbook on research.)

I guess it’s like political news. We must listen to, read, and study several points of view (or studies) for Diogenes’ lamp to work at all.