“Only when the last tree has been cut down. Only when the last river has been poisoned. Only after the last fish has been caught. Only then will you find that money cannot be eaten.”

Unfortunately we live in a world of people motivated by monetary reward. Naturally this is going to cause deception and bending of the truth in order to mislead. Often scientific literature reported in mainstream media is of no exception.

It would be extremely cynical to say that all information published has a sinister motive. Although some information is misleading, a large proportion is published with good intention at heart, but does that mean it is correct?

The aim of this article is to educate you on becoming a more informed consumer, making you less likely to wander down the misleading path of twisted messages and ulterior motives.

It would be naïve and short sighted to expect every member of the public to read relevant journal articles and make decisions based on recommendations, whilst also attempting to decipher between the most applicable and reliable information. However I’d hypothesise that most consumers tend to take an interest in personal health, fitness and happiness.

For many this may mean watching a relevant television program, surfing the web or reading the health section in your favourite newspaper. However it is always important to take in information with a sceptical eye.

I mean how much of the stuff you know, do you really know? Have you really confirmed the information or are you going on the word of a friend, or so called common knowledge?

Here are my top tips when reviewing health and fitness information in the mainstream media.

Reliability of the Source

Many will argue that you should look on all information with an objective eye removing all bias. This may be all well and good for someone with a scientific background but may not be so easy for a person with a different set of expertise and experience. For many, analysing the initial source will remove a lot of misinformation and save a lot of time.

For example, Dawn down the street is probably not as reliable as the latest peer reviewed journal article. Some things may not be as black and white but that’s where the other recommendations come in. For the most part analysing the credentials of the initial source should give you an initial idea about the credibility of the information, although by no means should you rule it out.

Anecdotal Evidence

Anecdotal evidence is exactly how it sounds. Evidence taken from anecdotes or singular data points. For example the most commonly heard is “it worked for me”.

Anecdotal evidence cannot be taken as conclusive evidence, it may provide a suggestion of a mechanism but certainly not applied to another individual with different characteristics until further investigated.

The number of variables that go into an end result is vast and changes between individuals. Attribution of a change between these variables is often misinterpreted and therefore cannot be taken as an absolute (see below.)

Special Populations

Although we are all very similar we also vary in many ways (that’s a confusing opening sentence if ever I’ve seen one). In that I mean that the basic physiology underlying human health and performance remains consistent but the variables that affect any intervention made are vast and have significant effect on end outcome. When the end outcome is as small and relatively insignificant, i.e. tastes, then the implications aren’t huge, but when it is as serious as potential long term mortality rate you can understand the importance of understanding the variables.

Certain information and data on specific populations will not be applicable to yourself.

Sample Size

Similar to anecdotal evidence but on a larger scale. There are many elements that ultimately decide the methodology of an experiment and budget, alongside recruitment of subjects which is often a stumbling block.

Often studies will recruit as few as 10-15 participants. Does this mean the findings are not worthwhile? No. However conclusions can’t be drawn on a singular study let alone with such a small subject group.

The Body of Literature

One study provides an insight into potential correlations and mechanisms. The body of literature in its entirety dictates manipulation of clinical practice and future recommendations.

With this in mind it is important not to change what you are doing because of one publication that you read. Always research and digest as much information as you can before making any conclusions or commitments.

Was the Outcome Statistically Significant?

Findings are often reported without looking at statistical significance. What do I mean?

Whenever an experiment is conducted there will be a certain amount of error associated in the findings. In science statistical significance is calculated using specific calculations to work out how reliable and robust the study is.  Often individuals will not report statistical significance in order to manipulate findings. If you’ve ever researched the sport’s supplement industry you will be well aware of this fact.

Therefore when reading literature it is important to assess the degree of change that occurred, or the strength of the correlation.

I’d argue that applying significance to your interpretation of findings is equally as important. For example when you read that the latest running spikes may knock 0.001s off of Usain Bolt’s 100m time, but you missed your bus this morning because you couldn’t run after it quickly enough, are you going to go out and buy the shoes?

Applying health and fitness advice into your everyday life should be interpreted in a similar way to statistical significance, but less about whether the findings were significant and more about to what degree this will help you, or is there anything you can do with greater effect. It’s important to get the 90% right before looking for the 1-2%. However we like to look at the 90% and the 1-2% changes to improve every aspect of health, performance and wellbeing!

Correlation Doesn’t Equal Causation

Trends and correlation are what you most likely experience the most in the press and other forms of media.

A correlation is simply a relationship between two sets of data points. Correlations are fantastic because they present a suggestion for a mechanism that aids a scientist’s understanding, and presents an opportunity for further research to identify the true cause.

Often a correlation is identified and reported in the media as a sole determining factor in order to generate interest.

Correlation data is often reported because it is often controversial or particularly intriguing, but because a correlation is present it does not mean that the initial variable is the cause for the findings, and until further research is conducted one cannot make a conclusion on an initial correlation.

Therefore reported correlation studies may be interesting, but viewed with a skeptical eye without jumping to conclusions or making sweeping decisions based on initial suggestions.

Interpretation, Attribution and Controlling the Variables

As alluded to previously the interplay of variables in dictating the outcome of an intervention is vastly complex and is often forgotten when viewing literature.

Most people only read the abstract of a scientific paper or perhaps skip to the conclusion. When taking this approach it is often forgotten that the conclusion is the scientist’s interpretation of the findings, which mean that the conclusion may not be fact, but opinion.

Sometimes interpreting data may not be as simple as one may first think, especially when considering the complexity of many experiments and interventions. One factor may be attributed to the change when simply it was a by-product of another variable, or perhaps not related at all.

This concept is exaggerated even further when considering mainstream media, which has a purpose to generate interest around a particular topic area. For example consider the following two headlines:

“Omega 3 supplementation shows marginal improvement in symptomology associated with depression”

 “Omega 3 consumption is proven to help cure depression!”

Which headline would sell more papers? It is therefore vital to analyse the data with a sceptical eye to see if the claims and attributions made are just.

Summary

The overall point of this article is to emphasise the importance of reading informative articles with a skeptical eye before applying the findings to everyday life.

Mainstream media may often exaggerate or misinterpret scientific literature leading to confused conclusions and less than optimal advice.

Hopefully it goes some way towards helping you become an educated reader/listener/viewer.

Jack (Researcher, Rescon Ltd)