Facebook Twitter

How to do Online Research
(Part 2 of 2)
Understanding Research Studies

What are researchers intentions?
What are researchers' intentions?

Last week I shared some sources you can turn to if you're trying to research something. This week, I'd like to tell you how to look at scientific studies critically. Learn what's worth repeating and what should be ignored. Here are the questions you should ask.

What are researchers trying to accomplish?

Read the introduction first to learn the researcher's intent and motivation. Figure out what they're trying to uncover. Are they asking the same things you are? If they are, then read through the rest of the paper. See if the way they've set up and conducted the experiment can adequately answer those questions.

Read the abstract only after you've finished everything else. It contains a neat summary of the entire paper, but it's subject to the researchers' biases. If you read it first, it might inadvertently bias you.

How big was the study?

Testing something on 10 people won't give you as much information as testing on 1,000 people. The more subjects that are involved, the more trustworthy the findings are.

How long was the study?

You need to test something long enough for the real effects to come out. When multivitamins were tested, most companies, if they tested at all, didn't go much beyond 6-12 weeks. But when 160,000 people were tracked for more than 10 years, researchers found that Multivitamins taken daily may be dangerous to your health. People who took multivitamins died sooner than those who didn't. But it took years to discover the problem.

Who paid for the study?

In 2015, as sales for sugary beverages dropped, Coca-Cola decided the best way to hang onto customers was to shift the blame. Coca-Cola gave over 1.5 million dollars to a new nonprofit called Global Energy Balance Network. The scientists at that nonprofit were hired to argue that Americans are too overly fixated on food and that we need to exercise more.

Coca-Cola wanted to make it look like independent scientists thought their sugary drinks weren't a problem. Their funding changed the tone of the research from that organization and tainted the results. Be wary of companies trying to push their agenda.

Is it a single study or a meta-analysis?

In a single study, researchers define the experiment's scope, document what's happening and use the data to come to conclusions. A meta-analysis is a study that looks in-depth at dozens of OTHER studies. They go through the scientific literature and find the most relevant data, then put it together to see the results.

For years we've been told organic produce contains more vitamins and nutrients than non-organic varieties. Several small studies seemed to support that conclusion. That was, until a team at Stanford's Center for Health Policy did the most comprehensive meta-analysis to date of existing studies comparing organic and conventional foods.

The researchers found that organic foods did not seem to carry fewer health risks than conventional foods. They also found that there was little evidence organic foods were any more nutritious than conventional foods. While organic foods were better for the planet, they didn't provide significant benefits for people eating them.

Is the study retrospective or prospective?

A retrospective study asks subjects what they've done in the past. Things like how many times you ate, drank, or exercised. They're less expensive because they just have to ask a bunch of questions and tabulate the results. They're also far less reliable because they rely on the subject's memory. Most people can't remember everything they ate a week ago, never mind two years ago.

In a prospective study, researchers follow the subjects over time. There are multiple meetings, with far more extensive record-keeping to verify what people are doing. Instead of putting everything together over a couple of interviews, a prospective study can take years before the results are known. These are much more reliable but not done as often because of time and money limitations.

Other Study Types

A Cohort Study is typically done over a long period of time (a longitudinal study). Many take years to complete. Researchers follow participants that share a common characteristic, like a job or place where they live. Then they divide the participants by those exposed to a specific risk factor, such as smokers and non-smokers. Cohort studies are particularly valuable in epidemiology, to help researchers understand what factors increase or decrease the likelihood of developing disease.

Case-Control Studies are a type of observational study. Two groups of people who have different outcomes are identified and compared. Researchers look through historical factors for the underlying cause that led to the different results. If exposure to a particular thing is more common in one group, researchers can hypothesize that exposure might have caused the outcome of interest. Case-control studies are cheaper to conduct than cohort studies, but they also provide less evidence than a randomized control trial and rely more on the memory of people reporting past behaviors.

Cross-Sectional Studies (also known as cross-sectional analysis, transverse study or prevalence study) are observational studies. You gather all the data from a population and measure what's happening at a single point in time. They're great for describing features of a research population, measuring the prevalence of health outcomes, and understanding determinants of health. Their most significant disadvantage is that they only measure what's happening at that specific moment. By their definition, they don't provide long-term or ongoing information.

How much of a supplement, drug or treatment is given to the subjects?

In the 1990s, resveratrol was proclaimed a miracle supplement. Mice that took it lived longer. Since resveratrol was found in red wine, companies began promoting a glass a day as a healthy habit. What they failed to mention was how much resveratrol those mice were given. A person would have to drink 5 gallons of wine a day, to get the equivalent amount of resveratrol that those mice received.

Doses in the experiments may be much higher or much lower than you would experience in real life.

Answer those seven questions, and you'll be much more able to judge how valid the study is. If the answer to any of those questions is suspect, that study may not be as reliable as you might want.

Now that you know, what are you going to research?

A Note About Debates

I'm not a big fan of debates. Let me explain.

In a debate, the person who is the best debater wins. The facts are only necessary for how they can be twisted to support your argument.

A common tactic on my high-school debate team was burying the opposition in pointless drivel so they would get distracted by emotions. The opposing team would take valuable time responding to made-up outrages while ignoring the fundamental flaws in my team's arguments. It's a great way to win a debate, but a terrible way to pass on useful information.

A personal trainer is supposed to help clients get results. If a better way to do something is discovered, it's essential to see where the evidence leads. Check on sources and make sure they're credible. Evaluate the evidence. Take the time to read and understand clinical studies.

I'm not interested in a spirited debate. But I'm always happy to learn more about the evidence you've collected. When a better method or program is proven, we all succeed when we share the truth.

Part 1 2

Call for a FREE Consultation (305) 296-3434
CAUTION: Check with your doctor before
beginning any diet or exercise program.

Updated 1/18/2021
Updated 9/19/2021
Updated 2/9/2022
Updated 3/10/2022