In the first part of this blog series, we elaborated on the need for a general understanding of how to assess the quality of a scientific publication and on some criteria that can tell you whether a study is trustworthy or not before even reading it.
Here, we want to concentrate on the study design and the methods of clinical studies. In order to draw conclusions from study data, it is important to have a clear hypothesis and a study design that is capable of assessing whether the hypothesis is correct or not without bias.
How is the study structured and how does the study design reflect everyday clinical practice?
If the study you are looking at seems trustworthy to you based on the criteria mentioned in the previous blog, you can go on to evaluating the content presented to you. Here’s what to consider when analyzing the design of the study:
- How many participants were involved?
A study which only includes a hand full of people, does not allow conclusions for a broad audience. However, the number of people included is highly dependent on how many people are affected by the condition that is being researched.
- What kind of participants were involved?
To draw conclusions, the study population has to be comparable to you/your patients.
- Were participants randomized?
If two or more treatments are compared in a study, it is important to know if the treatments were assigned randomly. If not, there could be a selection bias.
- Was the study blinded?
When comparing two or more treatments it is relevant to know if the participants and investigators knew who will receive the treatment (open label) or not (single or double blinded).
- How long was the study duration?
The study period should be long enough to assess potential safety concerns and to ensure a benefit over time.
- For trials comparing different treatments: Is the chosen comparator suitable and are the participant characteristics in both groups comparable?
Especially for conditions where effective treatments exist, it might be necessary to compare a new treatment to the current standard of care rather than to placebo. The characteristics of participants in the compared groups must be considered. Ideally, they are similar, if not a statistical adjustment of the results based on the baseline characteristics might be necessary.
Does the study have a clear hypothesis and are the presented methods suitable to assess this hypothesis?
The methods used in a study are crucial to evaluate the presented results. If the methodology is unclear, biased, or flawed the obtained results cannot be trusted.
- Is the hypothesis clear?
The authors need to state what the expected outcome of the study was.
- Is the study methodology suitable to assess the hypothesis?
The results can only support the hypothesis if the outcomes are well chosen.
- Who defined the outcomes?
It is a good sign if the outcomes are defined by an independent expert board.
- Which outcomes were defined?
It should be defined what was measured and explained why the results will be able to support or debunk the hypothesis.
