Interpreting results of clinical trials is essential in the probiotic field. Many in the field take great care to carefully review how studies are conducted prior to deciding how much confidence to have in a study’s conclusions.* Luckily, the conduct and reporting of clinical trials is improving; the transparency for studies conducted today is much greater than even 20 years ago.
I came across a paper recently that suggests that an important factor in reducing bias in reporting of clinical trial results is the process of trial registration. Although not a recent paper (published in 2015), I think it offers valuable insights about clinical research, insights which are important in considering evidence for probiotics.
The authors noted that prior to 2000, a greater number of trials showed a significant benefit of an intervention. They based this conclusion on a limited assessment. They looked at large (>$500,000) studies funded by the National Heart Lung, and Blood Institute (NHLBI) with a primary outcome of cardiovascular risk, disease or death. Although this is a narrow sliver of research, their analysis was careful and importantly their conclusions are careful and warranted.
They did not pick the year 2000 arbitrarily. They note that NHLBI was an early adopter of requiring trial registration on ClinicalTrials.gov. From 2000 on, all large NHLBI were registered prospectively. Why is this important? Trial registration requires investigators to a priori state primary and secondary objectives of the study. Once that is made public, it prevents investigators from looking at study results and choosing to report on outcomes that were most compelling, a clear expression of investigator bias.
Kaplan and Irvin found that of the 55 trials they included in their assessment, prior to the year 2000, 57% (17 of 30 studies) reported that an intervention was successful, whereas after 2000, only 8% (2 of 25 studies) reported success. This is a drastic increase in null trials.
This analysis is observational, not causal. We cannot conclude that the trial registration process necessarily led to the increase in null trials – other factors may play a role in this, including better treatment and lifestyle choices for CVD patients enrolled in studies, which could lead to a higher bar for interventions. But this study does provide a compelling argument that trial registration may be very important.
Recently, there have been several null clinical trials for probiotics (see here and here). We should expect that such reports will become more common, as null trials are on the rise. As Kaplan and Irvin conclude:
…null findings in large RCTs may be disappointing to investigators, but they are not negative for science. Properly powered trials might identify treatments that will improve public health. A growing collection of trials suggests that promising treatments do not match their potential when systematically tested and transparently reported. Publication of these trials may lead to the protection of patients from treatments that use resources while not enhancing patient outcomes.
In the probiotic field, we need to insist that all human studies are registered prospectively. Those that aren’t should be considered suspect. We should embrace any prospectively registered, properly conducted and CONSORT-compliant human clinical trial, regardless of the outcome, and glean from it what we can about appropriate probiotic use for the benefit of consumers and patients. Sometimes what we learn is that a specific probiotic is not effective in a particular setting or for a particular condition. That’s progress.
*I recently collaborated with 3 authors on a paper that summarized available knowledge on clinical use of probiotics for human health (Sanders et al. 2018 – open access), where we relied heavily on well-conducted systematic reviews to make conclusions about evidence.