Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial, open data platform and infrastructure that supports research on pragmatic trials. It is a platform that collects and shares clean trial data and ratings using PRECIS-2, allowing for multiple and diverse meta-epidemiological studies to evaluate the effect of treatment on trials that have different levels of pragmatism, as well as other design features.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. The term "pragmatic", however, is a word that is often used in contradiction and its definition and measurement need further clarification. Pragmatic trials must be designed to inform clinical practice and policy decisions, rather than confirm an hypothesis that is based on a clinical or physiological basis. talking to should aim to be as similar to the real-world clinical environment as possible, including in the recruitment of participants, setting and design as well as the execution of the intervention, determination and analysis of outcomes as well as primary analysis. This is a significant distinction from explanatory trials (as described by Schwartz and Lellouch1), which are designed to provide more thorough confirmation of an idea.
Truely pragmatic trials should not conceal participants or the clinicians. This can lead to bias in the estimations of the effects of treatment. Practical trials also involve patients from various health care settings to ensure that the results can be generalized to the real world.
Additionally studies that are pragmatic should focus on outcomes that are important to patients, such as quality of life or functional recovery. This is particularly important when trials involve surgical procedures that are invasive or may have dangerous adverse effects. The CRASH trial29 compared a two-page report with an electronic monitoring system for hospitalized patients with chronic heart failure. The catheter trial28 on the other hand utilized symptomatic catheter-related urinary tract infection as the primary outcome.
In addition to these features pragmatic trials should reduce the procedures for conducting trials and requirements for data collection to cut costs and time commitments. Additionally, pragmatic trials should aim to make their findings as applicable to current clinical practices as they can. This can be achieved by ensuring that their analysis is based on the intention to treat approach (as described within CONSORT extensions).
Despite these guidelines, many RCTs with features that challenge pragmatism have been incorrectly self-labeled pragmatic and published in journals of all types. This can lead to false claims of pragmatism, and the use of the term must be standardized. The development of a PRECIS-2 tool that can provide an objective, standardized evaluation of the pragmatic characteristics is a first step.
Methods
In a pragmatic study the goal is to inform clinical or policy decisions by demonstrating how the intervention can be implemented into routine care. Explanatory trials test hypotheses about the cause-effect relationship within idealised conditions. In this way, pragmatic trials could have a lower internal validity than explanation studies and are more susceptible to biases in their design, analysis, and conduct. Despite their limitations, pragmatic research can provide valuable data for making decisions within the healthcare context.
The PRECIS-2 tool assesses the level of pragmatism that is present in an RCT by assessing it across 9 domains, ranging from 1 (very explicative) to 5 (very pragmatic). In this study, the recruitment, organization, flexibility in delivery, flexible adherence and follow-up domains received high scores, however the primary outcome and the method of missing data fell below the pragmatic limit. This suggests that it is possible to design a trial that has good pragmatic features without harming the quality of the results.
However, it's difficult to assess the degree of pragmatism a trial is, since pragmaticity is not a definite quality; certain aspects of a trial may be more pragmatic than others. A trial's pragmatism could be affected by changes to the protocol or the logistics during the trial. Koppenaal and colleagues found that 36% of 89 pragmatic studies were placebo-controlled or conducted prior to the licensing. They also found that the majority were single-center. Thus, they are not as common and can only be called pragmatic when their sponsors are accepting of the absence of blinding in these trials.
A common feature of pragmatic studies is that researchers attempt to make their findings more meaningful by studying subgroups within the trial sample. However, this often leads to unbalanced comparisons and lower statistical power, thereby increasing the risk of either not detecting or misinterpreting differences in the primary outcome. This was the case in the meta-analysis of pragmatic trials due to the fact that secondary outcomes were not adjusted for covariates that differed at the time of baseline.
Additionally practical trials can present challenges in the collection and interpretation of safety data. It is because adverse events are usually self-reported, and are prone to errors, delays or coding variations. Therefore, it is crucial to improve the quality of outcomes for these trials, in particular by using national registry databases instead of relying on participants to report adverse events in the trial's own database.
Results
While the definition of pragmatism may not require that clinical trials be 100% pragmatic there are benefits to including pragmatic components in trials. These include:
Enhancing sensitivity to issues in the real world as well as reducing study size and cost, and enabling the trial results to be faster translated into actual clinical practice (by including routine patients). However, pragmatic studies can also have drawbacks. For example, the right type of heterogeneity can help a study to generalize its results to many different settings and patients. However, the wrong type of heterogeneity could reduce assay sensitiveness and consequently decrease the ability of a trial to detect small treatment effects.
Numerous studies have attempted to categorize pragmatic trials with various definitions and scoring systems. Schwartz and Lellouch1 created a framework to distinguish between explanation-based trials that support a physiological or clinical hypothesis as well as pragmatic trials that inform the selection of appropriate treatments in real-world clinical practice. The framework was composed of nine domains that were assessed on a scale of 1-5 with 1 being more explanatory while 5 was more practical. The domains covered recruitment and setting up, the delivery of intervention, flexible adherence and primary analysis.
The initial PRECIS tool3 had similar domains and a scale of 1 to 5. Koppenaal et. al10 devised an adaptation of this assessment, dubbed the Pragmascope which was more user-friendly to use for systematic reviews. They discovered that pragmatic systematic reviews had higher average scores in the majority of domains but lower scores in the primary analysis domain.
This difference in the main analysis domain could be due to the fact that most pragmatic trials analyse their data in the intention to treat way, whereas some explanatory trials do not. The overall score for pragmatic systematic reviews was lower when the domains of organization, flexible delivery, and follow-up were merged.
It is important to note that the term "pragmatic trial" does not necessarily mean a poor quality trial, and there is a growing number of clinical trials (as defined by MEDLINE search, but this is neither specific nor sensitive) which use the word "pragmatic" in their abstract or title. These terms may indicate that there is a greater understanding of pragmatism in abstracts and titles, however it's unclear whether this is evident in content.
Conclusions
As the value of evidence from the real world becomes more commonplace the pragmatic trial has gained traction in research. They are randomized studies that compare real-world treatment options with new treatments that are being developed. They include patient populations more closely resembling those treated in regular medical care. This method is able to overcome the limitations of observational research for example, the biases that come with the reliance on volunteers, and the lack of coding variations in national registries.
Other benefits of pragmatic trials include the ability to use existing data sources, and a greater probability of detecting significant changes than traditional trials. However, they may still have limitations which undermine their reliability and generalizability. Participation rates in some trials may be lower than expected because of the healthy-volunteering effect, financial incentives, or competition from other research studies. Many pragmatic trials are also restricted by the necessity to recruit participants quickly. In addition, some pragmatic trials lack controls to ensure that the observed differences are not due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described as pragmatic. They evaluated pragmatism using the PRECIS-2 tool, which consists of the domains eligibility criteria, recruitment, flexibility in intervention adherence, and follow-up. They discovered that 14 of these trials scored highly or pragmatic pragmatic (i.e. scores of 5 or higher) in one or more of these domains and that the majority of them were single-center.
Trials with a high pragmatism score tend to have more expansive eligibility criteria than traditional RCTs that have specific criteria that are not likely to be present in the clinical environment, and they contain patients from a broad variety of hospitals. The authors claim that these characteristics can help make the pragmatic trials more relevant and relevant to everyday clinical practice, however they do not guarantee that a pragmatic trial is completely free of bias. Moreover, the pragmatism of trials is not a definite characteristic; a pragmatic trial that does not possess all the characteristics of an explanatory trial can yield reliable and relevant results.