Towards better targeting of treatments to subgroups

Multiple testing of subgroups in RCTs risks 1 in 20 false-positives

Individual participants in any randomized controlled trial (RCT) differ in many ways and such heterogeneity modifies the effect of treatment on outcomes.

Treatment effectiveness also differs across patient subgroups, and individual and population health can be improved by better targeting of treatments to subgroups.

It is important to identify true-positive subgroups within an RCT, but the multiplicity of testing for subgroups will judge 1 in every 20 as statistically significant by chance, said Richard Grieve, Professor at London School of Hygiene and Tropical Medicine.

Missing or falsely identifying subgroups has different impacts depending on whether the information is being used to guide clinical decision-making , regulatory decisions, adoption and reimbursement decisions, or the conduct and design of future clinical studies, he added.

Novel interpretations of real-world data may help identify subgroups

Interpretation of subgroup analysis in RCTs can be challenging due to the risk of false-positive results arising from multiple testing compounded by lack of power. The application of novel methods to real-world data and biometrics are therefore becoming important to identify subgroups, but still pose significant analytic challenges.

The many levels of effectiveness (or cost-effectiveness)

Whose treatment is cost-effective?

Data on effectiveness are far more complex than appreciated, and hypothetically there are as many levels of effectiveness (or cost-effectiveness) as there are patients, said Warren Stevens, Principal Economist at PAREXEL.

For example, Joe receives a treatment that is cost-effective for him, but not cost-effective for society, while Bob receives a treatment that is not cost-effective for him but is cost-effective for society. “Whose treatment is cost-effective?” he asked.

Implicit trade-offs are made due to the limitations of current data evidence generation, so how can incentives for regulators, payers, and clinicians be better aligned to make more efficient and equitable decisions? added Dr Stevens.

Acceptance of the essential need to reflect heterogeneity in cost-effectiveness leads to the recognition of heterogeneity in ‘valuing’ health. For example, length of life versus quality of life and physical functioning versus mental acuity?

Cost-effectiveness has been derived from population statistics and the imagined homogenous “blob” of being a “people” — but not reflecting the true heterogeneity of the population risks inefficiency and inequality in healthcare provision, said Dr Stevens.

Reflecting the heterogeneity of a population enables efficient and equitable healthcare provision

Until recently, a lack of consistently high-quality, patient-specific data has hindered the use of methods to reflect heterogeneity of effect in cost-effectiveness and policy, but novel methods are now providing opportunities to address this, he concluded.

Two such approaches that are being investigated are:

• the development of a fully Bayesian framework for the Identification and estimation of Subgroup effects in Randomised Controlled Trials (BISECT), an ongoing Medical Research Council-funded project, which was presented by Neil Hawkins, Professor of HTA at University of Glasgow, Scotland
• the application of novel econometrics methods to real-world data, which was described by Anirban Basu, Stergachis Family Endowed Professor and Director, University of Washington, WA