Skip to main content

statswork

Stronger Analysis. Smarter Research. Bigger Savings!
Access expert statistical analysis and research support at Flat 24% Off
Stronger Analysis. Smarter Research. Bigger Savings!
Access expert statistical analysis and research support at Flat 24% Off.
Facebook X-twitter Youtube Linkedin Instagram
Transcripition | Corporate Statistics  | Hire a Statiscian
STATSWORK LOGO
Facebook X-twitter Youtube Linkedin Instagram
STATSWORK LOGO

Why Does IQWiG Reject Subgroup Findings That the EMA Accepted? Planning Effect-Modifier Analyses for German Benefit Assessment

Meta Analysis Service

Recommended Reads

Contact us

Summary

EMA and IQWiG assess subgroup evidence within different decision frameworks. This article explains why regulatory subgroup findings may not establish German added benefit, emphasizing interaction testing, statistical power, multiplicity, estimands, confidence intervals, and uncertainty. It also provides a practical framework for pharmaceutical companies to prospectively plan credible, clinically meaningful, and HTA-relevant effect-modifier analyses.

A subgroup result supporting regulatory evaluation does not automatically establish added benefit in Germany. EMA and IQWiG assess evidence within different decision frameworks, considering subgroup credibility, consistency, comparator relevance and patient groups differently. Under section 35a SGB V, added benefit must be demonstrated for defined patient groups versus the appropriate comparator. [3] Pharma teams should therefore predefine subgroup strategies.

Why EMA and IQWiG can reach different conclusions

The European Medicines Agency (EMA) considers subgroup analyses as part of confirmatory clinical-trial assessment and benefit–risk evaluation. Its subgroup guideline emphasizes pre-specification, biological plausibility, consistency, appropriate statistical methods and interpretation of heterogeneity. [1] The purpose is not simply to determine whether one subgroup has a statistically significant result.

German AMNOG assessment addresses a different question: what additional benefit is demonstrated for defined patient populations compared with the appropriate comparator? Section 35a SGB V explicitly requires assessment of added benefit, its extent and therapeutic significance, including the patient groups for which a therapeutically meaningful added benefit exists. [3]

IQWiG’s General Methods provide the methodological framework for its assessments and are currently available in Version 8.0.[2] Consequently, evidence that is informative for regulatory benefit–risk assessment may still require stronger or differently structured evidence to support a patient-group-specific conclusion in German benefit assessment.

Aspect

EMA regulatory assessment

German benefit assessment

Primary decision context

Marketing authorisation and benefit–risk assessment

Added benefit under AMNOG

Subgroup role

Assess consistency, heterogeneity and credibility of treatment effects

Determine whether evidence supports added benefit for relevant patient groups

Comparator

Depends on trial and regulatory question

Zweckmäßige Vergleichstherapie (appropriate comparator therapy)

Key statistical concern

Credibility and consistency of subgroup effects

Whether differential effects are sufficiently demonstrated for the assessment population

Practical implication

A credible subgroup signal may inform the overall regulatory interpretation

A subgroup claim needs to support a defined added-benefit conclusion

The important point is that regulatory acceptance and German HTA acceptance are not interchangeable evidentiary outcomes.

The Statistical Error Behind Many Subgroup Claims

The most common mistake is to compare p-values within separate subgroups.

Suppose a trial produces:

  • Treatment effect in subgroup A: HR = 0.70, 95% CI 0.55–0.89
  • Treatment effect in subgroup B: HR = 0.95, 95% CI 0.76–1.19

It is tempting to conclude that treatment works in A but not B. That conclusion does not follow automatically.

The correct statistical question is whether the treatment effect differs between A and B. This requires an interaction test.

For a Cox proportional-hazards model:

h(t)=h0(t) exp (β1T+ β2S+ β3 (T×S))

where:

  • T = treatment indicator
  • S = subgroup membership indicator
  • T×S = treatment by subgroup interaction term
  • β1 = treatment coefficient
  • β2 = subgroup coefficient
  • β3 = treatment by subgroup interaction coefficient, representing effect modification

The null hypothesis of no treatment-effect modification is:     

H0: β3=0

Here, β3 is the key parameter: if β3 = 0. the estimated treatment effect does not differ between the subgroups on the log-hazard scale.

A significant treatment effect in one subgroup and a nonsignificant effect in another is not itself evidence of interaction. This distinction is well established in the methodological literature. [4,5]

Interaction Confidence Intervals Matter

The interaction estimate should be interpreted using its confidence interval, not only its p-value.

A wide interaction confidence interval indicates substantial uncertainty about the magnitude of effect modification and may leave uncertainty even about its direction. This is particularly important when subgroups are small.

For example, an interaction HR of 0.75 with a 95% CI of 0.45–1.25 does not establish a precise 25% relative difference between subgroups. The estimate may be compatible with modest effect modification, little difference, or a substantially different effect.

Why Subgroup Analyses Are Often Underpowered

Trials are usually powered for the primary treatment effect, not for interactions. Brookes et al. showed that a trial with 80% power to detect the overall treatment effect could have only about 29% power to detect an interaction of the same magnitude [4]. Detecting smaller effect modifications requires still larger samples.

This has direct implications for German HTA planning. If a company expects a subgroup to support a differentiated added-benefit claim, it should not assume that a conventionally powered pivotal trial will automatically provide enough information.

Planning should therefore consider

  • Expected subgroup prevalence;
  • Anticipated treatment-effect difference;
  • Event rates for time-to-event endpoints;
  • Power for the interaction, not merely the main effect;
  • Precision of subgroup estimates;
  • Whether pooling across trials is scientifically justified;
  • Sensitivity to missing or sparse data.

What Makes an Effect Modifier Credible?

A credible effect modifier is more than a variable associated with different subgroup p-values. Evidence becomes more persuasive when several elements point in the same direction.

EMA guidance and methodological literature emphasize considerations such as: [1,5]

  • Definition through pre-specification: the definition of the subgroup and the hypothesis or analysis related to it occur prior to knowledge of any results.
  • Plausible rationale: a plausible biological rationale exists that can explain the difference in treatment effects.
  • Test for interactions: the analysis tests for differences in treatment effects.
  • Compatibility: consistency in finding relevant outcomes or analyses.
  • Magnitude: the finding involves a substantial difference in treatment effects.
  • Adequate statistical power: there is sufficient data to provide precise estimates.
  • Control of multiple testing issues: a small number of effect modifiers are used.
  • Robustness: the findings hold up under various sensitivity analyses.

These criteria are independent. If an interaction is statistically significant but implausible in its biological explanation, then the former does not imply the latter.

Continuous Effect Modifiers Need Careful Planning

Age, baseline disease severity, renal function and biomarker concentrations are often continuous variables. Arbitrarily dividing them into “high” and “low” groups can discard information and make results dependent on an artificial threshold.

Where clinically justified, pharmaceutical teams can model continuous effect modification using:

  • Treatment by continuous variable interactions;
  • restricted cubic splines;
  • prespecified clinically meaningful thresholds;
  • graphical assessment of treatment effects across the modifier range.

The choice should be specified prospectively and supported by the clinical question. Data-driven cut-points selected after seeing treatment outcomes can create unstable and difficult-to-defend subgroup findings.

Multiplicity Can Quickly Become a Problem

A trial may investigate treatment modification by age, sex, disease severity, biomarker status, renal function, prior treatment and other characteristics.

If 20 independent hypotheses are tested at a nominal two-sided α = 0.05, the probability of at least one false-positive result under a simplified independence assumption is:

This is an illustration rather than a direct calculation for every real trial because subgroup tests may be correlated. Nevertheless, it demonstrates why indiscriminate subgroup searching is problematic.

A trial may investigate treatment modification by age, sex, disease severity, biomarker status, renal function, prior treatment and other characteristics.

If 20 independent hypotheses are tested at a nominal two-sided α = 0.05, the probability of at least one false-positive result under a simplified independence assumption is:

This is an illustration rather than a direct calculation for every real trial because subgroup tests may be correlated. Nevertheless, it demonstrates why indiscriminate subgroup searching is problematic.

benefit assessment

A Real-World German Example: Sacubitril/Valsartan

The sacubitril/valsartan assessment provides a useful example of why subgroup evidence can become consequential in German benefit assessment.

In its assessment of Entresto, IQWiG identified evidence suggesting effect modification by diagnosed diabetes mellitus. Following the commenting procedure, the manufacturer submitted additional analyses and data; IQWiG subsequently assessed different added-benefit conclusions for patients with and without diabetes. [6]

Importantly, this was not simply a case of “EMA accepted it, IQWiG rejected it.” The German assessment involved its own evidentiary question and additional analyses. IQWiG reported an indication of minor added benefit for patients with diabetes and an indication of considerable added benefit for patients without diabetes. [6]

The EMA’s Entresto assessment, meanwhile, evaluated subgroup findings within the regulatory assessment of PARADIGM-HF and considered consistency of treatment effects across relevant clinical characteristics. [7]

The lesson for pharmaceutical companies is therefore more precise: the same trial can generate subgroup evidence that is interpreted differently because the evidence is being used for different decision purposes.

Plan effect-modifier analyses from the start

Effect-modifier analysis should be part of the development strategy rather than an analysis added after the primary result. The EMA specifically describes subgroup investigation as part of clinical-trial planning, analysis and inference. [1]

A practical planning sequence is:

1. Define the clinical hypothesis

Specify why treatment response could plausibly differ across the modifier.

2. Align the estimand

The subgroup analysis should target the same clinically relevant treatment-effect question as the primary analysis unless a different estimand is explicitly justified.

3. Define the modifier

Specify categorical or continuous form, thresholds, timing and derivation rules.

4. Interaction model must be prespecified

Specify the modifier-treatment interaction term and statistical model suitable for the outcome measure.

5. Assess interaction power

Estimate the information required to distinguish plausible treatment-effect differences.

6.Control flexibility in analysis.

Distinguish between confirmatory modifiers and exploration of data

7. Plan missing-data and sensitivity analyses

Specify how missing baseline modifiers, misclassification and alternative definitions will be handled.

8. Define interpretation rules

Interpret subgroup estimates alongside interaction estimates, confidence intervals, clinical plausibility and consistency.

9. Map the analysis to German HTA needs

Assess whether the planned subgroup definitions correspond to clinically meaningful patient groups and the relevant German comparator framework.

What pharmaceutical companies should check

Before locking the protocol and SAP, teams should ensure that:

  • the subgroup hypothesis has a documented clinical rationale;
  • the effect modifier is measured reliably at baseline;
  • subgroup definitions are clinically meaningful and reproducible;
  • interaction testing is explicitly specified;
  • the interaction model matches the endpoint and estimand;
  • interaction power and expected precision have been assessed;
  • multiplicity is addressed;
  • continuous modifiers are not unnecessarily dichotomised;
  • subgroup analyses are aligned with the relevant comparator;
  • sensitivity analyses are prespecified;
  • evidence can be reproduced from the analysis dataset and programming specifications.

This approach also reduces the risk that a potentially important effect modifier is discovered only after the primary analysis, when evidentiary credibility may already be weakened.

Why this matters for German benefit assessment

Under section 35a SGB V, the German assessment ultimately concerns demonstrated added benefit relative to the appropriate comparator and the patient groups for which that benefit exists. [3] Subgroup analysis can therefore influence not only interpretation of treatment heterogeneity but also how evidence is presented for distinct patient populations.

That makes effect-modifier planning a cross-functional issue involving clinical development, biostatistics, regulatory strategy and market access.

A statistically sophisticated subgroup analysis is valuable only when its design, estimand, population definition and interpretation are aligned with the decision it is intended to inform.

How Statswork can support effect-modifier planning

Statswork’s biostatistics and statistical programming services cover clinical-trial design, statistical analysis plans, sample-size and power analysis, advanced statistical modelling, programming, analysis and interpretation.

For pharmaceutical teams planning subgroup and effect-modifier analyses, support can include:

  • protocol and SAP development;
  • interaction-model specification;
  • power and sample-size assessment;
  • survival and regression modelling;
  • subgroup and sensitivity analyses;
  • statistical programming and validation;
  • interpretation and reporting of treatment-effect heterogeneity.

The objective is not to manufacture a favourable subgroup result. It is to ensure that clinically important effect modification is investigated using a pre-specified, statistically defensible and reproducible framework.

Conclusion

EMA and IQWiG may reach different conclusions about subgroup evidence because the analyses serve different decision frameworks. Regulatory assessment considers subgroup credibility within benefit–risk evaluation, while German benefit assessment asks whether added benefit is demonstrated for relevant patient groups against the appropriate comparator.

Companies should therefore plan prospectively: define the estimand, pre-specify effect modifiers, test interactions, assess power, control multiplicity, and interpret uncertainty. A subgroup p-value alone is insufficient. Robust analyses should begin at the protocol stage to ensure statistically credible, clinically interpretable, and HTA-relevant evidence.

Frequently Asked Questions (FAQs)

No. EMA regulatory assessment and German benefit assessment address different decision questions. Evidence considered informative by EMA may require additional justification or analysis for an AMNOG assessment.

No. A treatment effect that is statistically significant in one subgroup and nonsignificant in another does not establish that the effects differ. A formal treatment-by-subgroup interaction analysis is required.

There is no universal requirement. Interaction power depends on subgroup prevalence, event rates, treatment-effect differences and the endpoint. Trials powered for the overall treatment effect can be substantially underpowered for interactions.

Not automatically. Dichotomisation can discard information and create arbitrary thresholds. Continuous interaction models or clinically justified thresholds may be preferable.

The strategy should begin during protocol development and be finalized in the SAP before unblinding. Retrospective subgroup discovery can remain useful for hypothesis generation but generally carries greater interpretive uncertainty.

References:

  1. European Medicines Agency (EMA). Guideline on the investigation of subgroups in confirmatory clinical trials. EMA/CHMP/539146/2013. Adopted 31 January 2019; effective 1 August 2019. https://www.tga.gov.au/sites/default/files/2026-07/EMA_CHMP_539146_2013-Guideline-investigation-subgroups-confirmatory-clinical-trials_en.pdf
  2. Institute for Quality and Efficiency in Health Care (IQWiG). General Methods Version 8.0. 2026. DOI: 10.60584/General-Methods_V8.0. https://www.iqwig.de/methoden/general-methods_version-8-0.pdf
  3. Bundesministerium der Justiz. Social Code Book V (SGB V), section35a: Assessment of the benefit of medicinal products with new active substances. https://www.gesetze-im-internet.de/sgb_5/__35a.html
  4. Brookes ST, Whitely E, Egger M, et al. Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. Journal of Clinical Epidemiology. 2004;57(3):229–236. doi:10.1016/j.jclinepi.2003.08.009. https://pubmed.ncbi.nlm.nih.gov/15066682/
  5. Sun X, Briel M, Walter SD, Guyatt GH. Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ. 2010 ;340. https://doi.org/10.1136/bmj.c117
  6. Sacubitril/valsartan in heart failure: differing added benefit. 2016; including Addendum A16-29 to Commission A15-60. https://www.iqwig.de/download/a16-29_sacubitril-valsartan_addendum-to-commission-a15-60.pdf
  7. European Medicines Agency. Entresto (sacubitril/valsartan): EPAR Public Assessment Report. EMA/671279/2015. https://www.ema.europa.eu/en/documents/assessment-report/entresto-epar-public-assessment-report_en.pdf

Contact us