Skip to main content

statswork

Stronger Analysis. Smarter Research. Bigger Savings!
Access expert statistical analysis and research support at Flat 24% Off
Stronger Analysis. Smarter Research. Bigger Savings!
Access expert statistical analysis and research support at Flat 24% Off.
Facebook X-twitter Youtube Linkedin Instagram
Transcripition | Corporate Statistics  | Hire a Statiscian
STATSWORK LOGO
Facebook X-twitter Youtube Linkedin Instagram
STATSWORK LOGO

Will 30% Missing PRO Data Sink Your AMNOG Dossier? Designing Quality-of-Life Analyses IQWiG Won’t Downgrade

AMNOG Dossier

Recommended Reads

Contact us

Summary

Thirty percent missing PRO data does not automatically invalidate an AMNOG quality-of-life analysis. The real question is whether the missingness, and the assumptions used to handle it, could change the treatment conclusion. A stronger IQWiG-facing strategy characterises the missingness, defines the estimand, selects an appropriate primary analysis and tests whether the PRO conclusion survives plausible alternative assumptions.

Patient-reported outcomes (PROs) provide important evidence on symptoms, functioning, health status and health-related quality of life in an AMNOG dossier. Under Section 35a SGB V, pharmaceutical companies submit evidence supporting the claimed added benefit, and the G-BA usually commissions IQWiG to carry out the assessment [1].

When around 30% of PRO observations are missing, the rate warrants investigation, but it is not an automatic failure threshold. The key question is whether the missing observations differ systematically from the observed data, and whether plausible assumptions about the missing values could change the conclusion. Sponsors should therefore not try to “rescue” the dataset after missingness occurs. They should plan before submission how missingness will be investigated, handled and stress-tested, in line with IQWiG’s current General Methods Version 8.0 [2].

Start with the Pattern, Not the 30% Figure

A statement such as “30% of PRO data are missing” is incomplete without saying where and why the missingness occurred.

Suppose 30% of Week 52 KCCQ observations are unavailable. If missingness is similar in both treatment groups and is largely due to administrative issues unrelated to patient health, the implications are very different from a situation in which patients with worsening symptoms discontinue and stop completing the questionnaire.

The analysis should therefore examine missingness by treatment arm, visit, baseline PRO score, previous PRO score, discontinuation status and clinically relevant patient characteristics.

Diagnostic question

Why it matters for the PRO conclusion

Is missingness balanced between treatment groups?

Large differences between arms may indicate differential attrition.

Does missingness increase at later visits?

Attrition may be related to treatment duration or disease progression.

Do baseline PRO scores differ between patients with and without follow-up data?

Suggests missingness may be associated with outcome level.

Do patients discontinue treatment before PRO assessments?

Treatment discontinuation can create informative missingness.

Is missingness associated with clinical deterioration or adverse events?

Missingness may be related to the unobserved PRO outcome.

Table 1. Diagnostic questions for missing PRO data

This descriptive work is not merely supplementary. It provides the evidence needed to justify the assumptions behind the primary analysis.

Define the Estimand Before Choosing the Imputation Method

Multiple imputation cannot compensate for an unclear clinical question. Before deciding how to handle missing observations, the statistical analysis plan should specify which treatment effect the PRO analysis is intended to estimate, in line with the ICH E9(R1) estimand framework. For example, the estimand may concern the treatment effect at a specified time point under a treatment-policy strategy, or it may use a different strategy for intercurrent events.

This matters because discontinuation, rescue medication, treatment switching and death can affect both the availability and the interpretation of PRO measurements.

multiple imputation

Figure 1. From clinical question to robust results: the order in which PRO analysis decisions should be made.

Reversing this sequence risks choosing an analysis because it suits the available data rather than because it answers the intended clinical question.

Choose a Primary Analysis That Matches the Longitudinal PRO Data

For repeated PRO assessments, the primary analysis should use the full structure of the data rather than reducing the dataset unnecessarily to complete cases.

A mixed effect model repeated measure (MMRM), for instance, can incorporate observed PRO data at multiple visits without the necessity of a complete set of visits per patient. With proper specification of the model and within the framework of missing at random (MAR) condition based on the observed data, it can serve as a proper primary analysis method. Multiple imputation followed by analysis under proper conditions is one more helpful method.

What matters is not whether a method is labelled “advanced”, but whether it fits the estimand, the data structure and the missingness assumptions.

Approach

Potential role

Main concern at high missingness

MMRM

Repeated continuous PRO measurements with an appropriate covariance structure

Valid under MAR; biased if dropouts differ in ways the observed history does not capture

Multiple imputation + analysis model

Creates several plausible completed datasets and carries imputation uncertainty into the result

Results depend on the imputation model and its assumptions

Observed-case analysis

Describes the available data

Can be biased when missingness is related to prognosis or outcome

LOCF

Simple historical approach

Assumes no change after dropout, which is often implausible; understates uncertainty

Table 2. Common approaches and their main concern at high missingness

The EMA guideline on missing data explicitly discusses MCAR, MAR and MNAR assumptions alongside methods such as MMRM, LOCF and pattern-mixture models. This reinforces that missing-data handling must be linked to assumptions rather than treated as a mechanical replacement exercise [3].

Use Multiple Imputation as a model, not a repair tool

German HTA

Multiple imputation (MI) offers a principled way to handle missing PRO observations, but it does not repair the data. Its credibility depends on how well the imputation model represents the available patient information, on the assumptions made about missingness, and on whether the treatment estimate stays robust under alternative assumptions [4].

Figure 2. Multiple imputation as a modelling process: what makes it credible, and why it does not make missingness disappear.

Make the missingness assumption explicit

The critical distinction is between what is observed and what must be assumed about the values that were not observed.

The missing at random (MAR) assumption indicates that the probability of a PRO being missing is dependent solely on the information that is known such as the treatment arm and the PRO data collected prior to the study period. Once those are accounted for, it does not depend on the unobserved value itself. In practice, this means patients who drop out are assumed to have outcomes like those of patients with the same observed history who stayed in the study. This assumption cannot usually be verified from the observed data.

If patients drop out because their quality of life is deteriorating, in ways their earlier recorded scores do not reveal, MAR becomes questionable. That is why a convincing analysis should not stop at the primary MAR-based result.

The EMA guideline recognises the need to consider other mechanisms, including missing-not-at-random (MNAR) approaches [3]. For an AMNOG dossier, the practical implication is straightforward: the analysis should show how much the conclusion depends on the missing-data assumption.

Test whether the PRO conclusion survives

Sensitivity analysis is where the “30% missing” problem becomes quantitatively interpretable. Suppose a hypothetical PRO analysis (higher scores = better) estimates a treatment difference of 6.0 points, with a 95% confidence interval of 1.5 to 10.5, under the primary MAR-based analysis.

Now apply increasingly unfavourable assumptions to the missing values in the treatment arm, shifting each imputed value down by δ points (a delta-adjusted analysis). With 30% of treatment-arm values missing, each 1-point shift reduces the estimated difference by about 0.3 points.

Analysis

Assumption for missing values

Difference

95% CI

Interpretation

Primary analysis

MAR-based (δ = 0)

+6.0

+1.5 to +10.5

Favourable; CI excludes 0

Sensitivity 1

Missing treatment-arm values 3 points worse (δ = 3)

+5.1

+0.5 to +9.7

Still favourable

Tipping point

δ ≈ 4.3 points

+4.7

0.0 to +9.4

Lower bound reaches 0

Sensitivity 2

δ = 5

+4.5

−0.2 to +9.2

CI includes 0; statistical conclusion no longer holds

Sensitivity 3

δ = 8

+3.6

−1.2 to +8.4

Point estimate still positive, but the result is inconclusive

Table 3. Hypothetical delta-adjusted sensitivity analysis

These numbers are hypothetical, but they illustrate the question a dossier reviewer needs answered: how much worse would the unobserved outcomes have to be before the PRO conclusion changes?

A tipping-point analysis makes that dependence visible. Here the conclusion holds until missing treatment-arm values are assumed to be about 4.3 points worse than MAR predicts. The sponsor’s task is then to argue, with clinical reasoning, whether a shift of that size is plausible for patients who dropped out. Reporting only that “the sensitivity analysis was consistent” is far less informative when missingness is substantial.

Do not let responder analyses carry the entire PRO argument

PROs are often analysed using responder thresholds, such as the proportion of patients achieving a predefined improvement. It has been determined that a cut-off value of 15% of the scale range is required to show a substantial change in patient self-reports using responder analysis. This means that on a 0–100 scale, the value would be 15 points [2]. Responder analyses are clinically interpretable, but substantial missingness makes them vulnerable: each patient either counts as a responder or does not, so the result can depend heavily on how missing patients are classified.

For a continuous PRO, analysing the underlying score or change with an appropriate longitudinal model preserves more information. Where IQWiG assesses continuous results, it typically judges relevance using a standardised mean difference (Hedges’ g), checking whether the confidence interval lies entirely outside an irrelevance range of −0.2 to 0.2 [2]. A responder analysis can then serve as complementary evidence rather than carrying the whole quality-of-life argument.

This does not mean responder analyses should be dropped. It means their sensitivity to missing observations should be examined alongside the continuous analysis.

What an IQWiG assessment can teach: Empagliflozin

IQWiG’s 2021 assessment of empagliflozin in heart failure with reduced ejection fraction is particularly relevant because the issue directly involved health status and health-related quality of life.

Based on the EMPEROR-Reduced trial, IQWiG rated the risk of bias as high for health status (EQ-5D VAS) and health-related quality of life (KCCQ OSS). It cited a large proportion of LOCF-imputed values (27% versus 26%), plus the exclusion of patients without a baseline value or without at least one later value (7% versus 8%). IQWiG also noted that the imputation strategy was not prespecified, and that with so many imputed values, sensitivity analyses using other imputation strategies would have been useful to check robustness [5].

The lesson is not that “27% missing data automatically causes an IQWiG downgrade”. Several methodological issues were involved. The more useful lesson for sponsors is that the choice of imputation method, whether it was prespecified, and the proportion of imputed observations can all become part of IQWiG’s risk-of-bias judgement for a PRO result.

A Second AMNOG lesson: Evidence completeness matters

Mavacamten offers a different but complementary example.

In its initial 2023 dossier assessment, IQWiG concluded that an added benefit was not proven because of unclear and missing information in the dossier. Additional information was submitted during the subsequent procedure, and the 2024 addendum concluded a hint of considerable added benefit. This included a favourable result for health-related quality of life measured with the KCCQ OSS [6].

This is not a 30%-missing-PRO case, and it should not be presented as one. Its value is broader: it shows how the completeness and evaluability of evidence can affect an AMNOG assessment, and why sponsors should anticipate scrutiny of patient-relevant outcomes rather than treating the statistical analysis as a final reporting exercise.

For PROs specifically, that means designing the evidence package so that the reviewer can see not only the headline effect but also how the result behaves when its assumptions are challenged.

Design the missing-data strategy before database lock

A robust PRO strategy should be set while there is still an opportunity to influence data collection and analysis, not after the missingness has become visible.

Before database lock, the statistical team should have a documented plan covering:

  • the PRO estimand and treatment-effect definition;
  • expected patterns of, and reasons for, missingness;
  • variables to be included in the imputation model;
  • the primary analysis model;
  • the handling of intercurrent events;
  • the assumptions supporting MAR or another framework;
  • sensitivity analyses addressing departures from the primary assumption;
  • tipping-point or pattern-mixture analyses where appropriate; and
  • criteria for judging whether the PRO conclusion remains robust.

The distinction between primary and sensitivity analyses should stay clear. Sensitivity analyses should not be chosen after seeing which assumptions produce a favourable answer.

A prospective strategy makes the analysis more transparent and reduces the risk that missing-data decisions appear outcome-driven.

What 30% missing PRO Data really means for an AMNOG Dossier

Thirty percent missing PRO data is a warning signal, not an automatic verdict. The statistical risk depends on who is missing, when and why they are missing, what information predicts their outcomes, and how strongly the conclusion changes under alternative assumptions.

For pharmaceutical companies preparing an AMNOG dossier, this changes the practical objective. The aim is not to make 30% missingness disappear through imputation. It is to build a quality-of-life analysis in which the assumptions are visible, the uncertainty is quantified, and the conclusion can be challenged without collapsing.

That is the difference between a PRO analysis that merely reports a treatment effect and one that is prepared for methodological scrutiny.

How Statswork can support your PRO evidence strategy

Statswork supports pharmaceutical teams in designing and reviewing statistically defensible PRO analyses for German HTA submissions. This can include missing-data diagnostics, estimand definition, MMRM and multiple-imputation strategies, delta-adjusted sensitivity and tipping-point analyses, model diagnostics, interpretation of treatment effects, and reproducible statistical reporting.

The objective is not to guarantee an IQWiG outcome. It is to make the statistical evidence transparent, technically justified and robust enough to withstand reasonable methodological challenge.

Preparing PRO analyses for an AMNOG dossier? Talk to a Statswork biostatistician about your missing-data and sensitivity strategy before database lock.

Conclusion

A 30% missing PRO rate does not by itself sink an AMNOG dossier. What can weaken it is an unexplained missing-data pattern, an unsuitable or non-prespecified imputation approach, or a quality-of-life conclusion that changes substantially when plausible assumptions are tested. A stronger strategy begins before database lock: define the estimand, investigate the missingness, select an appropriate primary model, use multiple imputation where justified, and challenge the result with meaningful sensitivity and tipping-point analyses. For IQWiG-facing evidence, showing that the result is robust matters more than reporting how many observations were imputed.

Frequently Asked Questions (FAQs)

No. There is no general rule that a 30% missing PRO rate automatically invalidates an analysis. The assessment depends on the pattern of missingness, the analytical method, the assumptions and their impact on the certainty of the result.

No. Multiple imputation can be an appropriate method, but its validity depends on the imputation model and its assumptions. It should normally be accompanied by sensitivity analyses that examine departures from the primary assumption.

Not simply because it is easy to implement. LOCF assumes that a patient’s last observed value represents all later unobserved outcomes, which is often implausible. In the empagliflozin assessment, IQWiG rated PRO results at high risk of bias partly because of a large share of LOCF-imputed values [5]. Any approach must be justified for the specific PRO and trial.

The right choice depends on the estimand and the plausible missingness mechanism. Pattern-mixture approaches, delta-adjusted multiple imputation and tipping-point analyses show how departures from the primary assumption affect the treatment conclusion.

Before database lock, ideally during protocol and statistical analysis planning. Waiting until the final percentage of missing PRO data is known makes the analysis look reactive rather than prospectively justified.

References:

Contact us