Statistical Methods for Clinical Trial Analysis

Why Single-Point Sample Size Justifications Fail Regulators

A single sample size number can look neat in a protocol—but it often hides fragile assumptions.

What regulators typically expect to see

• Justified assumptions (with references or pilot evidence)
• Sensitivity analysis across plausible ranges
• Power curves or scenario grids showing operating characteristics
• Clear linkage to the estimand and primary analysis strategy

Practical recommendations

1. Define plausible parameter ranges
2. Present scenario tables (best / expected / worst credible)
3. Use power curves for transparency
4. Handle dropout assumptions separately

Post-hoc vs Pre-Specified Subgroup Analysis

Subgroup analysis is one of the most misunderstood—and most criticised—parts of clinical trial analysis plans.

Why regulators care about timing

• Subgroups influence treatment interpretation
• Post-unblinding decisions increase bias risk
• False-positive inflation is a key concern

Practical recommendations

1. Pre-specify only clinically meaningful subgroups
2. Document rationale in the SAP
3. Avoid confirmatory language for exploratory results
4. Anchor claims to the primary estimand

Estimands in Practice: Implementing ICH E9(R1) in Clinical Trials

The ICH E9(R1) addendum fundamentally changed how regulators expect treatment effects to be defined and interpreted. However, many protocols still describe statistical methods without clearly defining the estimand they are targeting.

What Is an Estimand?

• A precise definition of the treatment effect aligned with the trial objective
• A structured way to handle intercurrent events
• A bridge between clinical question and statistical analysis

The Five Attributes of an Estimand

• Treatment condition(s)
• Population
• Variable (endpoint)
• Intercurrent event strategy
• Population-level summary measure

Common Intercurrent Event Strategies

• Treatment policy strategy
• Hypothetical strategy
• Composite strategy
• Principal stratum strategy
• While-on-treatment strategy

Common Mistakes in Practice

• Confusing ITT analysis with treatment policy estimand
• Defining analysis model before defining estimand
• Failing to justify handling of rescue medication or discontinuation
• Misalignment between SAP and protocol objectives

Regulatory Expectations

1. Clear alignment between objective, estimand, and primary analysis
2. Explicit justification of intercurrent event strategy
3. Sensitivity analyses addressing alternative strategies
4. Transparency in documentation within the SAP

Sensitivity Analyses Under ICH E9(R1): When Are They Mandatory?

Under ICH E9(R1), the primary analysis alone is rarely sufficient to support a regulatory claim. Regulators increasingly expect evidence that conclusions are robust to alternative assumptions about intercurrent events, missing data, and modelling choices.

Why Sensitivity Analyses Matter

• Primary analyses rely on unverifiable assumptions
• Intercurrent event handling can materially affect conclusions
• Missing data mechanisms are rarely fully known
• Regulatory decisions depend on robustness, not just significance

Sensitivity vs Supplementary Analyses

• Sensitivity analyses test robustness to key assumptions
• Supplementary analyses provide additional clinical insight
• Only sensitivity analyses directly challenge core assumptions of the primary estimand

When Are Sensitivity Analyses Mandatory?

• When intercurrent event strategy relies on hypothetical assumptions
• When missing data may not satisfy MAR assumptions
• When model assumptions (e.g., proportional hazards) are uncertain
• When treatment discontinuation or rescue medication is frequent

Common Regulatory Expectations

• Pre-specified alternative intercurrent event strategies
• Tipping-point analyses for missing data
• Pattern-mixture or reference-based imputation approaches
• Clear documentation in the SAP

Practical Recommendations

1. Define sensitivity analyses at protocol stage
2. Ensure alignment with estimand definition
3. Avoid post-hoc sensitivity driven by results
4. Clearly separate robustness testing from exploratory work

The Outlier Paradox in Phase II Trial Design

When designing a Phase II trial based on pilot data, outliers often create an uncomfortable dilemma. Include them and variability increases, inflating sample size. Exclude them and variability shrinks—but the risk of underpowering the study may grow. Either choice can fundamentally alter the trajectory of a development programme.

How a Single Assumption Can Redefine a Trial

• Pilot mean difference = 10 units
• Two-sided α = 0.05, 80% power
• Including outliers: SD = 14.2 → ~32 patients per arm
• Excluding outliers: SD = 9.8 → ~16 patients per arm
• A 16-patient difference per arm driven purely by analytical handling

When Removing Outliers Backfires

• With outliers: SD = 1.87, effect = 1.50, Cohen’s d = 0.81 → ~25 per arm
• After removing extremes: SD = 1.44 (↓23%), effect = 0.94 (↓37%), d = 0.66 (↓19%) → ~38 per arm
• Removing 'noise' increased required sample size by ~52%
• Sometimes the signal is reduced more than the variability

Why This Happens

• Sample size depends on effect size (difference ÷ variability)
• Outliers may inflate variance but also inflate observed treatment difference
• Removing them can disproportionately shrink the numerator
• Effect size may decrease even if standard deviation improves

A Judgment-Led Framework for Interpreting Outliers

• Measurement or data errors → exclusion may be justified with documentation
• True biological variability → may represent the target population
• Distinct subpopulations → may justify stratified or pre-specified exclusion
• Extreme responders → approaches such as Winsorization may be explored

Designing Under Uncertainty

• Explore robust or Winsorized variance estimates
• Pre-specify sensitivity scenarios (conservative / primary / optimistic)
• Consider interim variance re-estimation where feasible
• Contextualise pilot findings with external or historical evidence
• Document assumptions transparently in the protocol and SAP

Why This Matters

• Overpowered trials expose unnecessary patients and increase cost
• Underpowered trials risk inconclusive results
• Development programmes are often shaped by early statistical assumptions

Final Reflection

• Do teams default to always removing outliers?
• Do they always retain them?
• Or is the decision case-by-case?
• Who ultimately decides—the statistician, the clinician, or both?

Bayesian Dynamic Borrowing in Clinical Trials: When and How to Use It

In early-phase and rare disease development, enrollment constraints often limit precision and delay decision-making. Bayesian dynamic borrowing provides a principled framework for incorporating external or historical control data into a current trial — while automatically down-weighting its influence if the new data disagree. When supported by comprehensive simulation-based evaluation, this approach can improve precision and potentially reduce enrollment requirements in appropriate settings.

What Is Dynamic Borrowing?

• Incorporating external or historical data into current inference
• Allowing the degree of borrowing to depend on statistical compatibility
• Automatically reducing borrowing when prior-data conflict is detected
• Providing a continuous weighting mechanism rather than binary include/exclude decisions

Why Sponsors Consider It

• Rare disease and small population trials where enrollment is constrained
• Ethical reduction of control-arm exposure
• Improved precision when historical data are high quality and comparable
• Leveraging natural history studies or registry data

Efficiency Gains — What Is Realistic?

• Effective sample size may increase when historical data are highly compatible
• Precision can improve through narrower uncertainty intervals
• Enrollment reductions are context-dependent and require scenario evaluation
• Automatic discounting protects against bias when incompatibility emerges

Common Statistical Approaches

• Power priors: discount historical likelihood by a data-driven weight
• Commensurate priors: hierarchical linkage between historical and current parameters
• Meta-Analytic-Predictive (MAP) priors: synthesis of multiple prior studies
• Robust mixture priors: safeguard against prior-data conflict

The Core Regulatory Question

• Is type I error controlled across borrowing and no-borrowing scenarios?
• Is clinical similarity between datasets justified?
• Have operating characteristics been evaluated through comprehensive simulation?
• Is sensitivity to prior assumptions explicitly assessed?
• Is the borrowing strategy pre-specified in the protocol?

Why Simulation Is Non-Negotiable

• Type I error must be evaluated under the global null
• Performance must be assessed under varying levels of historical-data bias
• Heterogeneity scenarios should be stress-tested
• Simulation-based operating characteristics form the core of regulatory review

When Dynamic Borrowing Works Well

• High-quality, clinically comparable historical data
• Stable standard-of-care environment
• Clear biological similarity between populations
• Transparent, pre-specified borrowing strategy

When It Becomes Risky

• Substantial heterogeneity across studies
• Temporal changes in standard of care
• Post-hoc borrowing decisions
• Insufficient simulation or undocumented assumptions

Illustrative Rare Disease Scenario

Practical Implementation Principles

1. Pre-specify borrowing rules and prior structures in the protocol
2. Conduct comprehensive operating characteristics evaluation across compatibility scenarios
3. Assess sensitivity to optimistic and skeptical prior assumptions
4. Document clinical justification for historical comparability
5. Engage regulators early when dynamic borrowing materially affects design

Key Takeaways

• Dynamic borrowing is a modelling framework — not a shortcut
• Efficiency gains are conditional on compatibility and rigorous evaluation
• Regulatory credibility depends on simulation-based evidence
• Transparency in assumptions is more important than magnitude of borrowing