
Data Analysis services

Meta-Analysis Research Services

Data Collection Services

Statistical Programming & Biostatistics services

Data Management Services

Research methodology services

Tool development services
Statistical Interpretation services

Statistical Interpretation services
Sample Size Calculation Services

Sample Size Calculation Services
Artificial Intelligence and Machine Learning Services

Artificial Intelligence and Machine Learning Services
Report generation Service

Report generation Services

Data Analysis services

Meta-Analysis Research Services

Data Collection Services

Statistical Programming & Biostatistics services

Data Management Services

Research methodology services

Tool development services
Statistical Interpretation services

Statistical Interpretation services
Sample Size Calculation Services

Sample Size Calculation Services
Artificial Intelligence and Machine Learning Services

Artificial Intelligence and Machine Learning Services
Report generation Service

Report generation Services
Summary
A static, deterministic machine-learning (ML) model can produce identical outputs and still approve a non-conforming batch. Irish pharmaceutical manufacturers need evidence of configuration control, execution repeatability and predictive performance. The July 2025 consultation draft proposes AI-specific GMP guidance, while the EMA’s October 2026 workshop report records stakeholder input intended to inform the ongoing revision of Annex 22. Neither should be mistaken for a final, binding Annex 22 requirement.
AI in pharmaceutical manufacturing requires risk-based validation when models support GMP-critical decisions. EU GMP Annex 22 AI validation must consider intended use, model performance and quality risk. Effective machine learning validation in pharmaceutical manufacturing must establish whether the approved model performs reliably under intended operating conditions and whether the complete application supports controlled, traceable decisions.
The European Commission opened its consultation on revised GMP Chapter 4, Annex 11 and proposed new Annex 22 on 7 July 2025; it closed on 7 October 2025. The draft addresses model selection, intended use, validation, training and test data, performance monitoring and change control. These remain proposed provisions. [1]
The EMA’s multistakeholder workshop took place on 30 June and 1 July 2026, with its report published on 2 October 2026. Discussions covered adaptive and probabilistic models, guardrails, human oversight, validation and lifecycle management. The report informs guidance development but does not establish binding requirements. [2]
The July 2025 draft excludes generative AI/LLMs, probabilistic, dynamic and adaptive models from critical GMP use. The 2026 EMA workshop explored a risk-based approach to these technologies, but no final permission was established. Static, deterministic models may fit the draft’s scope more closely, but suitability and regulatory acceptance are not guaranteed.
These are three distinct validation questions.
Property | Evidence required | What it does not prove |
Static configuration | Approved version, parameters and change history | Correct predictions |
Deterministic execution | Repeatable outputs under defined conditions | Performance on unseen cases |
Predictive performance | Independent testing against reference outcomes | Permanent future performance |
A model may repeatedly classify a defective batch as acceptable. Its output is deterministic, but its prediction is wrong.
A frozen model can also run within a changing application. Preprocessing, software dependencies, input systems and decision thresholds may alter results. Statistical evidence supports, but does not replace, wider GMP validation, including software lifecycle controls, data integrity, integration testing, audit trails and authorised release procedures.
Define the model’s decision authority explicitly. A model used to prioritise quality investigations presents a different risk from one whose output directly determines whether a batch may be accepted. Document how the output interacts with approved specifications, deviation handling, human review and the authorised batch-release process.
NIST’s voluntary AI Risk Management Framework supports principles of validity, reliability and representative testing, but is not a pharmaceutical GMP standard. [3]
Document the model artefact, version, learned parameters, preprocessing logic, thresholds and relevant software dependencies. Cryptographic hashes can help detect model-file changes but do not prove that the complete deployed application remains unchanged or scientifically valid.
Restrict update permissions, retain approved versions and document change authorisation. Retraining, parameter updates and preprocessing changes should trigger impact assessment and applicable change control and revalidation.
Maintain traceability from intended use and risk assessment through requirements, test cases, acceptance criteria, deviations and final approval.
Validate the model through a structured workflow covering configuration control, repeatability, independent performance testing, statistical risk assessment and ongoing monitoring, as illustrated below.
Test the complete application pathway, not just model inference. Repeatedly running one input 2,000 times cannot demonstrate performance across 2,000 independent production cases.
For GMP batch-release model validation, freeze the model and analysis protocol before examining final test results. Keep test data independent of training and model selection, and assess leakage involving related batches, products, sites and time periods.
Choose metrics according to the consequences of incorrect decisions. Sensitivity measures the proportion of genuinely non-conforming cases detected; specificity measures the proportion of conforming cases correctly identified. Report confidence intervals and prespecified acceptance criteria. FDA, Health Canada and MHRA Good Machine Learning Practice principles provide supplementary methodological guidance but concern medical devices rather than pharmaceutical manufacturing. [4]
Suppose an independent test set contains 200 non-conforming cases, of which the model flags 190.
“Sensitivity”=190/200×100%=95%
The observed false-negative rate is 10/200 = 5%. If an unflagged case can proceed to batch acceptance, these misses represent potential false acceptances.
The approximate 95% Wilson confidence interval for sensitivity is 91.0%–97.3% [5]. If a predefined criterion requires a lower confidence bound of at least 93%, the model fails despite its 95% point estimate.
This is an illustrative rule, not an Annex 22 threshold. Acceptance criteria must be justified by intended use and quality risk and specified before final testing.
Sensitivity alone does not establish the probability that an accepted batch is non-conforming. That probability also depends on prevalence, specificity and the decision pathway. Analyses must account for dependence when observations are clustered by batch or production line.
Pooled performance can conceal weaknesses. Where relevant, evaluate performance by product or strength, production line, equipment, raw-material characteristics, operating conditions and defect type.
Validation finding | Appropriate response |
Configuration mismatch | Block use pending investigation |
Repeatability failure | Investigate execution |
Performance criterion missed | Investigate errors and revalidate as justified |
Subgroup weakness | Restrict scope or validate mitigation |
Monitoring alert breached | Assess impact and escalate |
Justify subgroup selection using intended use, process variation and quality risk. Where sample sizes are insufficient, report uncertainty rather than assuming acceptable performance.
Define responses to missing values, out-of-range inputs and low-confidence predictions. Depending on the validated design, these may require input rejection, escalation or authorised human review. Do not relax acceptance criteria retrospectively to obtain a pass.
Monitor input distributions, missing-data rates, output patterns and performance when reference outcomes become available. Distinguish input drift from demonstrated deterioration in predictive performance.
Define monitoring frequency, alert thresholds, escalation responsibilities and suspension criteria. Link these controls to pharmaceutical quality risk management, applicable computerised-system requirements and data-integrity controls, including relevant Annex 11 provisions.
Statswork’s Data Analysis service supports Irish pharmaceutical manufacturers with statistical evidence for risk-based machine-learning validation. Support can include:
A structured readiness assessment can review intended use, available test data, acceptance criteria, subgroup coverage and monitoring plans. Agreed outputs may include a gap summary, prioritised findings, further-testing recommendations and a validation protocol with reproducible calculations and performance tables.
Assess your batch-release ML validation readiness
Reach Statswork’s data analysis service to discuss your model’s intended use, test-data coverage and statistical validation requirements. The assessment can inform validation and quality-review activities but does not guarantee regulatory acceptance or replace the manufacturer’s responsibility for GMP compliance and authorised batch-release decisions.
Validating a static ML model requires more than repeatable outputs. Manufacturers need evidence of configuration control, consistent execution, acceptable predictive performance on independent data and ongoing lifecycle oversight. For Irish pharmaceutical plants preparing for evolving Annex 22 guidance, risk-based statistical validation makes model behaviour measurable, traceable and assessable for its intended GMP use.
The July 2025 consultation draft proposes controls and restrictions for AI in critical GMP applications. The final applicable regulatory position must be established from official guidance.
No. Repeated execution tests reproducibility; predictive performance requires evaluation against appropriate reference outcomes.
Sensitivity, specificity, false-negative rate and confidence intervals may be relevant. Criteria should reflect the consequences of incorrect decisions.
Yes. Parameters can remain unchanged while incoming data or manufacturing conditions change.
Statswork can support validation study design, independent test-data assessment, performance analysis, subgroup evaluation and monitoring criteria through its Data Analysis service.
WhatsApp us