Back to the list

Methods Validation for Verification Testing in Medical Devices

Medical devices regulation

In the field of medical devices, method validation is essential to ensure the reliability and accuracy of verification testing. Validation ensures that test results meet regulatory requirements and standards, which is critical for patient safety and device effectiveness. In other words, the data must be correct, and the measurements must be accurate.


Necessary Vocabulary for Method Validation

Systematic Error: Systematic error, also known as bias, refers to a constant and predictable deviation of the measured value from the true value. It can be caused by factors such as:

  • Incorrect calibration of equipment
  • Constant influence of an environmental factor
  • Bias in the measurement method

Systematic error primarily affects the accuracy (trueness) of the measurements.



Random Error: Random error is the variability of measurements caused by unpredictable and uncontrollable factors. It can result from:

  • Environmental variations
  • Fluctuations in measurement conditions
  • Human errors

Random error primarily affects the precision (repeatability) of the measurements.



Precision: Precision refers to the dispersion of measurements when they are repeated. A measurement is considered precise if the results are close to each other, regardless of their proximity to the true value. Precision includes two aspects:

  • Repeatability: Consistency of results under identical conditions.
  • Reproducibility: Consistency of results under varying conditions.

Low random error indicates high precision.



Accuracy: Accuracy (trueness) refers to the closeness of measurements to the true value. It includes:

  • Systematic Error: The difference between the average of measurements and the true value.
  • Random Error: The variability of measurements around the average.

A measurement is considered accurate if it is both precise (low random error) and unbiased (low systematic error).



Relationships and Interactions between those terms

1. Systematic Error and Accuracy: Systematic error determines the bias of the measurements. A large systematic error means that the measurements are consistently deviated from the true value, reducing accuracy.

2. Random Error and Precision: Random error determines the dispersion of the measurements. A large random error means that the measurements are scattered, reducing precision.

3. Precision and Accuracy:

  • Precision is necessary but not sufficient for accuracy. A measurement can be precise (consistent results) but inaccurate (far from the true value) if it has a systematic error.
  • To obtain accurate measurements, both systematic and random errors must be minimized.

The illustration of these two types of errors (systematic and random) is often represented as targets as shown below:

  • Center: Represents the true value.
  • Around the center: Indicates limits (acceptance criteria in validation, alert threshold, specification limits, etc.).
  • Red points: Represent the measurements taken.



Evaluation of Influencing Factors

To ensure rigorous validation, it is important to consider the “5M”: Machine, Manpower, Material, Method, and Milieu (environment).

  • Machine: Equipment performance must be regularly calibrated and maintained to ensure test accuracy. The potential variability between different machines must also be evaluated.
  • Manpower: Operator competence is crucial. Adequate training and regular performance evaluation can minimize human errors.
  • Environment: Environmental conditions such as temperature, humidity, and cleanliness must be controlled, as they can affect test results.
  • Material: The quality and compatibility of materials used in the tests can affect the results. It is essential to ensure that materials meet the required specifications.
  • Method: Test procedures must be well-defined and standardized to avoid variations due to differences in test methods. Using design of experiments (DoE) can help optimize test conditions.



Test Method Validation (TMV)

Test method validation (TMV) is a process that demonstrates that:

  • The Test Method (TM) is appropriate for measurement applications.
  • Test equipment operates consistently and delivers consistent results.
  • The TM can differentiate conforming from non-conforming (N/C) products.

There are two main types of test method validation:

  • ATMC (Attribute TMV)
  • Used for attribute tests, such as visual inspections.
  • Ensures that attribute-based test methods (pass/fail) are precise and reproducible.
  • VTMV (Variable TMV)
  • Used for variable tests, such as measurements of weight, length, etc.
  • Ensures that test methods provide precise and reproducible quantitative measurements.



Repeatability and Reproducibility (R&R) Method

  • Repeatability: Measures the variation in results obtained with the same method, operated by the same operator, on the same sample, and with the same equipment under constant conditions. Repeatability is a measure of intra-operator precision.
  • Reproducibility: Measures the variation in results obtained with the same method but under different conditions (different operators, equipment, days). Reproducibility is a measure of inter-operator or inter-laboratory precision.



Statistical Methods for Method Validation

Average & Range Method

  • Average: Calculate the average of several measurements.
  • Standard Deviation: Calculate the standard deviation of the results to assess variability.
  • This method helps determine if observed variations are due to differences between operators, machines, or other environmental factors (see the 5M).



Analysis of Variance (ANOVA)

  • Compare Variations: Evaluate whether the variations observed between different groups (e.g., different operators or machines) are statistically significant.
  • Identify Sources of Variability: Determine which sources of variability (e.g., machine, operator, environment) have the greatest impact on the results.
  • ANOVA helps to better understand sources of variation in the measurement process and identify areas needing improvement.



Evaluating the Measuring Process (EMP)

  • Data Collection: Gather data from multiple measurement repetitions.
  • Statistical Analysis: Use statistical tools to analyze the data and assess the precision and accuracy of the measurement process.
  • Acceptance Criteria: A measurement process typically needs to have a total variability (coefficient of variation) of less than 10% to be considered acceptable.
  • EMP ensures that the measurement process is precise enough for the application’s needs and can provide reliable results.



Use of Design of Experiments (DoE)

Design of experiments allows the evaluation of the impact of several factors simultaneously and identifies interactions between these factors. This helps optimize test conditions to obtain reliable results.



Robustness Analysis

The robustness of a method is evaluated by introducing small variations in test conditions and observing if the results remain consistent. This includes variations in reagents, environmental conditions, and procedural parameters.



Outsourcing to Accredited Laboratories

To ensure rigorous validation, tests can be outsourced to laboratories accredited according to ISO 17025. These laboratories have the technical skills and management systems needed to perform reliable and precise tests. Accreditation ensures that the laboratory complies with international quality and competence standards.



Validation Process

1. Planning: Define the objectives of validation, acceptance criteria, and methods to be used.

2. Execution: Perform tests according to established plans, considering all influencing factors.

3. Data Analysis: Use statistical tools to assess the precision, accuracy, specificity, and sensitivity of the methods.

4. Documentation: Document all aspects of validation, including results, analyses, and conclusions.

5. Review and Approval: Submit the documentation for review and approval by experts and regulatory authorities.



Conclusion

The European Regulation MDR 2017/745 has tightened the requirements regarding the method validation for verification tests of medical devices. Method validation for verification testing is a critical step in the development and deployment of medical devices. By considering influencing factors such as machine, manpower, material, method, and environment, and using rigorous methods like R&R, ANOVA, and EMP, manufacturers can ensure reliable and regulatory-compliant results.


CSDmed is at your disposal to assist you in the method validation process. Our experts can help you design and execute robust validation plans, and outsource tests to ISO 17025 accredited laboratories. Contact us for more information and to discuss your specific method validation needs.


For any further assistance or to obtain specific visuals, please do not hesitate to contact us.