Method Comparison

Method comparison involves testing patient samples during a number of different analytical runs by both the new and current methods. Ideally, the comparison method should be a reference method, but usually it is the existing method in one's own laboratory or a reference laboratory. Method comparison should be combined with the between run precision study. At least 40 patient samples should be analyzed by both methods with at least 2 reagent lots on each analyzer. Any specimen with large differences between results of the two methods should be reanalyzed by both methods in the next run. Analyte concentrations should be distributed throughout the analytical range, represent a broad range of disease states, and be from both men and women.

The results are plotted in Excel using an XY plot. The test method (new) is plotted on the Y axis (dependent) and the reference (existing) method on the X axis (independent). A best fit line is inserted through the data points and the slope and Y intercept are calculated. The best fit line is defined by the equation; Y = mx + b, where m is the slope and b is the Y intercept. A perfect correlation will have all points lying on a line at a 45o angle to the X axis. This line will have a Y intercept of 0 and slope of 1. The correlation coefficient will be 1 and the standard error will be 0.

Unfortunately, most method comparisons are imperfect. In this situation, the slope estimates proportional error and the Y intercept estimates constant error between the two methods. If the points are widely scattered about the line, there is a significant amount of random error between the 2 methods. The types of error detected by method comparison studies are summarized in the following table.

Parameter	Random Error	Constant Error	Proportional Error
Slope	No	No	Yes
Y Intercept	No	Yes	No
Standard Error	Yes	No	No
Correlation Coefficient	Yes	No	No

Random error is a mistake in one test or test run that is independent of another test or test run. Correlation coefficient and standard error are used to estimate random error. Standard error indicates the scatter of points about the regression line. It is calculated as the standard deviation of the difference between the test and reference method values. Standard error can be a positive or negative value and can be due to the reference method, test method, or matrix effect. The correlation coefficient is an estimate of the degree of association between the two methods. An r = 1.00 indicates a perfect positive correlation, r = 0.00 indicates no correlation, and r = -1.00 indicates a perfect negative correlation.

In proportional error, the new method is a fixed percentage of the reference value as the level of analyte is increased. For example, if the reference values for glucose samples are 100 and 200 mg/dL and the new method values are 110 and 220 mg/dL, proportional error is 10%. Proportional error is defined as deviation from the ideal slope of 1.00. It is calculated as (slope - 1) x 100. For example, a slope of 0.98 indicates a -2% proportional error. Proportional error always occurs in one direction and can often be corrected by adjusting the calibration.

With constant error, new method values stay above or below reference values by a fixed amount as the level of analyte is increased. For example, if the reference values for two glucose samples are 100 and 200 mg/dL, the new method values might be 130 and 230 mg/dL. Constant error is indicated by the Y intercept. In the absence of constant error, the regression line passes through the origin and the Y intercept is 0.00. The amount of deviation from the origin indicates the degree of constant error. If the Y intercept is 2 mg/dL, then this much difference exists between all X & Y values. Constant error can often be corrected by adjusting the blank.

Systematic error is the sum of constant and proportional error and is an indication of accuracy. Total analytic error is random error plus systematic error. The ultimate goal, which may not always be attainable with current technology, is to have a total analytic error that does not exceed total allowable error. Appendix B lists the total allowable error for many analytes. Alternatively, it can be estimated by multiplying intra-individual variation by 0.5. Intra-individual variation is listed in Appendix A.

Limitations of Linear Regression:
Linear regression is most commonly used to analyze method comparison data. The following limitations must be observed to ensure valid interpretation of the results.

Data points must be limited to the linear range. The spread of sample values should test the total linearity of the instrument. Ten specimens should be tested at the high end, 20 in the middle, and 10 at the low end.
The data range must be broad enough to allow reliable linear regression. For an ideal linear regression it is best to have data where the highest value is three times the lowest value. Therefore, linear regression is not a good tool to compare methods with a narrow analytical or clinical range (e.g. sodium or potassium).
Outliers must be identified and reanalyzed. The linear regression line is "pulled" toward the outlier, with the greatest effects caused by outliers at the extremes of the data. No more than one outlier should be excluded in a set of 40 patient comparisons.




Introduction Initial Questions Linearity Precision Interference Recovery Method Comparison Lower Limit of Detection Carryover Reference Ranges Periodic Reevaluation Second Instrument of Similar Make Appendix A Appendix B