Control Charts for Precision

Ideally, the results of duplicates should be assessed in a way that allows for the determination of what level of chance is associated with a particular difference between duplicates. This will allow for the pre-determination of limits for the allowable differences between duplicates before an investigation into the cause of the large differences is made. For example, the warning level, or the level of discrepancy between duplicates which triggers an investigation, may be set at a five percent probability. This level is a difference between duplicates that is so large that, when compared with previous precision errors, should only be observed five percent of the time. A control limit, where further measurements should cease until the problem is corrected, may be set at one percent probability.

A control chart for duplicates is not as simple as a control chart used to monitor instrument performance, with a check source. This is because the instrument’s response to a check source should be fairly constant with time. Duplicates are performed at various radon concentrations, however, and the total difference between two measurements is expected to increase as radon levels increase.

Use of statistics such as the relative percent difference (RPD; difference divided by the mean) or the coefficient of variation (COV; standard deviation divided by the mean) can be used in a control chart for duplicate measurements at radon concentrations where the expected precision error is fairly constant in proportion to the mean, e.g., at levels greater than around 4 pCi/L or 0.02 WL. At lower concentrations, for example, between 2 pCi/L (or 0.01 WL) and 4 pCi/L (or 0.02 WL), a control chart may be developed by plotting these same statistics; however, the proportion of the precision error to the mean will be greater than that proportion at levels above 4 pCi/L or 0.02 WL At concentrations less than about 2 pCi/L, or 0.01 WL, the lower limit of detection may be approached, and the precision error may be so large as to render a control chart not useful.

Before a control chart can be developed, it is necessary to know, from a history of making good quality measurements with the exact measurement system (detectors, analysis equipment, and procedures), the level of precision that is routinely encountered when the system is operating well or “in control.”  It is that “in control” precision error that forms the basis of the control chart, and upon which all the subsequent duplicate measurements will be judged. There are two ways of initially determining this “in control” level. The first, and preferable, way is to perform at least 20 duplicate pairs of measurements at each range of radon concentrations for which a control chart is to be prepared. For example, if you will only assess precision at concentrations greater than 4 pCi/L or 0.02 WL, you will need at least 20 pairs of measurements at concentrations greater than 4 pCi/L or 0.02 WL, to assess the “in control” level. The average precision error (RPD or COV) should be the “in control” level.

The second way to initially set the “in control” precision error level is to use a level that has been used by others, and that is recognized by industry and USEPA as a goal for precision, for example, a 10 percent COV (corresponding to a 14 percent RPD). After at least 20 pairs of measurements are plotted, it will become apparent whether the 10 percent COV (or 14 percent RPD) is appropriate for your system. If it is not, a new control chart (using the guidelines below) should be prepared so that the warning and control limits are set at the correct probability limits for your system.

Control charts such as those in the following examples should be employed when documenting Quality.

Sequential Control Chart Based on Relative Percent Difference

The RPD (or relative percent difference) is another expression of precision error, and is calculated by:

The control limits for RPD can be obtained simply by multiplying the control limits for COV by the square root of two, or 1.41. These limits are shown in Graph 5-2. This sequential control chart for RPD should be used in the same way as the control chart for COV, that is, with the vertical scale in units of RPD and the horizontal scale in units of date and measurement numbers.

A control chart using the statistic RPD based on an “in control” level of 25 percent RPD is shown in Graph 5-1. The warning level and control limit are set at 50 percent and 67 percent, respectively. Use of these limits may be appropriate for measured radon concentrations less than 4.0 pCi/L.

RPD = difference between two measurements divided by the average

Example: Detector A=5.0 pCi/L, B=6.0 pCi/L, RPD=18%

If RPD exceeds the control limit – cease measurements until the problem is identified and corrected.

If RPD exceeds the warning level – follow guidance in Graph 5-5.

RPD = difference between two measurements divided by the average

Example: Detector A=2.0 pCi/L, B=3.0 pCi/L, RPD=40%

If RPD exceeds the control limit – cease measurements until the problem is identified and corrected.

If RPD exceeds the warning level – follow guidance in Graph 5-5.

Sequential Control Chart Based on Coefficient of Variation

It can be shown (Iglewicz and Myers 1970, EPA 600/9-76-005; U.S. EPA 1984) that when the expected precision is a constant function of the mean, control limits can be expressed in terms of the COV (COV=S/Xm; where S is the variance or the square of the standard deviation, and Xm is the mean or average of the two measurements). One method for obtaining percentiles for the distribution of the COV is to apply a chi squared (x2) test:

Referring to an x2 chart, you learn that the probability of exceeding a x2 of 3.84 is only five percent. Inserting this value of 3.84 for x2 and solving for COVn produces a COVn of 0.20. This level of probability forms the warning level shown in Graph 5-1. The control limit corresponds to a x2 of 6.63 and a COVn of 0.26, where the probability of exceeding those values is only one percent.

This sequential control chart should be used by plotting results from each pair on the y-axis, and noting the date and measurement numbers on the x-axis.

Range Control Chart

A range control chart (Goldin 1984) can be constructed to evaluate precision, using the statistics of the range (difference between two measurements) plotted against the average of the two measurements. The control limits are again based on the variability of the measurements, as decided upon from previous results or using an industry standard (e.g., 10 percent).

In this type of control chart, the limits are expressed in terms of the mean range (Rm), n=2.

Rm = 1.128 s(x)                            (Equation 6)

where s(x) is the standard deviation of a single measurement, which reflects counting and other precision errors. Goldin shows that the limits can be expressed as follows:

Control limit = 3.69 s(x)             (Equation 7)

Warning level = 2.53 s(x)                       (Equation 8)

An example range control chart, using an assumed s(x) equal to 10 percent of the mean concentration, is shown in Graph 5-4. The chart is used by plotting the range versus average concentration as duplicate measurements are analyzed.

Interpretation of Precision Control Charts

The control chart should be examined carefully every time a new duplicate result is plotted. If a duplicate result falls outside the control limit, repeat the analyses if possible. If the repeated analyses also fall outside the control limit, stop making measurements and identify and correct the problem. If any measurements fall outside the warning level, use the table in Exhibit C-5. Refer to the row showing the number of duplicate results outside the warning level. If the total number of duplicate results accumulated in the control chart is contained in column A, investigate the cause of the high level of precision error but continue making measurements. If the total number of duplicate results on the chart is contained in column B, stop making measurements until the cause for the high precision error is found, and it is determined that subsequent measurements will not suffer the same high level of precision error.

Note that the example control charts shown here are simplifications of actual conditions, because they are premised on the assumption that the precision error is a constant fraction of the mean concentration. In fact, the total precision error may best be represented by a different function of the mean concentration, for example, the square root of the concentration. The most accurate control chart can be rendered by a range control chart using the measurement uncertainty expressed as the standard deviation, s(x), expected at the concentrations where measurements are made. If the precision error is not a constant fraction of the mean, the control limits will not appear as straight lines, but may exhibit changing slope. However, methods discussed here present a conservative way to monitor, record, and evaluate precision error and are very useful for comparing observed precision errors with an industry standard.

If results exceed the control limit – cease measurements until the problem is identified and corrected. If results exceed the warning level – follow guidance Graph 5-5 shown below.