LC-MS/MS in the clinical laboratory delivers a powerful instrument with the ability to separate analytes of interest on orthogonal technology (chromatography and mass spectrometry), deliver on a wide selection of sample preparation and chromatography chemistries, and selectively identify species by their mass. Coupled with a wide quantitative range and minimal sample volume, these devices are especially attractive when developing an assay not otherwise available on the commercial market. Effective quality rules are a critical component in LC-MS/MS to manage this complexity.
Basic and Advanced Quality Rules
Based on the various assay components, basic rules (Figure 1, left) are easy to describe within an SOP and easy to implement with basic software. Observations which affect one sample do not necessarily inform the user of a problem with all samples, nor do the failures of one compound necessarily provide information about the batch. Advanced quality rules involve a multi-dimensional understanding of how the system operates over time, or the correlated relationships between basic rules, or the manifold associations that can act across multiple analytes (Figure 1, right). For more background on both rule types, and on the risk mediation that rules address, please see the first article in this series.
Figure 1. Basic (left) and advanced (right) quality rules.
The least documented of these advanced quality rules are those which rely on the broader sample context and how one observable characteristic influences another. Although any given rule will have both acceptable and impermissible exceptions to justify (or refute) the passing of an identified observation, the exceptions are rarely captured in the assay documentation. All the data for making the pass/fail decision exists within the instrument and the laboratory information system (LIS), but the exceptions-in-practice are applied based on reviewer judgment.
Case Study: Ion Ratio in Context
This case study focuses on ion ratio values collected from 160,000 samples across 14 months of production data. The ions present in the mass fragmentation pattern are anticipated to be compound dependent but concentration independent. Therefore, ion ratio in an LC/MS-MS context involves observing both the fragment used for calculating concentration (base ion) and a second fragment (confirmation ion) with the expectation that the ratio is constant. When this is not true, the common interpretation is that one or both signals are detecting some other fragment and causing interference. Ion ratio monitoring is the most common confirmation method for establishing compound identity.
Basic Quality Expectations
Some variation in base and confirmation peak area is expected between samples of the same concentration, which in turn creates some variation in the ion ratio. A review of multiple guidance documents shows that permissible ion ratio tolerance can be based on the relative peak areas of the base and confirmation ions. When the peaks are within a factor of two, tolerance is relatively tight at ±20 percent of the target value. The smaller the confirmation ion, the wider the tolerance, which expands to as much as ±50 percent when the confirmation ion peak is less than a tenth of the base ion peak. The compound in the first example (Figure 2) has a mean ratio of 0.52 and a symmetric distribution around that value. Treating the mean as the target and applying a ±20 percent tolerance around it results in 83 percent of the observations having an acceptable ion ratio.
Figure 2. Histogram of ion ratio values for a single compound. Dashed lines are ±20 percent of the target ratio.
When a fifth of the observations fail to meet the tolerance in what should be a well-behaved system, it calls to question whether the ion ratio is universally appropriate for monitoring sample health. Assuming the peak areas are part of a continuous distribution, reinjecting the sample is a simple solution which delivers new peak area measurements. This second ion ratio may be within tolerance while the first was not. Unfortunately, the inverse of this decision is disregarded: a sample where the ion ratio is within tolerance does not get reinjected in order to see if the subsequent observation is out of tolerance.
As an alternative to reinjection, it may be that the ion ratio distribution is concentration dependent. Sorting the same data into quartiles by calculated concentration (Figure 3) shows the upper half of the observations are significantly more successful in aligning the ion ratio with expected values. Less than three percent of these concentration observations fall outside the permitted tolerance. This concentration dependence breaks one of the initial assumptions about ion ratio, which could mean either the method is failing to operate correctly or these low concentration samples are fundamentally different from the upper quartiles.
Figure 3. Same compound as above, sorted to four quartiles by concentration. Grey box defines ±20 percent of the target ratio.
A second compound in this production dataset displays a bimodal distribution of observations (Figure 4) and is even less well behaved than the first example. Ion selection for the base peak and confirmation peak is subject to a variety of factors and base area can sometimes be smaller, leading to this split visualization so that ion ratio is always between zero and one. However, the bimodal distribution suggests either a complex peak area behavior or a dramatic tendency for unsuccessful data collection. Almost half the observations fail to meet the permitted tolerance of 0.88 ±20 percent. There is some improvement in behavior when sorting by concentration, but each of the concentration quartiles shows this distribution. By itself, ion ratio is insufficient for this compound to establish whether an observation is suitably acceptable.
Figure 4: Compound with a more complex distribution. Because the base peak is not always larger than the confirmation peak, the data is divided into two graphs so the ratio is always between 0 and 1. The target value is set according to the largest histogram bin, where confirmation area over base area is 0.88.
Advanced Quality Expectations
An advanced quality rule operates on chromatographic knowledge in context. Peak area is not the only quantity measuring ion similarity. It could even hide cases when contamination contributes equally to each peak. The cultural tendency to forgive an ion ratio warning as long as nothing else is apparently wrong in the data emphasizes the importance of secondary analysis for these exceptions-in-practice.
To correct for poorly predictive ion ratios, one alternative would systematically apply additional shape-based criteria in the base and confirmation peak comparison. The peak asymmetry (left/right half-width) and peak aspect (width/height) can be monitored independently for changes relative to the anticipated shape. Reevaluation of the compound in Figure 4, using a tiered similarity approach with a ±10 percent threshold along both the asymmetry and the aspect, brought the number of manual interventions down from 46 percent of all observed (cases with a failed ion ratio) to two percent (cases which failed the ion ratio, and then the asymmetry ratio, and then the aspect ratio). Only 3,200 of the 160,000 observations—just 50 a week—are left for technical review.
Figure 5. Modeling of overlapped peaks.
Advanced quality monitoring permits treating the data in context and taking advantage of all the context available to the system. A more complex analysis of peak shape beyond “width at” or “height at” which uses a peak modeling approach can compare the quality of fit between model and observation. One example is shown in Figure 5 which required multi-peak modeling within a given data stream to accurately establish the peak area of each component prior to starting the active data review process.
Advanced Quality Automation
Peak shape thresholds do not need to be restricted to uniform constraints. The ion ratio distributions in Figures 2 and 4 are continuous—even if the second compound appears bimodal—which means the generalized ±20 percent threshold in these compounds is an arbitrary limit based on desired expectation and not actual observation. This kind of limitation is no different than the desire to reject data more than three sigma away from the mean of a Gaussian—three out of every 1,000 observations are behaving correctly but get treated as if they are in error.
The ±20 percent ion ratio threshold could be replaced by a sliding scale dependent upon the concentration. Weaker peaks are inherently noisier, which might be accommodated using a larger threshold percentage. At the other end of the scale, peaks which nearly saturate the detector may have excessive per-point fluctuations that also expand the threshold. And well-behaved peaks in the middle range could narrow the threshold, all based on observed compound behavior.
Using a rigid threshold for the peak asymmetry and peak aspect terms, which capture manual evaluation constraints, is equally problematic. A factorial combination of all such observations, applied automatically, would deliver only those samples which failed to match expectations seen during assay development. Expanding the definition of similarity to apply a focused peak model instead of specific width/ height/ area geometries further improves the dimensions of the similarity measurement. Tracking the dependencies of these terms is easily automated, and helps ensure the peaks which report a concentration are based only on the observation of that compound.
Concluding Remarks
Exceptions to a quality rule are often the most difficult to address. The problem does not lie in the need for an exception, but rather in the frequency with which the exception is needed. The expedient response which expands the tolerance to accept a wide distribution of observations, as seen in another case study regarding internal standards, means the tolerance has no functional meaning. Alternatively, when the majority of the exceptions become accepted, as often happens when examining the ion ratio, the relevancy of the rule is called into question. Labeling an observation as an exception diminishes the importance of the basic rule. Treating the measurement as one facet of a complex rule acknowledges secondary considerations, which should be described and codified in equally persistent detail. Solutions exist today to enable this approach, allowing the lab to move past a reliance on exceptions and consider the system as it really is. In the next case study, basic quality rules focused on a single compound are expanded into advanced quality rules that consider the presence or absence of all compounds under observation, creating a report on metabolism or adulteration.
