Near-cutoff precision

Run this study#

Your question: where does the result change from negative to positive (C5, C50 and C95)?

Bring: samples with assigned concentrations spanning the positive/negative transition, and each individual qualitative result.

Suggested starting plan: 5 concentrations × 20 replicate tests = 100 results. Run a short pilot to find the transition. Choose a lowest level that is negative in nearly every test and a highest level that is positive in nearly every test, beyond where you expect C5 and C95. Add three levels between them, one near the expected midpoint. C5 and C95 are estimated only inside the tested range.

Calculation minimum: a curve needs 3 concentrations with reportable results, some of them mixed positive and negative. If results jump straight from all negative to all positive, add concentrations inside the gap. Wide or open-ended intervals mean the transition isn't pinned down yet. The FDA guidance shows why panels include samples near both C5 and C95.

1. Decide first. Create the study. On Set up, declare the negative and positive labels, any invalid or equivocal labels and the concentration scale. Under Acceptance limits, choose Enter your own limit and set the acceptable C5, C50 or C95 range. Note the assay's current cutoff and enter it as Assay cutoff (optional) for comparison.
2. Prepare each concentration. Keep its assigned value, unit and preparation record. Give each sample one concentration and its own ID.
3. Run the planned repeats. Record every result, including invalids. If testing spans several days, record the day. Results at the same concentration are pooled across days.
4. Import and calculate. On Data, choose Import a file and choose your file (start from Blank template (CSV) or qualitative_cutoff.csv). Answer any question the dialog asks, then choose Import into this study. Choose Calculate results and check the reportable counts before reading the curve. On Report, choose Download PDF.

Purpose#

Near its cutoff, a qualitative test can call the same sample positive one time and negative the next. This study finds the concentrations at which the test calls positive 5 %, 50 % and 95 % of the time: the C5, C50 and C95. The span from C5 to C95 is the uncertain zone.

The hit rate is the share of positive results at a concentration. A fitted S-shaped (probit) curve gives C5, C50 and C95, each with a confidence interval.

When to use it#

Use it to learn how reliably a qualitative test calls results near its positive/negative cutoff. To compare two methods on patient specimens, use Qualitative agreement. For the lowest detectable concentration of a quantitative method, use Detection limits.

Study setup#

1. In your project choose New study. Enter Analyte and Unit, and select Near-cutoff precision under Qualitative tests.
2. Choose Create study, or Create and import data if you already have results.
3. On Set up, under Qualitative performance (Study mode is already Near-cutoff binary precision (C5/C50/C95)), fill in:
   • Result categories · declared order: exactly two, for example negative, positive. Positive category: one of them.
   • Nonreportable labels (default equivocal, invalid, indeterminate): counted for each concentration and left out of the hit rate. Case and surrounding spaces are ignored.
   • Concentration scale: Linear concentration, or log10 concentration (values above 0) for concentrations spaced by multiplication (1, 2, 4, 8) or spanning a wide range. It changes the estimates, so choose it before looking at results.
   • Assay cutoff (optional): shown beside the C5–C95 interval for comparison.
4. Check Confidence level in Calculation settings (95 % by default, or 90 % or 99 %).
5. Under Acceptance limits, choose Enter your own limit and set a range for C50 concentration, C5 concentration or C95 concentration (see Acceptance limits).

Entering data#

One row per replicate test:

Rows are grouped by exact concentration.

Choose Import a file. Blank template (CSV) gives a matching empty file. See Importing and mapping.

Concentrations need exact values. Correct a value such as <0.01, or exclude the row with a reason. Correct any undeclared result label, or declare it on Set up. See Corrections and exclusions.

You still get hit rates, but no C5, C50 or C95, in these cases. A note on Results gives the reason.

• Fewer than 3 concentrations with reportable results.
• Every concentration all negative or all positive, or all-negative levels followed directly by all-positive ones with at most one mixed level between them (separation).
• Hit rates that don't rise with concentration.
• A fit that doesn't converge.

Statistics#

• Hit rate at each concentration = positives ÷ reportable replicates, with a two-sided Wilson score interval (no continuity correction).
• Probit model P(positive) = Φ(a + b·x), x = concentration or log10 concentration, fitted by maximum likelihood on the grouped counts (Newton–Raphson on centered x, step-halving, convergence below 1e−10, at most 100 iterations). Covariance = inverse observed information.
• C_p = (Φ⁻¹(p) − a)/b for p = 0.05, 0.50, 0.95, back-transformed as 10^x on the log10 scale.
• Fieller confidence limits as in SAS/STAT PROC PROBIT, with z = Φ⁻¹((1 + confidence)/2). They are reported when b² − z²·Var(b) > 0 and the discriminant is positive. No heterogeneity scaling.
• Pearson chi-square with m − 2 degrees of freedom, for information.

See Methods and sources.

Acceptance limits#

The dialog and its choices are in Acceptance limits. For this study:

• The limit is compared with the C5, C50 or C95 estimate itself. The Fieller limits don't enter.
• A limit is Undecided when the estimate couldn't be calculated, lies outside the tested concentrations or has open-ended Fieller limits, or when any included replicate is nonreportable. The FDA guidance, section 6.2 explains the bias from leaving such results out.
• See Criterion outcomes.

Worked example#

qualitative_cutoff.csv: 105 synthetic urine hCG replicate results (mIU/mL), 15 per sample at 7 concentrations (6, 8, 9, 10, 11, 12 and 13). One replicate at 10 is equivocal.

Setup: default categories and nonreportable labels, Positive category positive, Linear concentration, Assay cutoff (optional) 10, 95 %. One acceptance limit: highest acceptable C95 concentration 15 mIU/mL.

What the saved result shows#

• Probit curve fit: intercept a −6.5150, slope b 0.6630, Pearson chi-square 0.880 with df 5.
• The assay cutoff, 10 mIU/mL, lies within the C5–C95 interval.
• Counts: 7 concentrations, 104 reportable results, 1 nonreportable, 7 samples.
• The C95 limit (≤ 15 mIU/mL): Why shows the estimate as 12.31 mIU/mL, but the outcome is Undecided because one replicate was nonreportable. The study is Undecided.

On log10 concentration (values above 0) the same file gives C5 7.55 (6.44 to 8.20), C50 9.72 (9.19 to 10.22) and C95 12.53 (11.65 to 14.32) mIU/mL, Pearson chi-square 1.314 (df 5).

Reading the results#

Hit rate by concentration plots each hit rate with its Wilson interval and the fitted curve. Guides mark 5, 50 and 95 %, markers show C5, C50 and C95, and a line shows the cutoff if you entered one.

Per-concentration results gives the counts, hit rate and Wilson CI at each concentration. A concentration with only nonreportable results has no hit rate and is left out of the fit.

C5 · C50 · C95. At C50 a result is equally likely to be positive or negative. A well-placed cutoff is usually near it. The Fieller limits are wider for C5 and C95, because the tails are less well determined.

• outside studied range marks an estimate beyond the tested concentrations.
• Unbounded Fieller limits mean the slope is too uncertain to bound that concentration.

Probit curve fit. A Pearson chi-square much larger than its degrees of freedom suggests a poor fit. It is for information only.

Look out for few replicates per concentration, no concentration with a hit rate between about 20 % and 80 %, or a cutoff outside C5–C95.

What to do when the study fails#

• Criteria not met: recheck concentrations, unit, Positive category and Concentration scale.
• Undecided or no curve (separation): add concentrations between the all-negative and all-positive levels, or beyond an extreme estimate, and retest.

Common mistakes#