Where One Term Ends and the Next Begins

A Seven-Part Series on Measurement Uncertainty, CMC, Bias, TUR, Decision Rules, Guardbands, and PFA/PFR

A customer or someone who I hope is a friend wrote to me with a suggestion. There is a messy mix of terms in conformity assessment, and no single place that says where each one stops, how they connect, and when to use which. This series is the answer. It takes several articles and documents we have worked on, adds some new material, and tries to put everything together into a series of posts, where we hope breaking this up into digestible pieces helps someone put it all together. Part 1 is the map. Parts 2 through 7 each take one boundary that gets crossed most often and walk it with the same force-calibration examples used in the Decision Rule Guidance, the GUM in Plain English series from Morehouse, and the Guidance on Uncertainty Budgets for Force Measuring Devices. Read them all together or individually. Each part stands alone, or at least it should; if it does or doesn't, you decide, then email us your answer with comments at info@mhforce.com. We prefer nice comments to nasty ones, though we can take negative criticism well on Thursdays.

Part 1: Calibration Decision Rules in One Chain

"If you choose not to decide, you still have made a choice." Neil Peart wrote that line for a song about free will, and we have quoted it in our decision rule guidance for years. It fits because every pass/fail statement is a decision, whether or not anyone admits to making one. And for those that say, what about non-binary decision rules, I will say, eventually someone needs to make a binary decision. Are you going to see x movie in the theatre? Your answers could be yes, no, maybe. Eventually, the movie leaves the theatres, and you saw it, or you did not. Okay, if it's Star Wars, the maybe could have lasted 50 years, though that is an exception to the rule, which, if we adjust the question to “Are you going to see the movie upon initial release?”, we solve the issue.  These metrologists always try to get me with something. So, I have to triple-check everything, and I can 100 % guarantee there’s a mistake somewhere in all of this. Dilip, Craig, Jeff, Ryan, Andreas, Rob, Kevin, Klaus, Heather, Joseph, Eric, Phil, Paul, Cory, Ginger, Danae, John, Daniel, Chuck, Stephen, Seth, Howard, Shawn, Gary, Daniel, Mike, Tim, Paul, Han, Chewbacca, whatever your name is, or Greg C will find one for sure.

A customer/friend recently told me why the deciding is so hard. His words: there seems to exist a messy mix of measurement uncertainty, CMC, TUR, guardband, bias compensation, decision rule method, PFA and PFR, and there does not seem to exist a clear set of rules defining the boundary of each, their relation to each other, or when to use what.

He is right. The standards define each term well enough on their own. ISO/IEC 17025:2017 defines the decision rule. ILAC-G8 kind of defines the guardbanding rules. JCGM 106 defines the risks (likelihood and consequences). ILAC-P14 defines the rules for reporting the CMC. The GUM defines uncertainty, and the VIM defines most of the definitions. Nobody put them on one page in the order they actually get used. So here is that page.

The Chain, in the Order It Runs

Think of a conformity decision as a short assembly line. Each term is a station. Each station takes something from the one before it and hands something to the one after it. Skip a station, or run them out of order, and the part that comes off the end is a pass/fail statement nobody can defend. Below is the map, and it's not perfect. It’s here to give a generalized sense of it all.

Figure 1. The seven stations in the order they run. Two things enter from outside the lab: the customer's tolerance and the risk the customer agrees to carry.

Station 1 is measurement uncertainty. This is the doubt that remains after you have done everything right: the reference standard's uncertainty and drift, repeatability, resolution, temperature, alignment, and the rest, combined by the GUM method into one number with a coverage factor. It belongs to one measurement result. It is the raw material for everything downstream. No uncertainty, no decision, and per ISO/IEC 17025:2017 Clause 7.6 and the VIM, no traceable result either, and that uncertainty should be backed up by statistical process control, trend analysis, and demonstrating via proficiency testing that your claims are what they are. ISO/IEC 17025:2017 Clause 7.7 turns that from good practice into a requirement: monitor the validity of results, record the data so trends are detectable, and compare your performance with other laboratories through proficiency testing or interlaboratory comparisons. The budget is the claim. An E_n ratio below one is the evidence, and Part 2 covers what that takes.

Example: Your GPS says you will arrive at 6:12, and you read it as 6:12 give or take a few minutes without thinking about it. An uncertainty budget writes down that give-or-take: what it is made of, how the pieces combine, and how sure you are of the total.

Station 2 is bias, and it is the one station that is not about doubt at all. Bias is the known, systematic difference between what your instrument reads and what the reference says. The GUM is direct on this (Clause 3.2.3): identify systematic effects, correct for them, then carry the uncertainty of the correction. Bias compensation means applying that correction to the measured value. It happens before the uncertainty is combined, and it changes the value, not the doubt. Part 3 covers why people get this backward.

Example: A marksman with bias hits the same wrong spot every time. That is the easiest shooter at the range to fix: a few clicks of windage and the group moves onto the bull. The shooter who sprays shots all over the target is the one who will have a very large uncertainty budget.

Station 3 is the CMC, the Calibration and Measurement Capability on a laboratory's scope of accreditation. The CMC uncertainty parameter (one part of several on the scope) is the smallest uncertainty the lab can offer a customer under normal conditions, for the best existing device it is likely to see. It is a floor, not a promise. ILAC-P14 requires that the uncertainty on your certificate include the contributions of your own device, so the number on a certificate is normally larger than the CMC, never smaller. Part 2 draws that line.

Example: A sprinter's personal best is a time set on a perfect track, in perfect weather, with nothing on his back. The CMC is the personal best: ideal conditions, best existing device. The number on your certificate is what he runs carrying your instrument, and it is always slower.

Station 4 is the TUR, the Test Uncertainty Ratio, or in JCGM 106 language, the measurement capability index (C_m is the preferred term as TUR has been botched and has several definitions). It is the tolerance of the item under test divided by the expanded uncertainty of the calibration process. It takes the uncertainty from Station 1 and the tolerance from the customer and tells you whether the measurement is capable enough to make the decision at all. It is a screening ratio. It is not a decision rule, and Part 4 explains why a 4:1 on a purchase order does not settle anything by itself.

Example: Nobody checks a 1 mm gap with a ruler marked in centimeters. TUR is that instinct written as a ratio: how fine the measuring process is compared with the tolerance it is judging. And owning a micrometer still does not tell you where to call the gap good or bad, which is why a healthy ratio is not a decision.

Station 5 is the decision rule itself. ISO/IEC 17025:2017 Clause 3.7 defines it as a rule that describes how measurement uncertainty is accounted for when stating conformity with a specified requirement. It is an agreement between the lab and the customer (Clause 7.1.3), documented with its risk (Clause 7.8.6.1), and reported on the certificate (Clause 7.8.6.2). The standard requires that a rule exist and be stated. It does not require any particular rule. Part 5 is the selection guide.

Example: In tennis, a ball that touches the line is in. Agreed before the first serve, printed in the rulebook, the same call for every player. That is a decision rule. Deciding how to treat the close ones after you have seen the shot is what a lab does when the certificate carries no rule.

Station 6 is the guardband. A guardband is the distance between the tolerance limit and the acceptance limit, w = |TL − AL|. It is the tool most decision rules use to carry out their intent. You don't pick its width on a whim. It comes from the uncertainty and the risk you agreed to accept. Part 6 gives the one formula that produces it.

Example: The speed limit is 65; you do not fully trust your speedometer, so you hold 62. That margin is a guardband: the acceptance limit pulled in from the tolerance limit by your doubt, so a reading that passes leaves you comfortably legal. How far to pull it in is not taste. It falls out of the speedometer's uncertainty and how much ticket risk you will accept.

Station 7 is PFA and PFR, the probability of false accept and the probability of false reject. These are the outputs. A specific-risk rule calculates them for the one result in front of you. A global-risk rule is designed so that they stay below a target across a whole population. Either way, they are what comes off the end of the line. Part 7 explains why a 2 % global PFA and a 50 % specific PFA can describe the same instrument at the same time.

Example: An airport metal detector has one dial and two ways to fail. Set it hot and the line backs up with belt buckles: false reject. Set it cool and the real thing walks through: false accept. PFA and PFR are that dial read off a calibration bench, and turning it to shrink one number grows the other.

Traceability Is a Property of the Result, Not the Instrument

One more term rides the whole chain and gets defined wrong more often than any station on it. The VIM (JCGM 200:2012, 2.41) defines metrological traceability as the property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty. Every word in that definition is doing work. Property of a result, not of an instrument, a laboratory, or a sticker. A documented unbroken chain of calibrations, not a single certificate. Each link contributes to the measurement uncertainty, which is why Station 1 is where traceability lives or dies.

Most certificates get this backward. A typical one devotes a section titled Traceability to an equipment list: a force standard with a NIST number and a calibration due date, a standard indicator with a NIST number of its own, a test indicator with another, and a sentence certifying that the force measurements are traceable to primary standards at NIST. The format treats traceability as something the hardware carries, renewed each time the hardware is recalibrated.

NIST disagrees with the certificates. Its position is that only measurement results are traceable, and that merely having an instrument calibrated, even by NIST, is not enough to make the result obtained with that instrument traceable. The NIST numbers prove even less: NIST GMP 13 states that test report numbers are issued solely for administrative purposes, and holding one provides no assurance or evidence that a measurement is traceable. A calibration result is strictly valid at the time and place it was performed. The moment the standard goes back on the shelf, drift starts working on it. What carries traceability forward is the process wrapped around the instrument: the uncertainty budget at Station 1, the monitoring and control charts Clause 7.7 requires, and a documented chain in which every link states its uncertainty.

So, when a certificate's traceability case is a list of NIST numbers, ask the questions that matter instead. Does it identify the reference standards used and their uncertainties? Does it report measured values with an expanded uncertainty, a coverage factor, and a coverage probability? Can the provider produce the unbroken chain from your result back to the SI? Yes to all three is traceability. A NIST number with a due date is an inventory record. You can use our NIST number, 8675309, for force or torque (NIST does not do torque), and tell 'em Tommy T gave it to you.

If you want proof instead of a number, it lives in the BIPM Key Comparison Database (kcdb.bipm.org). The KCDB lists every calibration and measurement capability an NMI has had internationally reviewed and approved under the CIPM MRA, searchable by metrology area, quantity, range, and country. Run a CMC search for the parameter, force, torque, or anything else, and you can see whether the institute actually supports it, over what range, and at what uncertainty. If the parameter is not there, the institute's name on a certificate is not evidence of traceability for that measurement.

Note: the proper definition of Test Uncertainty Ratio (TUR) is equivalent to JCGM 106’s definition of Measurement Capability Index.

Figure 2. What you specify on paper and what you can verify in practice are not the same thing. Uncertainty turns each sharp limit into a band of ambiguity, and the wider the band, the less of the specification is usable.

When to Use What, in One Paragraph

If the customer does not ask for a pass/fail, report the measured value and its uncertainty and stop. No rule is required. If they do ask, the rule has to exist before the measurement is made. When the TUR is high, and the consequence of a wrong accept is small, simple acceptance with a stated TUR or uncertainty precondition can be defended. When you have little or no history of how that instrument population behaves, you should likely use a specific-risk guardband, because it is conservative and needs only the uncertainty and the result. When a program invokes ANSI/NCSL Z540.3 and its 2 % limit, and you lack reliability data, you can use the managed guardband (Method 6, which, if the TUR is 4.6:1, will limit your risk to 2 %). When you have real end-of-period reliability data, global-risk methods will tell you where the acceptance limits should sit to protect yield. When a false reject is the expensive mistake, guarded rejection exists for exactly that. Part 5 turns this paragraph into a tree.

Walk It Backwards Before You Walk It Forwards

The chain above runs left to right, which is how a lab experiences it. A customer should read it right to left. Our Start With the End in Mind paper puts the whole method in two questions: how good does the measurement need to be, and how good can it be made? The first is a business question about what a wrong decision costs. The second is a technical question about what your equipment, your process, and your provider can deliver. Answer the first, and the chain tells you what to demand at every station upstream.

Here is the arithmetic in its simplest form. If you want to run simple acceptance against a global PFA of 2 % or less, you probably need a TUR of about 4.6:1, which means the calibration process uncertainty has to be at or below the tolerance divided by 4.6. A 0.10 % tolerance needs U ≤ 0.0217 %. A 0.05 % tolerance needs U ≤ 0.0109 %. A 0.01 % tolerance needs U ≤ 0.0022 %. If the lab you are using cannot get there, you have three choices, and the chain shows all of them: add a guardband at Station 6, improve the capability at Station 1 or Station 3, or bring reliability data to Station 7 and justify the risk at the population level. What you cannot do is skip the question and hope.

What to Do This Week

Pull three recent certificates that say PASS from the drawer or digital file system where they were filed, to never be looked at again😊. For each one, find the decision rule, the guardband, and the uncertainty with its coverage factor. If any of the three is missing, the chain was broken somewhere between Station 1 and Station 7, and the pass is an opinion with a decimal point. The rest of this series shows where to look, and if the k’s are not k’s, there’s no MU, or nothing on the certificate says the result is traceable, maybe question your calibration provider a little.

-Henry Zumbrun, Morehouse Instrument Company

Who is Henry? Henry is a non-feathered bi-pod who stands upright and talks. He likes Metallica and the Grateful Dead; he tries to play guitar, and might be a goober. Oh, and he won some awards in the metrology field; he’s written some books, and he’s too old to have received participation trophies, though he should have had a room full of them. 

References

Sources cited across Parts 1 through 7, listed alphabetically. JCGM documents follow the citation form the BIPM recommends, with the member-organization author line and the JCGM DOI. Clause numbers given in the text refer to these editions.

A2LA. R205 — Specific Requirements: Calibration Laboratory Accreditation Program. American Association for Laboratory Accreditation, Frederick, MD.

ANSI/NCSL Z540.3-2006 (R2013). Requirements for the Calibration of Measuring and Test Equipment. NCSL International, Boulder, CO, 2006.

ASME B89.7.4.1-2005. Measurement Uncertainty Conformance Testing: Risk Analysis. The American Society of Mechanical Engineers, New York, 2005.

ASTM E74-18. Standard Practices for Calibration and Verification for Force-Measuring Instruments. ASTM International, West Conshohocken, PA, 2018. https://doi.org/10.1520/E0074-18

BIPM. The BIPM key comparison database (KCDB). Bureau International des Poids et Mesures, Sèvres. https://www.bipm.org/kcdb/

BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML. Evaluation of measurement data — Guide to the expression of uncertainty in measurement. Joint Committee for Guides in Metrology, JCGM 100:2008. https://doi.org/10.59161/JCGM100-2008E

BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML. Evaluation of measurement data — The role of measurement uncertainty in conformity assessment. Joint Committee for Guides in Metrology, JCGM 106:2012. https://doi.org/10.59161/JCGM106-2012

BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML. International vocabulary of metrology — Basic and general concepts and associated terms (VIM), 3rd edition. Joint Committee for Guides in Metrology, JCGM 200:2012. https://doi.org/10.59161/JCGM200-2012

BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML. Guide to the expression of uncertainty in measurement — Part 5: Examples. Joint Committee for Guides in Metrology, JCGM GUM-5:2026. https://doi.org/10.59161/JCGMGUM-5-2026

Carroll, L. Alice's Adventures in Wonderland. Macmillan, London, 1865.

Cenker, G., Zumbrun, H. and Shah, D. Decision Rule Guidance. Morehouse Instrument Company, York, PA. https://mhforce.com/documentation-tools/

Dobbert, M. "A Guard-Band Strategy for Managing False-Accept Risk." Proceedings of the NCSL International Workshop & Symposium, Orlando, FL, 2008.

Eagle, A. R. "A Method for Handling Errors in Testing and Measuring." Industrial Quality Control, Vol. 10, pp. 10–15, March 1954.

Grubbs, F. E. and Coon, H. J. "On Setting Test Limits Relative to Specification Limits." Industrial Quality Control, Vol. 10, pp. 15–20, March 1954.

Harris, G. L. "GMP 13 — Good Measurement Practice for Ensuring Traceability." In: Selected Laboratory and Measurement Practices and Procedures to Support Basic Mass Calibrations, NIST IR 6969, National Institute of Standards and Technology, Gaithersburg, MD, 2019.

Hayes, J. L. Factors Affecting Measuring Reliability. U.S. Naval Ordnance Laboratory, Technical Memorandum No. 63-106, Corona, CA, 1955.

ILAC-G8:09/2019. Guidelines on Decision Rules and Statements of Conformity. International Laboratory Accreditation Cooperation, 2019.

ILAC-P14:09/2020. ILAC Policy for Measurement Uncertainty in Calibration. International Laboratory Accreditation Cooperation, 2020.

ISO/IEC 17025:2017. General requirements for the competence of testing and calibration laboratories. International Organization for Standardization, Geneva, 2017.

ISO/IEC 17043:2023. Conformity assessment — General requirements for the competence of proficiency testing providers. International Organization for Standardization, Geneva, 2023.

Mimbs, S. M. "Using Reliability to Meet Z540.3’s 2 % Rule." NCSL International Workshop & Symposium, National Harbor, MD, 2011. NASA KSC-2011-151.

Morehouse Instrument Company. Decision Rules Interactive Calculator. https://decisionrules.mhforce.com

Morehouse Instrument Company. Guidance on Uncertainty Budgets for Force Measuring Devices. York, PA. https://mhforce.com/documentation-tools/

Morehouse Instrument Company. GUM in Plain English (series). York, PA. https://mhforce.com/documentation-tools/

Morehouse Instrument Company. Morehouse CMC Workbook. York, PA. https://mhforce.com/documentation-tools/

Morehouse Instrument Company. Morehouse True Cost Calculator and The True Cost of Measurement Decisions. York, PA. https://mhforce.com/documentation-tools/

Morehouse Instrument Company. Start With the End in Mind. York, PA, 2026. https://mhforce.com/wp-content/uploads/2026/05/Start_With_The_End_in_Mind.pdf

Morehouse Instrument Company. Probability of False Reject (PFR) (webinar). York, PA.

NASA-HDBK-8739.19-4. Estimation and Evaluation of Measurement Decision Risk. NASA Measurement Quality Assurance Handbook, Annex 4. National Aeronautics and Space Administration, Washington, DC, 2010.

NCSL International. Handbook for the Application of ANSI/NCSL Z540.3-2006 — Requirements for the Calibration of Measuring and Test Equipment. NCSL International, Boulder, CO, 2009.

Peart, N. "Freewill." Performed by Rush on Permanent Waves. Anthem Records, 1980.

Reese, P. and Harben, J. "Risk Mitigation Strategies for Compliance Testing." NCSLI Measure, Vol. 7, No. 1, 2012.

UKAS. LAB 48: Decision Rules and Statements of Conformity, Edition 5. United Kingdom Accreditation Service, Staines-upon-Thames, 2024.