Measurement Risk in Business Decisions: The Likelihood and the Consequences of Measurement Risk

Measurement Risk in Business Decisions: The Likelihood and the Consequences of Measurement Risk

I’ve worked on this topic for many years and continue to teach, refine, and learn. This article reflects that ongoing process and is intended to clarify what measurement risk is, how it shows up in practice, and why it matters.

Measurement risk is often misunderstood because we rarely discuss it as part of the full measurement system. In conformity assessment, calibration, and testing, the conversation can narrow to the numbers—uncertainty values, probabilities, and whether a particular statistical method was used. Those details matter, but when they become the whole discussion, the real drivers of risk get pushed aside.

In practice, measurement risk rests on three equally important foundations: the requirements being applied, the equipment used to make the measurement, and the process used to perform it. Scott Mimbs described these as a three-legged stool. The system is only stable when all three legs are sound. If any one leg is weak—unclear requirements, incapable equipment, or a fragile process—the decision becomes unstable, no matter how sophisticated the math looks.

ISO/IEC 17025 is consistent with that reality. The standard does not require laboratories to calculate probabilities, apply guard bands, or adopt a particular risk model. Instead, it expects laboratories to understand and manage risk the way risk is commonly understood: the likelihood of a wrong decision and the consequences if that wrong decision occurs. The standard leaves the “how” flexible on purpose, because no single prescribed method can fit every application.

At its core, measurement risk is not a statistical obligation; it is a decision-making responsibility. Organizations must decide what level of uncertainty is acceptable, what happens when a

decision is wrong, and who ultimately bears the risk. Those choices are shaped not only by technical capability, but by safety, cost, reputation, and business objectives.

Why Measurements Exist—and Where Risk Enters

Measurements are not made for their own sake. They exist to support decisions, such as:

• Accepting or rejecting a product
• Releasing or stopping a process
• Declaring something safe—or unsafe

Every measurement-based decision follows the same fundamental path:

Measurement → Decision → Consequence

Most of the time, the measurement itself is not what creates harm. The harm shows up after the decision, when consequences materialize—sometimes immediately, sometimes much later, and sometimes undetected for years.

When a measurement cannot reliably support the decision it is meant to inform, the risk doesn’t disappear. It moves downstream. Downstream risk is usually larger, more expensive, and harder to correct than addressing the issue at the point where the measurement is made.

Why Measurements Are Never Exact

Every measurement carries uncertainty. Even with capable equipment and skilled people, there is always some doubt about how close a result is to the true value. That isn’t a defect in measurement; it’s a fundamental reality of measurement.

The problem usually isn’t uncertainty itself. The problem is making decisions without understanding what the uncertainty implies.

When results fall well inside specification limits, uncertainty may have little practical impact. But when results fall near a limit, uncertainty dominates the decision. This is precisely why decision rules exist—not to eliminate uncertainty, but to define how much uncertainty is acceptable given the consequences of being wrong.

Understanding Risk: Likelihood and Consequences

Across engineering, safety, finance, and project management, risk is consistently defined as a combination of two factors:

• Likelihood – How likely is a wrong decision?
• Consequence – What happens if that wrong decision occurs?

ISO/IEC 17025 follows this same, well-established definition. When the standard requires an organization to consider the level of risk, it is referring to both elements.

One point that often helps: risk is not the same thing as probability. Probability is one way to express likelihood, but it is not always necessary or even useful, and ISO/IEC 17025 does not mandate that likelihood be quantified as a numerical probability.

That flexibility matters because likelihood can be managed in practical ways without ever calculating a probability—reducing uncertainty, improving equipment capability, applying conservative acceptance criteria, or using an agreed decision rule that fits the application. These approaches still manage risk, even when no probability is computed.

The Two Decision Errors That Drive Risk

Every measurement-based conformity decision can be wrong in two ways. When quantified, these are often expressed as the probability of false acceptance (PFA) and the probability of false rejection (PFR). The tradeoff exists even when you never calculate probabilities.

False acceptance risk

Accepting something that is actually nonconforming. This is typically the higher-consequence error and can result in:

• Safety incidents
• Product failures
• Recalls
• Legal liability
• Loss of trust

False rejection risk

Rejecting something that is actually conforming. False rejects usually do not endanger safety, but they can cause:

• Scrap and rework
• Delays
• Higher operational costs
• Customer frustration

Every decision rule represents a tradeoff between these two risks. Reducing one almost always increases the other. There is no “zero-risk” option—only informed choices.

Note: false rejection risk only applies when you issue a nonconforming (“fail”) statement. If you are not making that kind of statement, false rejection is not the risk you are managing at that moment.

What ISO/IEC 17025 Actually Requires

ISO/IEC 17025 requires laboratories to define and document a decision rule when making statements of conformity. That decision rule must:

• Take measurement uncertainty into account
• Consider the level of risk associated with the decision
• Be communicated to and agreed upon with the customer (when not inherent in the specification)

That is the requirement.

Just as important is what the standard does not require. ISO/IEC 17025 does not mandate guard bands, probability calculations, PFA/PFR calculations, or any specific mathematical method for risk evaluation. In fact, the word “probability” does not appear anywhere in ISO/IEC 17025, underscoring the standard’s intent to allow multiple valid approaches to managing risk.

Probability-Based Methods Are Optional Tools

Some organizations choose to calculate probability-based metrics such as PFA and PFR. These methods can be valuable, particularly when consequences are severe, regulatory frameworks demand quantified risk, or statistically meaningful data exists.

Value does not equal requirement. Some standards or customer specifications may set explicit probability targets (for example, limiting false acceptance to a defined threshold), but that is not a default expectation under ISO/IEC 17025.

Many organizations manage measurement decision risk effectively without probability calculations by controlling measurement uncertainty, applying minimum capability ratios such as TUR, and aligning acceptance criteria with real-world consequences.

Decision Rule Guidance and Risk Ownership

One point emphasized in the Decision Rule Guidance (Zumbrun, Cenker, Shah) is that measurement risk isn’t something the lab can “solve” on its own. The lab can quantify uncertainty and apply a decision rule correctly, but it can’t determine how much risk is acceptable in the real-world use case.

Only the organization relying on the measurement can make that call.

ISO/IEC 17025 recognizes this explicitly. When a statement of conformity is required, the laboratory must apply a documented decision rule and take uncertainty and risk into account. But unless the decision rule is inherent in a referenced specification, the customer must define—or explicitly agree to—the decision rule being used. This is not a paperwork exercise; it is a risk ownership decision.

Organizations that fail to understand their own requirements often default to whatever decision rule is offered, without realizing they may be accepting far more risk than intended.

The Hidden Risk of Simple Acceptance

The Decision Rule Guidance document calls out a common and potentially costly misconception: that simple acceptance (declaring a pass whenever the measured value falls within the tolerance) is a neutral, low-risk choice. It might not be.

With simple acceptance, the decision boundary is the specification limit itself. When measurement uncertainty is large relative to tolerance, results near that limit can carry a specific risk of false acceptance that becomes surprisingly high. In common scenarios, the guidance indicates that this specific risk can approach 50%, meaning a result that appears to “pass” may be no better than a coin flip in determining whether the item is truly conforming.

In practical terms, this means:

• A result can look like a clear “pass” while still having a substantial chance of being out of tolerance.
• Customers may unknowingly accept a much higher level of risk than intended because the decision rule was never made explicit.
• Downstream consequences—failures, rework, recalls, disputes, or safety concerns—become more likely.

The takeaway is straightforward: accepting risk by default is still accepting risk. When a decision rule or risk requirement isn’t defined, risk isn’t eliminated; it is implicitly shifted to whoever ultimately bears the consequences.

Note: the above figure shows only one probability density function, indicating that the end-user is concerned with an instantaneous liability and does not consider the population. It is important to note that when using global risk models, the 50% risk could be much lower. In some cases, it might even fall below a PFA of 2 %. 

With that in mind, the Decision Rule Guidance document expands the discussion beyond specific (single result) false acceptance to a broader view of how risk is created, interpreted, and allocated across an operation, whether you’re making one decision at the bench or managing a stream of decisions across an entire product population.

Risk Exists at Multiple Levels

Another critical insight from the Decision Rule Guidance document is that measurement risk is not confined to a single reported result. Risk shows up at multiple levels, and organizations need to understand each one to manage it intentionally rather than accidentally.

Specific Risk (also called bench-level risk) is based on a specific measurement result.

• It triggers a response based on measurement data gathered at the time of the test.
• It may be characterized by one or two probability distributions, depending on the method.
• Any representation with only one probability distribution is always a specific risk method.

Global Risk (also called process-level risk) is based on a future measurement result.

• It is used to ensure the acceptability of a documented measurement process.
• It is based on expected or historical information and is usually characterized by two probability distributions.

Focusing only on a single pass/fail decision while ignoring global and lifecycle effects is how organizations end up with measurements that appear technically compliant yet still produce expensive—or even catastrophic—outcomes in the real world.

When Measurements Go Wrong: Real-World Consequences of Measurement Risk

History provides painful reminders of what happens when measurement risk is misunderstood or ignored:

• Hyatt Regency Walkway Collapse — A design change doubled the load on a critical connection without proper verification. The structure met only about 60 % of the required capacity. 114 people died.
• Aircraft Sensor Miscalibration — Sensors were calibrated without accounting for moisture contamination. The calibration was technically performed, but under the wrong assumptions, resulting in a multi-billion-dollar accident.
• Medical Radiation Overdosing — A miscalibrated radiation system overdosed 152 cancer patients, some by as much as 70 %, due to inadequate processes and a lack of verification.
• Hubble Space Telescope — Test data revealed a mirror error far outside specification. The data was dismissed. Fixing the problem required a $1B+ servicing mission.

In every case, the measurements themselves were only part of the story. The real failures were decisions made without fully understanding risk and consequences. There are several more examples, which we cover in more detail in a separate article.

Requirements Drive Risk—Not the Other Way Around

A recurring theme in Decision Rule Guidance is that requirements must be understood before equipment is purchased, processes are designed, or decision rules are selected. Too often, organizations do the opposite: buy equipment first, pick a vendor second, and ask about risk last. That sequence almost guarantees suboptimal outcomes.

Understanding requirements means more than knowing the tolerance. It includes where the tolerance came from, what nonconformance means in the real world, how you want to balance false acceptance versus false rejection, and what costs actually matter (rework versus failure), including safety, liability, and reputational impact. Only then can you decide what uncertainty is acceptable and what decision rule makes sense for the application.

Decision Rules Are Tools—Not Answers

The guidance document intentionally presents multiple decision rule options. This is not because one is universally “better” than another; it is because different risks demand different tools.

Some applications benefit from:

• Simple acceptance with strict capability requirements (for example, 4.6:1 TUR/C_m)
• Guard banding to reduce false acceptance risk
• Explicit probability limits when consequences are severe
• Reporting results with uncertainty only, leaving the decision to the customer

What matters is not the sophistication of the method, but whether the method aligns with the organization’s actual risk tolerance.

Using an advanced probability model without understanding the consequences is just as dangerous as ignoring uncertainty altogether.

Why Organizations Must Be Honest About Risk

One of the strongest and most uncomfortable messages in the guidance is that many organizations do not truly want to know how much risk they are accepting.

Risk analysis can reveal equipment that is not capable, unstable processes, unrealistic requirements, or decisions driven by cost rather than safety or quality.

Ignoring these realities does not make them go away. It simply delays the appearance of the consequences and increases their severity.

As the guidance bluntly notes, choosing not to decide is still a decision.

Measurement Risk as a Strategic Asset

When organizations understand measurement risk properly, it stops being a compliance burden and becomes a strategic tool.

Effective decision rule selection can reduce unnecessary rework and scrap, prevent costly failures and recalls, improve long-term reliability, align quality decisions with business objectives, and save millions—or billions—over the product lifecycle.

This is why ISO/IEC 17025 intentionally avoids prescribing a single “correct” approach. The standard assumes that competent organizations will evaluate their risks and make informed decisions.

Final Perspective

Measurement risk does not belong to the laboratory. It belongs to the organization making the decision.

Laboratories provide the data, uncertainty, and technical framework. Standards provide the structure. But only the organization can determine what level of risk is acceptable for its products, processes, and customers.

Different measurements involve different risks. A cosmetic dimension does not carry the same consequences as a safety-critical measurement. Rejecting a conforming item may be trivial in one application and extremely costly in another. Accepting a nonconforming item may be manageable in one case and catastrophic in another.

Every decision rule answers the same fundamental questions: How much uncertainty are we willing to accept? What happens if we are wrong? Who bears the risk—producer, consumer, or both? These questions cannot be answered by mathematics alone. They require judgment, context, and an understanding of safety, cost, liability, reputation, and customer trust.

Understanding that responsibility—and acting on it deliberately—is the difference between merely complying with ISO/IEC 17025 and actually using measurement to protect people, products, and long-term value.

Measurement risk is not about doing more math. It is about making better decisions.

Risk is about likelihood and consequences. Probability is optional, not mandatory. Guard bands are a tool, not a requirement. Decision rules are chosen—not imposed.

When organizations understand this, they move beyond box-checking and begin using conformity assessment as it was intended: to protect people, products, and long-term value.

For additional information on measurement risk and decision rules, please download our guidance document at https://mhforce.com/wp-content/uploads/2024/04/Decision-Rule-Guidance-1st-Edition-V1.1.pdf.

Additional Information

Appendix A — Decision Rules and Risk Management Compliance (ISO/IEC 17025:2017)

The laboratory’s approach to decision rules complies fully with ISO/IEC 17025:2017.

ISO/IEC 17025 defines a decision rule as the method by which measurement uncertainty is accounted for when making statements of conformity (Clause 3.7). The standard does not prescribe a specific decision rule methodology, nor does it mandate the use of guard bands or probability-based calculations.

In accordance with Clause 7.1.3, when a statement of conformity is requested, the applicable decision rule is defined and, unless inherent in the referenced specification, is communicated to and agreed upon with the customer.

Measurement uncertainty is evaluated in compliance with Clause 7.6, ensuring that all significant contributors are identified and appropriately considered.

In compliance with Clause 7.8.6.1, the laboratory applies the documented decision rule and considers the associated risks, including the risks of false acceptance and false rejection.

The applied decision rule is clearly identified in the report, as required by Clause 7.8.6.2.

ISO/IEC 17025 permits multiple decision rule methodologies. The absence of guard bands or probability calculations does not constitute a nonconformance provided that measurement uncertainty, risk consideration, and customer agreement requirements are met.

Appendix B — Decision Rules & Measurement Risk Compliance Checklist (ISO/IEC 17025:2017)

This checklist demonstrates the laboratory’s conformance with ISO/IEC 17025 requirements related to decision rules, measurement uncertainty, and risk management when issuing statements of conformity.

1. Decision Rule Definition (Clause 3.7)

☐ A decision rule is defined as the method by which measurement uncertainty is accounted for when making statements of conformity.

☐ The laboratory has defined applicable decision rule(s) for conformity assessment activities.

☐ The decision rule(s) selected are appropriate for the measurement application and associated risks.

☐ The laboratory recognizes that ISO/IEC 17025 does not prescribe a specific decision rule methodology.

2. Statement of Conformity Determination (Clause 7.1.3)

☐ The laboratory determines whether a statement of conformity is required for each applicable request.

☐ When a statement of conformity is requested, the applicable decision rule is identified.

☐ If the decision rule is inherent in a referenced specification or regulation, it is applied as written.

☐ If the decision rule is not inherent, the decision rule is communicated to and agreed upon with the customer prior to issuing results.

☐ Customer agreement is documented in accordance with laboratory procedures.

3. Measurement Uncertainty Evaluation (Clause 7.6)

☐ Measurement uncertainty is evaluated for applicable measurement activities.

☐ All significant contributors to measurement uncertainty are identified and considered.

☐ Appropriate methods are used to estimate measurement uncertainty.

☐ Measurement uncertainty is reviewed to ensure it is suitable for the intended conformity decision.

4. Risk Consideration in Decision Rules (Clause 7.8.6.1)

☐ The laboratory applies the documented decision rule consistently when issuing statements of conformity.

☐ The level of risk associated with the conformity decision is considered.

☐ Risks of false acceptance and false rejection are recognized and addressed.

☐ The decision rule selected aligns with the consequences associated with incorrect decisions.

☐ The laboratory acknowledges that risk may be managed through multiple valid approaches, including but not limited to uncertainty control, capability requirements, or decision rule selection.

5. Reporting of Decision Rules (Clause 7.8.6.2)

☐ The applied decision rule is clearly identified in reports containing statements of conformity.

☐ Reporting clearly indicates how conformity was determined.

☐ Reports are sufficiently clear to allow the customer to understand the basis of the conformity decision.

6. Guard Bands and Probability Calculations

☐ The laboratory understands that ISO/IEC 17025 does not require the use of guard bands.

☐ The laboratory understands that ISO/IEC 17025 does not require probability-based calculations (e.g., PFA or PFR).

☐ Guard bands or probability calculations are used only when appropriate to the application or when required by specification, regulation, or customer agreement.

☐ The absence of guard bands or probability calculations does not constitute a nonconformance when measurement uncertainty, risk consideration, and customer agreement requirements are met.

7. Risk Ownership and Customer Responsibility

☐ The laboratory recognizes that the customer is responsible for determining acceptable risk for their application.

☐ The laboratory supports the customer by providing measurement results, uncertainty information, and decision rule options.

☐ The laboratory does not assume or impose risk acceptance beyond the agreed decision rule.

8. Application-Specific Decision Rules

☐ Decision rules are selected based on the specific measurement application.

☐ The laboratory recognizes that different measurements may require different decision rules.

☐ Decision rule selection considers safety, regulatory, cost, and business impacts where applicable.

9. Continual Review and Improvement

☐ Decision rules and uncertainty evaluations are periodically reviewed for continued suitability.

☐ Changes in requirements, equipment capability, or process are evaluated for their impact on risk.

☐ Lessons learned from nonconformities, customer feedback, or measurement failures are considered.

10. Overall Conformance Statement

☐ The laboratory’s decision rule practices conform to ISO/IEC 17025:2017.

☐ The laboratory’s approach supports informed risk management and transparent conformity assessment.

☐ The laboratory’s practices align with the intent of ISO/IEC 17025 to allow flexibility while ensuring technically valid and defensible results.

About Morehouse   

We believe in changing how people think about Force and Torque calibration in everything we do, including discussions on how to calibrate cable tensiometers.

This includes setting expectations and challenging the "just calibrate it" mentality by educating our customers on what matters and what may cause significant errors. 

We focus on reducing these errors and making our products simple and user-friendly. 

This means your instruments will pass calibration more often and produce more precise measurements, giving you the confidence to focus on your business. 

Companies around the globe rely on Morehouse for accuracy and speed. 

Our measurement uncertainties are 10-50 times lower than the competition, providing you with more accuracy and precision in force measurement. 

We turn around your equipment in 7-10 business days so you can return to work quickly and save money. 

When you choose Morehouse, you're not just paying for a calibration service or a load cell. 

You're investing in peace of mind, knowing your equipment is calibrated accurately and on time. 

Through Great People, Great Leaders, and Great Equipment, we empower organizations to make Better Measurements that enhance quality, reduce risk, and drive innovation. 

With over a century of experience, we're committed to raising industry standards, fostering collaboration, helping with understanding risk, and delivering exceptional calibration solutions that build a safer, more accurate future. 

Contact Morehouse at info@mhforce.com to learn more about our calibration services and load cell products. 

Email us if you ever want to chat or have questions about a blog. 

We love talking about this stuff. We have many more topics other than expressing SI units! 

Our YouTube channel has videos on various force and torque calibration topics here. 

# Consequences of Measurement Risk