Advice on How Many Decimal Places Are Enough for Equipment Resolution

May 23, 2024

Decimal Places & Equipment Resolution

How many decimal places are enough for equipment resolution?  That’s the subject of the latest article by Morehouse Instrument Company president Henry Zumbrun, which appears in the May issue of Quality Digest.

The article, How Many Decimal Places Are Enough, isn’t about formulas or rounding. Rather, it is literally about determining how many decimal places are enough.

"We’ve been asked by customers how many decimal places are enough. It’s either because they want to maximize the resolution of their purchase or they expect far more resolution from the equipment than is reasonable.  We’ve seen a lot, even a meter being set to read 100,000.001. That’s 100 million counts, though this instrument (i.e., readout) wasn’t even stable to 1 count. The extra 0.001 meant nothing," writes Zumbrun.

Read the article here: How Many Decimal Places Are Enough? | Quality Digest

Quality Digest is an online and printed publication that reaches the entire quality spectrum, including metrology, Six Sigma, lean, inspection, testing, SPC, software, and international standards such as ISO 9001, ISO 14001, AS9100, and ISO/TS 16949.

About Morehouse Instrument Company

Companies worldwide rely on Morehouse for accuracy and speed. The company turns around equipment in 7-10 business days so customers can return to work quickly and save money.

Morehouse Instrument Company, a trusted and accredited provider of force and torque measurement services for over 100 years, offers measurement uncertainties 10-50 times lower than the competition.

Morehouse helps commercial labs, government labs, and other organizations lower their measurement risk by lowering equipment uncertainties for torque and force measurement. Contact Morehouse at info@mhforce.com.

More Information about Morehouse

We believe in changing how people think about force and torque calibration in everything we do.

This includes setting expectations on load cell reliability and challenging the "just calibrate it" mentality by educating our customers on what matters and what causes significant errors.

We focus on reducing these errors and making our products simple and user-friendly.

This means your instruments will pass calibration more often and produce more precise measurements, giving you the confidence to focus on your business.

Companies around the globe rely on Morehouse for accuracy and speed.

Our measurement uncertainties are 10-50 times lower than the competition.

We turn around your equipment in 7-10 business days so you can return to work quickly, saving you money.

When you choose Morehouse, you're not just paying for a calibration service or a load cell.

You're investing in peace of mind, knowing your equipment is calibrated accurately and on time.

Contact Morehouse at info@mhforce.com to learn more about our calibration services and load cell products.

Email us if you ever want to chat or have questions about a blog.

We love talking about this stuff.

Our YouTube channel has videos on various force and torque calibration topics here.

 

 

Please share if you found this helpful.

Newsletter Subscription

  • We're committed to your privacy. Morehouse Instrument Company uses the information you provide to us to contact you about our relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our Privacy Policy.

Find Related Articles

When You're Looking for More Accurate Measurements

Morehouse would like the opportunity to earn your business. Contact us today.
Contact Us
  • Type

Top cross linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram