Repeatable and reproducible measurements are crucial especially in the medical device industry where process traceability is a requirement by the FDA. With such a large selection of instruments on the market finding the right one can be daunting. At minimum, a system’s accuracy and resolution must be in-line with tolerance requirements. Precision is more dependent on technique which may involve various levels of automation which is discussed in detail.
TABLE OF CONTENTS
- Accuracy vs. Precision
- The Pros and Cons of Automation
- Data Import/Export and Other “Nice to Have” Features
- Metrology Recommendations
As any operations manager knows the cost and the time of adding a measurement step can be high. Not only is there the initial costs of the system, but there are also cost associated with training, maintenance, documentation, method development, etc. With so many metrology tools on the market with an assortment of features and various levels of automation, finding the right one can be daunting. Accuracy and precision are also important but the misconception is that the more expensive a system; the more accurate and precise it is. This is not always the case. Accuracy and resolution of a system must be in-line with tolerance requirements, while repeatability and reproducibility has more to do with technique and can be improved on even the most basic systems. This application note addresses accuracy and precision. It also addresses some of the options that are available on metrology tools today that reduce the cost of ownership, assist with analysis and reduce overall error.
Accuracy is how close you come to hitting your target. Precision is how repeatably and reproducibly you hit the same spot whether it is the target or not. If you hit your target 9 out of 10 times your accuracy may be good, but can you imagine if you measured an entire lot of medical devices after a process step and 10 percent of your measurements were off? How would you know if there was a problem with your process or a problem with your measurement method? You wouldn’t because you can’t have any confidence in your process until you have confidence in your measurement method. This is why precision is so important. You must confirm that you have a repeatable and reproducible measurement method. The FDA requires you to have this in the form of a gauge repeatability and reproducibility (GR&R) test.
There are various factors that determine the accuracy of a system. If you are making small XY measurements that are within the field of view then the main limitations are camera and optical resolution. If you are making larger measurements that are outside the field of view then camera and optical resolution becomes less relevant. As the measurement distance increases and stage resolution and stage levelness become more relevant. Stage flatness and vibration gets worse over distances so tolerances for larger measurements should be reasonable and in-line with the system limitations. For Z measurements the depth of field of the lens system and the resolution of the focus column set the limitations for micro measurements. For larger step heights, accuracy decreases as Z travel increases. It’s important to know the accuracy and resolution limitations of the system you purchase and how they align with your tolerance requirements. More expensive systems do not necessarily have greater accuracy and resolution. It’s also important to keep your system calibrated and in high working performance to insure the best accuracy.
Precision of a measurement method can be determined by running a gauge repeatability and reproducibility test (GR&R). There are some basic techniques that will improve GR&R results and they are as follows:
You must have consistent lighting to have consistent measurements. In the image below one can see how the measurement differs from a brighter back light to a darker back light (see image). With halogen systems lighting degrades over time which changes the results over time and inhibits the software from reliably detecting an edge. Many metrology tools on the market today use LEDs for this reason. Not only does this improve reliability but it also eliminates maintenance costs from continually having to replace bulbs.
Use back light to make measurements whenever possible. Back light gives the best contrast and improves the software’s capability to automatically detect the edge. When top light lighting is the only option manual edge detection may be required depending on how well the edge detection software reliably detects the desired edge. If the edge detection algorithms are not powerful enough or if the part to part variation is too great revert back to manual edge detection for better R&R.
Automatic Edge Detection
Automatic edge detection is highly recommended when at all possible. Even the best operators get fatigued and can’t consistently detect an edge as well as most software algorithms can. Operator to operator variation is also extremely unpredictable and falls short of even the best training and test procedures. Back light gives the best contrast and is full proof for most all auto edge detect. Top light can be more difficult for auto edge detection and some detection algorithms are not powerful enough when part to part variation is too great. If this is the case go back to having the operator select the edge. Most metrology software has algorithms that allow the operator to “help” the software find the edge. A movable box and/or arrows are common so that the operator can size it over the edge or point to the desired edge (see image). Some software algorithms “help” the operator by indicating when they’ve come across an area of contrast. In the image below, the cross hair changes color when the contrast changes to indicate an edge.
Magnification and Same Edge to Same Edge Measurement
Always use the highest magnification when taking a measurement requires an operator to define an edge. The resolution is better and the operator can see the desired edge and more clearly.
Always take the same edge when manually or automatically selecting one edge and measuring to the next (See image below). When using a tool such as a cross-hair always measure with the edge of the cross hair and use the same cross-hair edge when measuring from one point to the next. This is because cross-hairs have a width and the width could be great enough to skew the measurement (see image).
Manufacturing and quality engineers know that the more humans are involved in the measurement process the greater the chance for errors to occur so the logical solution is to remove humans from the process. By removing the human the operator cost decreases as well as the training costs and in most cases through-put increases. If you consider the advantages, although the initial cost of a fully automatable system is a lot higher, the long term return on investment of a fully automated routine could be very good. This could be the case; however, there are a lot of hidden costs that need to be addressed. First of all, although most systems have macros to create routines the complexity of the programming and the time and the skill level of the programmer especially for very complex parts with multiple measurement requirements must be carefully considered. Special fixturing may need to be designed and built to insure that the part is placed in the exact same location every single time. Any slight change in the way the part is oriented could completely throw off the software’s ability to detect the correct edge. If the fully automated measurement routine is not robust enough to withstand any slight variation in the process it could constantly hang and require operator intervention, so in the end the return on investment could be diminished. There are a variety of instruments on the market with various levels of automation capability. Consider your budget, the time and the resources that you have available and the sheer volume and complexity of the parts when determining how much automation is really necessary.
The ability to import/export and analyze data quickly is always a welcome feature for any technician, production manager or engineer. Almost all systems have the ability to export to a .doc and .exe or a .txt file. Some systems allow you to import and export CAD files. Importing CAD overlays is an excellent alternative to a large, bulky comparator for go-no-go applications with the absence of Mylar ™. Exporting CAD files is a valuable tool for the reverse engineering of parts. Some software has report writing capability that includes SPC (statistical process control charts) charts to monitor your process through measurement. All these features are highly recommended “nice to have” features that add a lot of value to a metrology tool.
Field of View Measurement Leica Application Suite Measurement Module Image Pro and Image Pro Plus Non-Contact Video Measurement Systems The XT 1000/2000 VME/VMS/VMP Series
The XY accuracy of a system depends on the camera and optical system for micro-measurement, but stage resolution and planarity are more crucial for larger measurements. Make sure the accuracy is in-line with the tolerance requirements before purchasing and know that cost does not guarantee accuracy. A system must be calibrated and maintained. Precision, or the repeatability and reproducibility of a measurement method, is dependent on technique. There are standard techniques that should be followed to insure the most consistent measurements and they include back lighting and automatic edge detection whenever possible.
There are various metrology instruments with a wide range of automation capability. Fully automatable machines that allow you to create routines that remove any human influence can be an excellent return on investment as far as operator costs, training, and throughput; but there are a lot of hidden cost to consider such as the time and the skill level of the routine creator and the fixture designer. Consider budget, development costs and part complexity and volume before investing in a costly fully automatable system.
Data import/export and analysis tools if not standard can be worth the extra cost of a system because they can be used to monitor the process on the fly. CAD import and export is an excellent feature for creating go-no-go overlays and reverse engineering.