Measurement Equipment with Outlier Filter

Measurement, assessment and control often involve acquiring large amounts of data to analyze and characterize signals effectively. In these operations, spurious points or doubtful observations, also known as outliers, can appear and inappropriately bias the analyses. Outliers in streams of data obscure useful interpretations, and characterization of many different types of measurements or signals (e.g., images, sounds, positions, accelerations, temperatures, topographies). However, current methods to remove outliers still can leave some doubtful points in data streams, and might even remove legitimate data in some instances.

Researchers at WPI have recently developed a novel method to filter outliers in large amounts of data that can have significant irregularity, but where some continuity or short-range auto-correlation is expected, like in the weather or the stock market. Using multiscale curvature analysis methods, the algorithm scans data streams to continuity as a function of scale and position. This approach values curvature, change in curvature, or any other higher derivatives, to identify continuity outliers in profiles and multi-dimensional measurements. This scale-based analysis method works on data streams that should be continuous, at least through some segments and scale ranges, but not necessarily differentiable, which includes fractal-like data . Multiscale curvature analysis software may be integrated with any kind of measurements that can be digitized, like digital microscopes or telescopes, or other data acquisition systems where outliers can appear.