Noisy data

Differences in real-world measured data from the true values come about from multiple factors affecting the measurement.

Random noise is often a large component of the noise in data. Random noise in a signal is quantified as the signal-to-noise ratio. Random noise contains a wide range of frequencies, and is also called white noise (as wide range of colors of light combine to make white).

Random noise affects the data collection and data preparation processes, where errors commonly occur. Noise has two main sources: errors introduced by measurement tools and random errors introduced by processing or by experts when the data is gathered.

Improper filtering can add noise if the filtered signal is treated as if it were a directly measured signal. As an example, Convolution-type digital filters such a moving average can have side effects such as lags or truncation of peaks. Differentiating digital filters amplifies random noise in the original data.

Outlier data are data that appear to not belong in the data set. It can be caused by human error such as transposing numerals, mislabeling, programming bugs, etc. If actual outliers are not removed from the data set, they corrupt the results to a small or large degree, depending on circumstances. If valid data is identified as an outlier and is mistakenly removed, that also corrupts results.

Individuals may deliberately skew data to influence the results toward a desired conclusion. Data that looks good with few outliers reflects well on the individual collecting it, and so there may be incentive to remove more data as outliers or make the data look smoother than it is.