Reference articles on history, science, culture and more
Surprise meSurprise meSubscribe
Encyclopedia

Numerical error

Wikipedia contributorsAdapted for Fathomly · View authors
1 min read
Source updated March 11, 2026
Image credit is listed at the end of this article.

Numerical error is the error that arises in numerical computations due to limitations in representing real numbers and performing arithmetic operation on computers. These errors commonly appear in software engineering and mathematics.

01Types

Numerical error in computations generally arises from several sources. The most common sources are round-off error and truncation error.

It can be the combined effect of two kinds of error in a calculation. The first is referred to as round-off error and is caused by the finite precision of computations involving floating-point numbers. The second, usually called truncation error, is the difference between the exact mathematical solution and the approximate solution obtained when simplifications are made to the mathematical equations to make them more amenable to calculation.

02Measure

Floating-point numerical error is often measured in ULP (unit in the last place), which represents the distance between two adjacent floating-point numbers.

Watch videos about Numerical errorExplainers and documentaries on YouTube (opens in a new tab)

Sources and credits

This article is adapted from the Wikipedia article “Numerical error”, written by its contributors and licensed under CC BY-SA 4.0. Fathomly has changed the layout, removed citation markers, navigation and maintenance notices, and adjusted punctuation. This adapted version is shared under the same license. For references, see the original article.

Images, from Wikimedia Commons:

Fathomly is not affiliated with or endorsed by the Wikimedia Foundation. Spotted a problem? Tell us.

Continue exploring

Related topics

Catastrophic cancellation

In numerical analysis, catastrophic cancellation is the phenomenon that subtracting good approximations to two nearby numbers may yield a very bad approximation to the difference of the original numbers. For example, if there are two studs, one L 1 = 253.51 cm {\displaystyle L_{1}=253.51\,{\text{cm}}} long and the other L 2 = 252.49 cm {\displaystyle L_{2}=252.49\,{\text{cm}}} long, and they are measured with a ruler that is good only to the centimeter, then the approximations could come out to be L ~ 1 = 254 cm {\displaystyle {\tilde {L}}_{1}=254\,{\text{cm}}} and L ~ 2 = 252 cm {\displaystyle {\tilde {L}}_{2}=252\,{\text{cm}}} .

Numerical analysis

Numerical analysis is the study of algorithms for the problems of continuous mathematics. These algorithms involve real or complex variables, and typically use numerical approximation in addition to symbolic manipulation.

Error analysis (mathematics)

In mathematics, error analysis is the study of kind and quantity of error, or uncertainty, that may be present in the solution to a problem. This issue is particularly prominent in applied areas such as numerical analysis and statistics.