Reference articles on history, science, culture and more
Surprise meSurprise meSubscribe
Encyclopedia

Softplus

Smoothed ramp function

Wikipedia contributorsAdapted for Fathomly · View authors
1 min read
Source updated February 20, 2026
Image credit is listed at the end of this article.

In mathematics and machine learning, the softplus function is

f(x)=\ln(1+e^{x}).

It is a smooth approximation (in fact, an analytic function) to the ramp function, which is known as the rectifier or ReLU (rectified linear unit) in machine learning. For large negative x it is \ln(1+e^{x})=\ln(1+\epsilon )\gtrapprox \ln 1=0, so just above 0, while for large positive x it is \ln(1+e^{x})\gtrapprox \ln(e^{x})=x, so just above x.

The names softplus and SmoothReLU are used in machine learning. The name "softplus" (2000), by analogy with the earlier softmax (1989) is presumably because it is a smooth (soft) approximation of the positive part of x, which is sometimes denoted with a superscript plus, x^{+}:=\max(0,x).

01Alternative forms

This function can be approximated as:

\ln \left(1+e^{x}\right)\approx {\begin{cases}\ln 2,&x=0,\\[6pt]{\frac {x}{1-e^{-x/\ln 2}}},&x\neq 0\end{cases}}

By making the change of variables x=y\ln(2), this is equivalent to

\log _{2}(1+2^{y})\approx {\begin{cases}1,&y=0,\\[6pt]{\frac {y}{1-e^{-y}}},&y\neq 0.\end{cases}}

A sharpness parameter k may be included:

f(x)={\frac {\ln(1+e^{kx})}{k}},\qquad \qquad f'(x)={\frac {e^{kx}}{1+e^{kx}}}={\frac {1}{1+e^{-kx}}}.

Additionally, the softplus function is equivalent to the log of the sigmoid function in the following way:

-\ln({\text{sigmoid}}(-x))=-\ln \left({\frac {1}{1+e^{x}}}\right)=\ln \left(1+e^{x}\right)={\text{softplus}}(x)

Watch videos about SoftplusExplainers and documentaries on YouTube (opens in a new tab)

Sources and credits

This article is adapted from the Wikipedia article “Softplus”, written by its contributors and licensed under CC BY-SA 4.0. Fathomly has changed the layout, removed citation markers, navigation and maintenance notices, and adjusted punctuation. This adapted version is shared under the same license. For references, see the original article.

Images, from Wikimedia Commons:

Fathomly is not affiliated with or endorsed by the Wikimedia Foundation. Spotted a problem? Tell us.