Overflow Error in Python's numpy.exp function
As fuglede says, the issue here is that np.float64
can't handle a number as large as exp(1234.1)
. Try using np.float128
instead:
>>> cc = np.array([[0.120,0.34,-1234.1]], dtype=np.float128)
>>> cc
array([[ 0.12, 0.34, -1234.1]], dtype=float128)
>>> 1 / (1 + np.exp(-cc))
array([[ 0.52996405, 0.58419052, 1.0893812e-536]], dtype=float128)
Note however, that there are certain quirks with using extended precision. It may not work on Windows; you don't actually get the full 128 bits of precision; and you might lose the precision whenever the number passes through pure python. You can read more about the details here.
For most practical purposes, you can probably approximate 1 / (1 + <a large number>)
to zero. That is to say, just ignore the warning and move on. Numpy takes care of the approximation for you (when using np.float64
):
>>> 1 / (1 + np.exp(-cc))
/usr/local/bin/ipython3:1: RuntimeWarning: overflow encountered in exp
#!/usr/local/bin/python3.4
array([[ 0.52996405, 0.58419052, 0. ]])
If you want to suppress the warning, you could use scipy.special.expit
, as suggested by WarrenWeckesser in a comment to the question:
>>> from scipy.special import expit
>>> expit(cc)
array([[ 0.52996405, 0.58419052, 0. ]])
The largest value representable by a numpy
float is 1.7976931348623157e+308, whose logarithm is about 709.782, so there is no way to represent np.exp(1234.1)
.
In [1]: import numpy as np
In [2]: np.finfo('d').max
Out[2]: 1.7976931348623157e+308
In [3]: np.log(_)
Out[3]: 709.78271289338397
In [4]: np.exp(709)
Out[4]: 8.2184074615549724e+307
In [5]: np.exp(710)
/usr/local/bin/ipython:1: RuntimeWarning: overflow encountered in exp
#!/usr/local/bin/python3.5
Out[5]: inf
A possible solution is to use the decimal
module, which lets you work with arbitrary precision floats. Here is an example where a numpy
array of floats with 100 digits precision is used:
import numpy as np
import decimal
# Precision to use
decimal.getcontext().prec = 100
# Original array
cc = np.array(
[0.120,0.34,-1234.1]
)
# Fails
print(1/(1 + np.exp(-cc)))
# New array with the specified precision
ccd = np.asarray([decimal.Decimal(el) for el in cc], dtype=object)
# Works!
print(1/(1 + np.exp(-ccd)))