You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
rule for tanh has catastrophic cancellation for |x| > 20 #102
One could simply revert #30 to fix this, but if one wants CSE then it can be calculated from first principles. I started a discussion which has numerical examples.
Note that the catastrophic cancellation happens way sooner than |x|=20 … by x = 4 the 1 - tanh(x)^2 formula has already lost 2 digits, by x = 6 it has lost 4 digits, and by x = 14 it has lost 10 digits.
Activity
devmotion commentedon Dec 13, 2023
ChainRules should suffer from the same problem, possibly the rule was just copied to it from DiffRules: https://github.com/JuliaDiff/ChainRules.jl/blob/40b9058c1a6798a4f72bda1227786d584b44c822/src/rulesets/Base/fastmath_able.jl#L41
stevengj commentedon Dec 13, 2023
Note that the catastrophic cancellation happens way sooner than |x|=20 … by
x = 4the1 - tanh(x)^2formula has already lost 2 digits, byx = 6it has lost 4 digits, and byx = 14it has lost 10 digits.See also @tpapp's discourse thread on this.
Avoid catastrophic cancellation in tanh and inverse trig/hyperbolic d…
Avoid catastrophic cancellation in the derivatives of `tanh` and inve…
Accurate derivative of tanh for large |x|