laurent.orseau
2022-3-5 10:40:46

No, it’s just plain wrong, and leads to inconsistent results: > (define (sigmoid x) (/ 1. (+ 1 (exp (- x))))) > (sigmoid (/ 1. (max 0 -0.))) 0.0 > (sigmoid (/ 1. (max -0. 0))) 1.0 Both results should return +nan.0


laurent.orseau
2022-3-5 11:02:20

We should have: (max 0. 0) = 0. ; because 0. is +0. and thus (/ 1. max(0. 0)) = +inf.0 (max -0. 0) = 0 ; (/ 1. (max -0. 0)) = +nan.0 (max -0. 0.) = 0. ; (/ 1. (max -0. 0.)) = +inf.0 and same results if input order is reversed.

Dividing by a nonnegative value should certainly not return a negative value.


rokitna
2022-3-5 16:24:19

Since it’s nonnegative but not necessarily exactly known, I mostly expect (max -0.0 0) to be 0.0 regardless of the order. I could also believe 0 if precision is thought of as a positive multiplicative factor and hence negative numbers and positive numbers are kind of separated into two worlds… but the use of max to arrive at this result suggests the application cares about a topology like the number line, where -0.0 is pretty close to some positive numbers.

I’m not sure what I expect for (/ 1.0 0); I probably prefer the current behavior, an exception, since it might be good to keep floats out of exact arithmetic, but I could also believe +nan.0, +inf.0, or even -inf.0. I think the use of division to arrive at this result suggests that a Riemann circle is a relevant topology for thinking about the precision, in which case the signed infinities are the two best approximations and we just have to break the symmetry and pick one.


laurent.orseau
2022-3-5 16:40:18

I’m fine with an exception for (/ 1. 0) or +nan.0 but certainly not +inf.0 or -inf.0. That’s precisely what +0.0 and -0.0 are for, and what they do. If you define (/ 1. 0) as +inf.0 then you should get rid of +0.0 and -0.0 altogether. I’m fine with imprecision. I’m not fine with crossing to the negative side when an operation such as max should expressively prevent that. It’s about user expectations, consistency and principle of least surprise. If I manage to obtain -inf.0 by dividing two positive numbers, that’s certainly surprising.