Reorder a summation, move it to another compiler, let the optimiser contract a multiply and an add into one instruction — and a result that looked settled shifts in the eighth decimal place. Sometimes that does not matter. Sometimes it decides whether an iteration converges.
These are working notes on when it matters, and on how to bound the damage rather than hope it stays small.
I work on numerical software and keep running into the same class of problem: code that is correct in exact arithmetic and unreliable in practice. The notes here are what I write down while sorting it out — partly for colleagues, mostly so I stop rediscovering the same things.
Nothing here is a product. There is no library to install and no service to sign up for. If a note saves you an afternoon, that is the whole intent.
Naive summation of a million single-precision values loses roughly half the significant digits you assume you have. Kahan's compensation costs three extra flops per element and recovers almost all of them. The argument for skipping it is usually speed; the measurement rarely supports that argument.
Fused multiply-add changes results, and whether the compiler emits it depends on flags, target architecture and optimisation level. If you need bit-identical output across machines, you have to say so explicitly — the default is that you will not get it.
A condition number is not a warning label, it is an exchange rate: it tells you how many digits of input accuracy convert into digits of output accuracy. Treating it as a budget you spend makes the arithmetic of error propagation much easier to reason about.
Corrections are welcome, and I would rather hear about an error than not. Write to hello@mathsecure.space.