A note on Comparison Tolerance

Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 123

⎕CT is a system variable in APL which is applied when any two numbers are compared. Two numeric values are considered to be equal if their absolute difference is less than or equal to the value of ⎕CT times the larger of the absolute values of the numbers being compared. Hence the applied tolerance varies with the magnitude of the numbers being compared.

For the majority of applications, this system is quite satisfactory, however anomalies can arise which cause a breakdown in the usual rules of mathematics. For example:

      A←÷1E16
      B←0
      ⎕CT←÷1E15

      A=B
0
      (1+A)=1+B
1

… which is contradictory to the assumption that if A, B, C are real numbers and A=B, then A+C = B+C.

The primitive functions affected by ⎕CT are:

… and in Dyalog APL:

When the equality of numbers to within an absolute tolerance is required, the following expression could be applied:

((÷⎕CT)×|A-B)≤1

This will overcome the previously mentioned difficulties.

This method is also suitable for comparing any number with zero as, the = primitive in APL will always return zero when comparing any non-zero value A with zero, irrespective of how close to zero A ever becomes.

This APL expression can be adapted easily to provide absolute comparisons with other aforementioned APL primitives.

Editor’s note: this is one of the few Heffalump traps set by conventional APLs. It recently caught out the ‘nutritional analysis’ in our recipe modelling system; this was supposed to ignore ‘insignificant’ traces of things like Nutmeg, but of course these never quite became equal to zero. The OR scientist who discovered this phenomenon made some distinctly unladylike noises when it was suggested that she add 1 to both sides of the equation!