APL and J (3): Operators Reduce, Scan, Outer Product
Vector 13:3, page 118
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 118
This series of articles entitled APL and J will try to explain the J way of thinking, by contrasting APL and J coding examples. The term APL will be used generically to refer to facilities found in most commercial implementations of APL.
For the next few articles, I will discuss operators. To start off with, we will take a look at J’s use of operators as a whole, then look specifically at reduce, scan and outer product.
Housekeeping¶
As usual, we use the letters V, M, and A for vectors, matrices and arrays of any dimension, optionally followed by numbers to denote their shape. Thus V for any vector, V5 for a vector of length 5, M45 for a 4-by-5 matrix, A245 for a 2-by-4-by-5 array. Where values are required, V5 will be defined as ⍳5, M45 as 4 5⍴⍳20, for example:
APL terminology will be used, except where J terminology is required for clarity. Since J uses origin 0 only, here ⎕io←0 is assumed throughout.
Overview¶
J makes much greater use of operators than APL, and this is probably the hardest aspect of J to grasp. For example, if you tried to write programs without using operators at all, in APL this would be a minor inconvenience, whereas in J you would be completely stuck - since all J programs are created with operators!
In general, while operators are common in APL, they typically occur in a relatively few expressions such as +/ and ∧.= which become familiar and are used as if they were primitive functions. Thus the APL programmer may try to solve a problem by thinking solely in terms of the "functions" that can be brought to bear on it. This approach will not work in J; or at least, if you try to think solely in terms of functions, you will find it tough going! Operators are essential in solving problems, and it is important in J that you develop a good understanding of how they work.
Moreover you not only use operators, you also need to be able to define them, whereas in APL it would be unusual to define new operators to solve problems. Indeed one major implementation, APL+Win, does not even allow you to define new operators.
This focus on operators in J can be a bit intimidating to the newcomer, who may think that programming with operators is difficult. True there is some learning involved, but once mastered, it becomes pretty simple. Moreover, it gives you a much clearer understanding of how APL works - a good reason to learn J is that it will teach you so much about APL!
Tacit Definition¶
An operator expression like +/ is called a tacit definition, in that it defines a new function without referring to the new function’s argument. Compare with explicit definitions which do refer to the function’s argument (e.g. any APL function defined with the function editor). Tacit definitions occur frequently in both APL and J, though the terminology is new to J and can be a source of difficulty. I have heard experienced APL programmers say they do not understand tacit definition - clearly they are unaware that they use it all the time in APL.
Tacit definition may be clarified by giving the new function a name, which in APL is only possible with Dyalog APL (but which should be possible in all APLs):
sum← +/ |
Dyalog APL |
sum=. +/ |
J |
Using Dyalog’s composition operator, it becomes easy to generate the kind of tacit definitions so often seen in J:
When should you use tacit, and when explicit definition? Whenever you feel like it! For example the following APL function uses both tacit and explicit definition:
Is tacit definition better than explicit definition? It certainly can be - try writing an APL function that simulates +/ without using tacit definition. Then modify this function to handle the general case of f/.
Now let’s look at some APL operators.
APL Reduce¶
At first sight, reduce in J (called insert) seems to be the same as in APL:
However, with a matrix argument, APL applies +/ along the last axis, and J along the first axis:
So +/ in J is closer to APL’s +⌿ or +/[0], i.e. plus reduce along the first axis. To apply +/ in J along the last axis as in APL, apply it rank 1, using the rank operator:
Since reduce applies a function between each element of a vector, you might think that ,/ is an identity on a vector. This is true in J, but not in APL:
Instead, ,/ in APL encloses its result:
What is happening is that insert in J inserts the function between the items of its argument, whereas reduce in APL inserts the function between the elements of the vectors along the last axis, and encloses the result for each vector, so that:
means:
Strictly speaking, the latter is the APL2 definition of reduce, seen in APL2, APL+WIN and Dyalog APL. The treatment in SHARP APL is essentially that of J, except items are based on the last axis. It is interesting that APL does an implicit enclose. It is very often the case that an APL definition uses enclose, where J does not. For example:
,/ M34 J
0 1 2 3 4 5 6 7 8 9 10 11
display ,/ M34 APL
┌→────────────────────────────────┐
│ ┌→──────┐ ┌→──────┐ ┌→────────┐ │
│ │0 1 2 3│ │4 5 6 7│ │8 9 10 11│ │
│ └───────┘ └───────┘ └─────────┘ │
└∊────────────────────────────────┘
Since there is no difference in APL between a scalar and an enclosed scalar, the final result of an APL reduce may be simple. For example, +/M34 is:
which is:
which is:
The effect of ,/ M34 in APL can be produced in J using link insert:
The distinction between the way reduce applies to vectors, and the way insert applies to items becomes clearer with a 3D argument:
display ,/ A234 APL
┌→──────────────────────────────────────────┐
↓ ┌→──────┐ ┌→──────┐ ┌→────────┐ │
│ │0 1 2 3│ │4 5 6 7│ │8 9 10 11│ │
│ └───────┘ └───────┘ └─────────┘ │
│ ┌→──────────┐ ┌→──────────┐ ┌→──────────┐ │
│ │12 13 14 15│ │16 17 18 19│ │20 21 22 23│ │
│ └───────────┘ └───────────┘ └───────────┘ │
└∊──────────────────────────────────────────┘
;/ A234 J
┌─────────┬───────────┐
│0 1 2 3│12 13 14 15│
│4 5 6 7│16 17 18 19│
│8 9 10 11│20 21 22 23│
└─────────┴───────────┘
,/ A234 J
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
16 17 18 19
20 21 22 23
Replicate¶
In APL, the symbol / is used both for reduce and for replicate, whereas in J, replicate is given by a different symbol, # (copy).
APL Scan¶
Scan is called prefix in J, and is clearly not the same as in APL:
The difference is that in APL, scan applies the verb with reduce, while J just applies the verb. Thus
means:
Note that in the APL case f is dyadic, whereas in J, f is monadic.
In J, each application of f returns an item of the result, so that
is:
J assembles these four items into a matrix, padding with zeroes as required.
To simulate APL scan, you need to use reduce explicitly:
The benefit of not having an automatic reduce in prefix, is that many operations do not require it. Such operations are straightforward in J, but may be awkward in APL. The following is a reverse scan in J – how would you do this in APL?
<@|.\ i.6
┌─┬───┬─────┬───────┬─────────┬───────────┐
│0│1 0│2 1 0│3 2 1 0│4 3 2 1 0│5 4 3 2 1 0│
└─┴───┴─────┴───────┴─────────┴───────────┘
For another example, to enclose successive prefixes of a variable, you can use box prefix in J, but not enclose scan in APL.
Curiously, since ,/ is an APL reduction that encloses its argument, it turns out that at least for a vector, ,\ performs an enclose scan!
display ,\ 1 2 3 4
┌→───────────────────────────┐
│ ┌→──┐ ┌→────┐ ┌→──────┐ │
│ 1 │1 2│ │1 2 3│ │1 2 3 4│ │
│ └───┘ └─────┘ └───────┘ │
└∊───────────────────────────┘
Inverse¶
Some uses of prefix have inverses. For example, the inverse of +/\ is the first differences, and of */\ is the successive ratios:
Suffix¶
Suffix is analogous to prefix, except it applies to lists of elements taken from the end. Thus a reverse sum scan is:
APL Outer Product¶
In J, the verb defined using insert may also be used dyadically (when it is called table), and is similar to APL’s outer product.
1 2 3 +/ 10 20 30 40 J
11 21 31 41
12 22 32 42
13 23 33 43
1 2 3 ∘.+ 10 20 30 40 APL
11 21 31 41
12 22 32 42
13 23 33 43
Again, the definitions are not quite the same. Table in J applies the function between each cell of the left argument, and the entire right argument, whereas outer product in APL applies the function between each pair of elements from the left and right argument. Consider the following APL example of catenating each pair of elements of two vectors:
display 1 2 ∘., 10 20 30
┌→─────────────────────┐
↓ ┌→───┐ ┌→───┐ ┌→───┐ │
│ │1 10│ │1 20│ │1 30│ │
│ └────┘ └────┘ └────┘ │
│ ┌→───┐ ┌→───┐ ┌→───┐ │
│ │2 10│ │2 20│ │2 30│ │
│ └────┘ └────┘ └────┘ │
└∊─────────────────────┘
You might expect that ,/ does the same thing in J, but it does not:
This is because the rank of catenate is unbounded, hence the left argument has only one cell (the entire argument), and this is applied to the entire right argument. The above expression is therefore the same as:
You can change this behaviour by using rank. For example, if both ranks are 0, then J will catenate each pair of elements as in APL. The individual results are assembled in a simple array, rather than enclosed:
To duplicate APL’s behaviour exactly, use box atop catenate:
1 2 <@,"0/ 10 20 30
┌────┬────┬────┐
│1 10│1 20│1 30│
├────┼────┼────┤
│2 10│2 20│2 30│
└────┴────┴────┘
Alternatively, by setting the left rank to 0 and leaving the right rank unbounded, you can catenate each element of the left argument to the entire right argument:
Leaving the left rank unbounded and setting the right rank to 0 catenates the entire left argument to each element of the right argument.
N-Wise Reduction¶
In APL2, a reduce may take a left argument that specifies the number of items to be used in each application of the function, called n-wise reduction, e.g.
This is:
In J, this is handled by prefix (where it is called infix). As in APL2, the left argument specifies the number of items to be used. If positive, the items overlap; if negative, the items are non-overlapping. For example:
As with prefix, the verb argument to infix need not be applied with reduce. For example:
A neat way of creating a matrix with a given number of columns from a vector, is to use identity infix:
Reference¶
Iverson, K.E. J Introduction and Dictionary, Iverson Software, 1996