APL and J (1) - Function Rank
Vector 13:1, page 96
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 96
Though J shares many concepts with APL, in many respects it is radically different, and almost all APL constructions that are in J differ in some way. These differences can be a stumbling block to the experienced APL programmer who would like to learn a little about J.
In a series of articles entitled APL and J, we will try to explain the J way of thinking, by contrasting APL and J solutions to specific problems. The term APL will be used generically to refer to facilities found in most commercial implementations of APL.
We will start off by taking a look at function rank — one of J’s most important new facilities — and also look at the related topics of J’s agreement and assembly rules.
Housekeeping¶
We are going to need several arrays, and will use the letters V, M, and A followed by numbers to denote arrays and their shape. Thus V5 for a vector of length 5, M34 for a 3-by-4 matrix, A243 for a 2-by-4-by-3 array and so on. Where actual values are required, V5 will be defined as ⍳5, M34 as 3 4⍴⍳12, etc.
APL terminology will be used, except where J terminology is required for clarity. ⎕io←1 is assumed throughout.
Agreement¶
Suppose we want to multiply two arrays together (or apply any scalar function to the arrays). What shapes must they have in order for the operation to be successful? In APL, the answer is straightforward: they must either have the same shape, or else one of them must be a scalar, in which case it is reshaped to match the other (this is known as scalar extension).
What happens if these conditions are not met, but the operation required is otherwise quite reasonable? For example, it is often the case that you want to multiply two arrays, where the shape of one array matches some leading elements, or trailing elements, of the shape of the other. Let’s look at some simple examples, and typical APL solutions.
Example 1¶
The arrays are M34 and V4 and we want to multiply each row of the matrix by the vector. Straight multiplication fails:
The error is a rank error, since APL requires two conforming non-scalar arrays to have the same rank. A standard solution in APL is to reshape the vector into a matrix:
Another solution uses the following three-step process, which is of general utility. The data is split up into enclosed items, the function is applied to each item in turn, and the results are then re-assembled:
Example 2¶
Now let’s consider the case M34 times V3, where we want to multiply each column of the matrix by the vector. Reshape still works, but the shape of the matrix must be reversed, and the result must be transposed:
Similarly, three-step needs the result transposed:
Example 3¶
In the next example, we multiply each 3-by-4 block of the array A234 by the matrix M34. The methods of Example 1 can be used essentially unchanged:
Example 4¶
Finally, we multiply each 3-by-4 block of A342 by the matrix M34, as follows:
Solutions in J — Examples 2 and 4¶
In APL, the default axis is the last axis, whereas in J, it is the first axis. Therefore, while in APL, Examples 2 and 4 are more complex than Examples 1 and 3, in J they are simpler:
Why do these multiplications work in J? The answer is that two array arguments agree if the frame of one array is a prefix of the frame of the other. This is an extension of the rules of APL.
The term ‘frame’ will be explained later, but for now, you can substitute the word ‘shape’ with the same effect. Thus, in Example 2, the shape of V3 is the first element of the shape of M34, while in Example 4, the shape of M34 is the first two elements of the shape of A342. In such cases, J extends the smaller array to match the larger; this is known as prefix extension.
Solutions in J — Examples 1 and 3¶
Direct multiplication does not work this time in J, but note that the error is a length error, not a rank error:
The solutions use the rank operator:
You can get a rough idea of what is going on by thinking of the numeric argument to rank as being the dimension to which the function applies. Thus in J Example 1, the function is applied to the rank 1 arrays, which are the rows of M34, and the vector V4 as a whole. In J Example 3, the function is applied to the rank 2 arrays, which are the matrices of A234, and the matrix M34 as a whole.
Function Rank¶
In APL, data may be of arbitrary size and dimension, where the dimension of the data is referred to as its rank. J extends this notion to functions — all functions are assigned a rank which determines the behaviour of the function on certain subarrays called the cells.
The shape of an array can be split up into two parts, a leading part called the frame, and the remaining part called the cells. Either frame or cells can be empty. The frame can be thought of as the holding structure for the cells. The term k-cell specifies a cell of rank k.
For example, the shape of the matrix M46 can be split up in three different ways:
- the size of the frame is empty, and size of the cell is 4 by 6. The cell is a 2-cell and is the entire matrix.
- the size of the frame is 4 and the size of the cells is 6. The cells are 1-cells and are the rows.
- the size of the frame is 4 by 6, and the size of the cells is empty. The cells are 0-cells and are the scalar elements of the matrix.
A function of rank k acts on the k-cells, with the following three-step process:
- the arguments are split up into k-cells.
- the function is applied between each pair of k-cells.
- the individual results are then reassembled.
Note that these steps are analogous to the three-step APL examples. However, they are not identical. In APL, the steps must be given explicitly, and the interpreter carries out each step in turn. In J, the steps are only notional; in general the interpreter knows how to evaluate functions with rank and carries out the evaluation directly.
The rank operator can be used to enforce different argument pairings, which are independent of the function to which it is applied. The function is then applied to each of the pairs, and in turn takes care of its own pairings.
Applying Function Rank¶
Now let’s see how function rank is applied in the examples above. In J Examples 2 and 4 no function rank was specified, therefore the multiplications are carried out with the rank assigned to *, which is 0. This means that the cells are the scalars, and the frames are the entire argument shapes. This explains why, in the discussion on J Examples 2 and 4, it was stated that the term ‘frame’ could be replaced by the word ‘shape’. In J Example 1, the pairing is specified with rank 1. The three-step J process is thus:
Step 1. Break arguments into 1-cells. The 1-cells of M34 are the rows:
The 1-cell of V4 is the entire vector: (1 2 3 4)
Step 2. The function is then applied between the cell pairs:
Step 3. The individual results are then assembled into:
In J Example 3 the pairing is specified with rank 2. The three-step J process thus uses the 2-cells, which for A234 are the matrices:
The 2-cell of M34 is the entire matrix:
This cell is multiplied with the 2-cells of A234 and the results assembled into:
Rank vs Each¶
J’s rank operator " and APL’s each operator ¨ are notationally similar, and are often confused. To understand the difference, note that the each operator is used for two essentially different expressions, for which J has two quite different solutions.
The first case is that discussed above, where APL each is used as part of the three-step solution to applying a function to non-conforming arrays; this usage corresponds to J’s rank operator.
APL each also can be used to simply apply a function to each element of an array, typically an enclosed array. The corresponding expression in J is also called each, which is a standard utility defined as &.>. In such cases, the following are the same:
Assembly¶
The third step of applying rank is to re-assemble the results. In the examples given here, assembly presented no problem since each item had the same shape. In practice, J can assemble items of different rank and shape as follows: the items are first brought to a common rank by introducing leading unit axes to any of lower rank, and are then brought to a common shape by padding with an appropriate fill element. This allows J to assemble results which in APL would signal rank or length errors:
The combination of J’s more general agreement rules, function rank and permissive assembly allows many more calculations to be expressed in J without looping or reshaping.
References¶
- Iverson, K.E. J Introduction and Dictionary, Iverson Software, 1996
- Hui, Roger K.W. Rank and Uniformity, APL Quote Quad, June 1995 (APL95)
- Neitzel, Martin. Untying the Gordian Knot: Agreement in J, APL Quote Quad, June 1995 (APL95)