J-ottings 6: Indexing Arrays in J

Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 25

There is a stage at which fluency in reading any language increases through recognition of idioms, and J is no exception to this rule.

One such idiom is "hook with commute". The simple hook

   (g h) y

means perform h on y, and then, using y as left argument, do g to obtain

   y g (h y)

The second verb h thus acts as a kind of pre-processor for the first verb g. If the first verb is to be the pre-processor, that is if what is required is

   (g y) h y

the commute adverb can be used to reverse the order of g and h, so that the above line is equivalent to

   (h~g) y

The advantage of this "bridged hook" is that only one reference is made to the argument. Informally it may be read as "use g as a transformation, then reverse the arguments".

To take a specific example, suppose that ip=.+/ .* defines the standard matrix inner product, and m′ denotes the transpose of a matrix m. m′m is obtained as

   (ip~|:) m

which is not to be confused with

   ip~|: m

Removing the brackets means that this is no longer a hook, that is, the first verb, |:, is now executed in isolation to obtain m′, Then ~ has the meaning "both" (not "commute") so that m′ is applied as both left and right argument to ip to give a final result m′m′.

Another example of a "bridged hook" is $~$ as used in e.g.

   0($~$)m

to define a matrix of the same shape as m but consisting entirely of zeros.

Another idiom to look out for is the J equivalent of the APL "each" operator. This is the adverb "under open" which applies its verb to a boxed list. It is convenient to define

   each=.&.>

as a general purpose adverb, so that, continuing the above example,

   0 1 ($~$) each <m

defines a boxed list of two matrices shaped like m, one all zeros, the other all ones. If the final boxing is not required (giving the result as a three dimensional array in the present example), use "with open" ( &> ) instead of "under open".

Indexing Arrays in J¶

One of the most difficult areas of the J repertoire to memorise is the "braces" area, which is deeply involved with indexing. This is an area in which J diverges greatly from APL on account of the priority of dimensions inherent in J data structure. New habits must thus be acquired in order to index J arrays.

There are occasions when what are essentially iterative algorithms require descent to the level of array elements, and recourse to looping cannot be avoided. It is therefore important to be able to locate array elements of any order with reasonable confidence.

Broadly speaking, there are two kinds of indexing, rectangular indexing and scatter indexing.

For vectors, { and } are interchangeable, although the former is recommended for consistency with higher rank arrays, leaving } to its primary meaning of "amend". v{w is equivalent to w[v] in APL, where both v and w are vectors.

For higher rank arrays, indexing a single element involves repeating { n times where n is the rank of the array. However, the order of appearance of the indices is the reverse of that in APL, that is if a is a three-dimensional array, k{j{i{a locates element a[i;j;k] in the APL sense. Indices may themselves be arrays, but care has to be exercised in applying APL intuition. For example, if

   ]t=.2 3$'carmen'
car
men

0 1{t selects both rows of t, and so 0 and 1 are the only valid indices at the next level of indexing. Thus 0 2{0 1{t is an index error in J, and is not equivalent to t[0 1;0 2] in APL. However, the latter does have a direct J equivalent. It is obtained by using boxed lists, with one list corresponding to each dimension thus:

   (<0 1;0 2){t
cr
mn

Scatter indexing is the process of selection using sequences of unrelated rectangular indices which select individual items. An essential intermediate process is that of calculating "linear indices" which translate rectangular indices into the corresponding indices in the ravelled array. The verb "base", #., is important in this operation, and is the subject of yet another of the "bridged hooks" described above, namely (#.~$). This converts the index 1 3, say, in a 3 by 4 matrix to the linear index 7.

   1 3(#.~$)i.3 4
7

To apply this as an index, the array right argument must be ravelled, and finally "from" ({) applied. This sequence of two initial transformations followed by a third operation on the transformed data is the essential characteristic of a fork. In the verb ri below, superfluous brackets are retained on the right to emphasise the fork structure.

   ri=.(#.~$) { (,@])     NB. ri = Rectangular Index
   1 1 ri t
e

Finally, to obtain scatter indexing

   si=.>@(ri each)<

uses a hook (ri each)< followed by "open" which sheds the boxed structure.

   (0 0;1 2)si t
cn

Suppose for definitiveness that n is an integer and m a matrix.

n{m        is row n of m
n{"1 m     is col n of m (so also is n}"1 m although
n}m        is a rank error)

Update operations in programming languages have a fundamental requirement for three parameters in order to answer the questions:

What object is to be updated?
Which locations within it are to be changed?
.. and with what data?

The J update verb is "amend", which, alone amongst the primitive verbs cannot be fitted into a monadic/dyadic framework, and thus into patterns of hooks and forks. The order of arguments for what is essentially a triadic verb should be memorised, viz:

   (new data) (locations) } object

Amend allows locations to be expressed as linear indices whatever the rank, and so ri can be adapted to obtain a "linear index" verb:

   li=.>@((#.~$)each <@] )
   'by' ((0 0;1 2) li t) } t
bar
mey

This method of update is an alternative to using the adverbs A and B suggested in the Dictionary under AMEND.

Rows and columns in a matrix can be replaced as follows:

v (n{i.$m) } m     NB. Replaces row n of matrix m with vector v
v (n{"1 i.$m) } m  NB. Replaces col n of matrix m with vector v

When the braces are qualified with . and :, there is little in the physical shapes of the symbols to help indicate which of them denote head, tail and so on.

There are three binary sources of variation present, viz:

{ and } , . and : , +ve and -ve

and my suggestions are to read …

{ as "all the" and } as "all but the"
. as "first" and : as "last" (monadic only)
- as "from the right", and + as "from the left".

Thus n{.m means "all the first n rows of m"
n{."1 m means "all the first n columns of m"

n}.m means "all but the first n rows of m"
n}."1 m means "all but the first n columns of m"

adding "from the left/right" according to whether n is +ve or -ve.

The monadic forms are simply the dyadic case with n equal to 1, so

{.m is the first row (head), and {:m the last row (tail),
{."1 m is the first column, and {:"1 m the last column,
}.m is all but the first row, and }:"1 m all but the last column.

Be aware that "tail" in J does not have the same meaning as say, tail in LISP, which is matched in J by "behead" (}.).

Errata¶

I have had a few minor errors pointed out in J-ottings 5:

p.27 line -6   "rank" should be "shape"
p.27 line -5   "joined" should be "padded and joined"
p.27 line -3   "shape 3 2 4" should be "shape 4 2 4"
p.28 line 1    "shape 4 4" should be "shape 3 5"
p.28 line 2    "matched with" should be "extended and joined to"