J-ottings 35: How to Do Things with Words
Vector 19:3, page 40
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 40
David Ness recently drew my attention to a fascinating little book called "How to Do Things with Words". This is an edited version of lectures given in 1955 at Harvard by an Oxford philosopher called John Austin, but the title is equally appropriate as a succinct answer to an enquirer seeking to discover what J is all about. The gist of the book is that there are some things in life which can only be done by making utterances, "I name this ship …", "I baptise this baby …" and so on. The pursuit of analogies with the grammars of natural languages has always been one of the driving forces of J, and it is not accidental that what in other languages are called "functions" are called "verbs" in J, while "constants" and "variables" are called "nouns".
Observers of the jsoftware forum will know about a new charged-for version of J called J5.01 which has just been introduced and which heralds J’s maturity as a computer product widely used on many different platforms and within a great many environments. The increase in reported problems, some presumably more significant than others, is an inevitable part of this complexity, and demonstrates how the language interpreter has shrunk to a disproportionately small part of the total J structure. The topics in Strand Software’s advanced course in New York last autumn were things like OCX, ODBC, sockets and so on. By contrast, the predecessor version called J4.06d with its near perfect interpreter remains downloadable and free to all, and in Delphic style, provides answers to questions about what happens when words are put together in combinations which Roger himself could not possibly have ever conceived. While it seems that an increasing number of computer specialists are now using J in conjunction with the many other products and adjuncts of modern computing systems and will thus sign up for J5.01, the perfection of J4.06d looks likely to remain a de facto standard for those whose quests are restricted to what are perhaps more enjoyable and less frustrating linguistic endeavours.
In its rigorous judgements of J competence, the J4.06d interpreter contrasts with the manner in which "perfect" English is not an absolute standard but an inherently unattainable ideal. When my German born travel agent answers my question "Is that Brigette?" with the response "This is she", by every formal rule of English grammar this has to be judged totally correct, but not many native English speakers would instinctively consider it so. On the other hand Roger’s creation answers all possible questions of grammar and syntax (including millions which have not yet been asked) with total certainty and clarity. It is in effect a super-intelligence to which even the most accomplished J "speakers" must submit an implicit validity test with every J phrase which they utter or compose. Such speakers rarely react to an error report or unexpected result by presuming a bug is present, but rather accept that their instinctive and developing notions of the language do not yet quite measure up to the performance ideals of the unforgiving language examiner!
Much was made of APL as a notation for thought, and of the idea that, to quote A.N. Whitehead, "by relieving the brain of unnecessary work a good notation sets its users free to concentrate on more advanced problems thereby increasing the powers of the race". It might be supposed that this notion carried over into J, but this is not entirely correct. APL is first and foremost a shorthand, J only incidentally so. At an early age we develop fluency in the language we hear all around us, and we rapidly achieve speech as opposed to language skills. For example the child who says "throw Daddy" has acquired a language skill in combining words in a fashion which mimics the rhythm of the social interactions he hears in the adult speech surrounding him. The speech skill by which this utterance develops into "throw it to Daddy" comes at a later stage. Combining words in J and submitting them to an interpreter is like submitting "throw Daddy" to the full glare of perfect adult intelligence, leading to a possible unwanted response where a physically strong and highly spirited Mummy happens to be present!
This analogy must resonate clearly with anyone who has crafted and executed a J verb with a clear expectation of what it will perform. Surprise is frequently conveyed not by an error report, but by the delivery of some result, which, if carefully observed, can advance a general background goal of bringing the user’s fluency in J closer to the perfection which only the super-intelligence can attain. In other words, reflective consideration of faulty utterances is the prime route towards the attainment of language mastery. It is all too easy to set such explorations aside without fulfilling good intentions to return and learn from the experience. Thus many J speakers never progress much beyond the "throw Daddy" stage, or at best graduate only as far as the language equivalent of infanthood!
I said above that J, at least in the perceptions of some of us, is about doing things with words, and I return now to that theme in the context of the two types of rank which seem to be one of the principal stumbling blocks for initial language learners. The two types of rank are noun rank and verb rank. The former is a relatively simple idea to grasp, namely the result of applying #$ to an object to give the tally of its shape, or, in a single word, its dimensionality. Shape itself has geometrical connotations, so that a scalar has rank 0, akin to a point in space, a list has rank 1 like a line, a matrix has rank 2 analogous to points in a plane, and so on.
Verb rank is somewhat more subtle, but has more than a passing association with the grammatical idea of "number", that is singular or plural. It is also helpful to consider rank alongside the concepts of box and cell, both of which are means of "singularising" things, the former in a much stronger way by disguising the nature of the contents from external agents such as verbs. The Object Oriented concept of encapsulation is highly relevant here, so that a box is a scalar whereas a cell is not, the key difference being the property of penetrability.
The majority of J verbs have rank of either zero or infinity. It is only for verbs which have special semantic requirements such as shape and matrix inverse that any other rank is associated with the verb. Basically, the "safe" assumption about the rank of an unknown verb is that it is infinite, which is why all but the most trivial user-defined verbs, are assigned infinite rank. (There are occasions where, with analysis, a "true" rank could in principle be worked out, but such computing effort would not only overload the interpreter for little practical gain, but would also cost greatly in performance.)
Simple addition at different rank levels involves thinking about cells with possibly complex list structures. Addition without explicit rank can penetrate list structures by applying the basic verb (that is the one to the left of the rank conjunction) at rank 0 :
x=.i.2 3 NB. a list of two 3-lists
y=:>:x NB. another list of two 3-lists
x;y;x+y
+-----+-----+------+
|0 1 2|1 2 3|1 3 5|
|3 4 5|4 5 6|7 9 11|
+-----+-----+------+
However, adding objects with different structures may result in shape incompatibility, for example in the following case
]z=.10*i.2 2 3 NB. two 2-lists of 3-lists
0 10 20
30 40 50
60 70 80
90 100 110
x+ z NB. two 2-lists plus two 3-lists
|length error
| x +z
you cannot meaningfully add 3-lists to 2-lists! However, addition at rank 1 is valid
x +"1 z NB. add two lists to two lists of lists
0 11 22 NB. this is 0 1 2 + plane 0 of z ..
30 41 52 NB. ..with list 0 1 2 replicated.
63 74 85 NB. and this is 3 4 5 + plane 1 of z ..
93 104 115 NB. ..with list 3 4 5 replicated.
because it is an addition of two compatible 2-lists. Each of x’s two lists is a 3-list, whereas for z, both its lists are 2-lists, each item of which is a 3-list, and it is at the level of the 3-lists that the actual addition takes place. Addition at rank 2 is also valid:
x+"2 z NB. add one list to two lists
0 11 22 NB. this is x + plane 0 of z ..
33 44 55
60 71 82 NB. .. and this is x + plane 1 of z
93 104 115
In this case, a single 2-cell is added to a list of 2-cells, and so in this case the left argument x is replicated in its entirety.
+ is one of a subset of verbs, mainly arithmetic, which "penetrate" indefinite layers of structure to operate at atomic (that is numeric or character) level, as in the case of x+y above. In APL these are called scalar verbs, and their counterparts in J are said to possess rank 0. Where scalar verbs operate on conformable arguments as with x+y, increasing verb rank has no effect on the result. This is because, for example, x+"1 y is the addition of two pairs of 1-cells and x+"2 y is the addition of a single pair of 2-cells, and in each case penetration takes place down to atomic level. In J terms
Returning to box, whereas rank 0 verbs penetrate list structures, they do not penetrate boxed structures:
In general, the permitted operations on boxed scalars are restricted to shaping and joining, since encapsulation means that the nature of their contents is not available to verbs. To repeat the warning given earlier, do not confuse cells and boxed scalars!
Applying adverbs to verbs with rank continues the same rigorous logic. For example applying table :
$x+/z NB. All possible atomic pairs are added
2 3 2 2 3
$x+"1/z NB. A 2 by 2 table (items=3-lists)
2 2 2 3
$x+"2/z NB. 1x2 table(items=2-lists of 3-lists)
2 2 3
In the first case the shape of the result is the catenation of shapes. If this were not so it would be impossible to accommodate all possible number pairs. However, when two lists are added, addition takes place only between matching items, which means that in forming the second table the two 3s at the lowest shape level in each argument are merged. Similarly, in the last case it is the two 2 3s which are merged by addition to give a 2-list, each item of which has shape 2 3. If you want to confirm your understanding of all this, try to predict the results of x+/"1 z and x+/"2 z. It helps to appreciate that "table addition" is not a scalar verb, that is, it has infinite rank. You can use verb rank to specify the level at which tables are to be formed so that, for example, x+/"0 y is identical to x+y. However, no penetration takes place below the level specified by an explicit rank. Thus :
$x+/"1 z NB. A 2 by 2 table of 3 by 3 tables
2 2 3 3
$x+/"2 z NB. A 1 by 2 table of 2 by 3 tables
2 2 3 2 3
In the first case above the verb +/operates item by item on two pairs of lists to produce a 2 by 2 structure of 3 by 3 matrices.
Now return to natural language. Verbs like "quit" and "bid" make no number distinction – "he quit, they quit" and so on – as opposed to verbs like "spell" – "he spells", "they spell" – which do. The concept of "plurality" with respect to verbs is a little more subtle than it might at first appear. For example if I say (talking about a group of people) "they sang", it could be taken to mean that each individual in the group sang, that is, the singing was performed at rank 0. On the other hand, the statement "they sang" could equally be applied to, say, a crowd at a football match where it was almost certainly not the case that every single individual sang. On the other hand "they multiplied" (in the reproductive sense!) could not have been done by individuals, and so we have here a verb which whose rank is certainly greater than zero! Similarly, if an utterance like "they quit" meant that all of them did as a body, then we have an instance of infinite rank. At best English verbs allow the distinction between one and many, that is between singular and plural. Explicit verb rank in J allows all possible gradations in between, 0, 1, 2 and so on up to infinity, and further, it allows ranks to be specified separately for subject and objects, much as if English were to provide separate inflections for "he hitss her, "he hits them", "they hit them" and "they hitt him". With a commutative verb like + there is no difference in result between x+"0 1 y and x+"1 0 y. However, the distinction is immediately apparent with a non-commutative verb :
x-"0 1 y NB. atoms of x minus rows of y
_1 _2 _3 NB. 0 minus 1 2 3
0 _1 _2 NB. 1 minus 1 2 3
1 0 _1 NB. 2 minus 1 2 3
_1 _2 _3 NB. 3 minus 4 5 6
0 _1 _2 NB. 4 minus 4 5 6
1 0 _1 NB. 5 minus 4 5 6
x-"1 0 y NB. rows of x minus atoms of y
_1 0 1 NB. 0 1 2 minus 1
_2 _1 0 NB. 0 1 2 minus 2
_3 _2 _1 NB. 0 1 2 minus 3
_1 0 1 NB. 3 4 5 minus 4
_2 _1 0 NB. 3 4 5 minus 5
_3 _2 _1 NB. 3 4 5 minus 6
In English we tolerate the absence of such inherent number precision, which makes explicitly ranked J verbs harder to grasp than English ones – but on the other hand what interesting and varied things one can do with them!