J-ottings 21: Time for amendment of data ...
Vector 16:1, page 56
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 56
Item and atom are two words which differ only in a pair of vowels, yet in J their meanings are a world apart. What is an item? If you can’t answer this with immediate assurance, then write no more J until you have read this article! Think of the USA as an object to be subdivided down to an eventual level of its individual citizens as atoms. If the highest level of breakdown is states, a typical item is Florida or California or Alaska. Wrap the first layer off the USA parcel, and ask how many items are inside. Answer, 50. Each of these items is itself an object and contains items, counties perhaps, and so the hierarchy continues. Conceptually the terms “object” and “item” cascade downwards, the former being always just one level higher in the hierarchy. Thus the object a=:i.2 3 4, has two items each of which is a 3 by 4 matrix. The “parcel unwrapping” verb which drops a level from object to item and reveals the shape of what is inside is $"_1 :
Introducing “amend”¶
Selective assignment as in A[3 5 9]←14 25 90 in APL is a great convenience, but disguises the fact that there are really two processes telescoped into one. The first process involves a data transformation which selects those parts which are to be changed, while the second process does the actual replacement with a second set of data. Informally you can think of the selector as “sandwiched” between the new and old data as in the following illustration in which a=:i.2 3 4 (that is 2 planes, 3 rows and 4 columns):
new selector } old
(9) 1 } a NB. Replace the second item (plane) with 9's
0 1 2 3 NB. Parens are needed round either the 1 or
4 5 6 7 NB. the 9 or both. "Amend" (}) is
8 9 10 11 NB. an adverb which qualifies "selector".
NB. "Selector" is in general a verb, however
9 9 9 9 NB. it can also be a noun as here. Think
9 9 9 9 NB. therefore of 1} as a verb with
9 9 9 9 NB. arguments "new" and "old".
The phrase (9) 1} a demonstrates that a J adverb, unlike an adverb in ordinary grammar, may qualify either a noun or a verb. A later example will show } qualified by a verb.
By default selection takes place at the level of items within objects, that is at rank 3, but the rank conjunction allows indexing to apply at lower levels:
12 9 14 15
16 9 18 19
20 9 22 23
(9) 1}"2 a NB. Replace all second ROWS with 9's.
0 1 2 3 NB. Selection is made within rank 2 objects,
9 9 9 9 NB. that is vectors of vectors.
8 9 10 11 NB. These objects are the items of a.
12 13 14 15
9 9 9 9
20 21 22 23 NB. (9) 0}"0 a replaces all atoms in a with 9.
A Data Amendment Problem¶
The problem is to masculinise American state names, for which two distinct kinds of technique are available, namely transformation followed by assignment, and amendment. With the former, and where “real” data is concerned, there is frequently a choice of two further possibilities, namely using simple lists or using boxed lists. Using lower and upper case names respectively to correspond to these options define
USA=:'Florida' ; 'California' ; 'Alaska' NB. Data definitions
]usa=:>USA
Florida NB. USA is a boxed list
California NB. usa is a simple list
Alaska
masc=: ,&'o'@}: NB. verb "masculinise" (simple)
MASC=:masc each=:&.> NB. verb "masculinise" (boxed)
MASC USA NB. masculinise a boxed list
┌───────┬──────────┬──────┐
│Florido│Californio│Alasko│
└───────┴──────────┴──────┘
dtb=:{.~i.&' ' NB. delete trailing blanks (hook)
(masc@dtb)"1 usa NB. masculinise items of simple list
Florido
Californio
Alasko
Here are a few comments as asides:
- (a)
- Compared with the boxed list the simple list incurs a space penalty imposed by padding, which in larger applications might be significant.
- (b)
- the verb
dtbdemonstrates a typical context in which a hook arises, when data has to be processed according to some function of itself, in this case “index of first blank” (i.&' '). Since the result of this transformation becomes the left argument of “take”, the “reflex” adverb (~) is required. - (c)
-
The verb “catalogue” (
{) converts a simple list into a boxed list while retaining any fill characters, for example
so if application data is in the form of a simple list, but boxed list processing has been chosen, it can be applied “with catalogue”. For example suppose that MASC is applied to usa following right justification using the same technique as in dtb:
rj=.(|.~ i.&' ')"1 NB. Right justify character matrix
MASC&{ rj usa
┌──────────┬──────────┬──────────┐
│ Florido│Californio│ Alasko│
└──────────┴──────────┴──────────┘
Returning to “amend” ...¶
The alternative to transformation with subsequent assignment is amendment. In the previous illustration of amendment, the new data and selectors were explicit nouns (9 and 1). Now a selector verb is required since the set of indices of atoms (i.e. characters 'a') which have to be changed is a function of the data itself.
An implicit assumption so far in the states problem is that the state names have been left justified. This assumption makes it simple to find the index of the final 'a' in each name:
To transform these into indices to define atoms of the list usa, attach the index of the corresponding item, and then box each of the pairs of indices so obtained:
ilc=:<"1@(i.@# ,. llc"1) NB. index last character, e.g. <0 6 for Florida
'o' (ilc usa)} usa NB. new data is 'o', selector indices are
Florido NB. the result of ilc usa
Californio
Alasko
Repeating the data-name usa in the above phrase is inherently unpleasing. However the specification of “amend” allows this to be tidied up by using “tie” to produce a gerund, which at the same time allows the replacement character to be parameterised:
masca=:[`(ilc@])`] } NB. The 3 verbs in the gerund describe
'o' masca usa NB. transformations which produce(left to rt)
Florido NB. (1) the replacement data;
Californio NB. (2) the selectors; and
Alasko NB. (3) if required, transfn. of the old data
Without the gerund option, it would be difficult to accommodate “amend” in explicit definitions. A non-trivial transformation of the old data (i.e. verb (3) in the gerund) might be
lcc=:3 : '(t-32*96<t=.a.i.y.){a.' NB. Convert any l/case chars to upper
mascc=:[`(ilc@])`(lcc@]) } NB. Masculinise and convert
'O' mascc usa
FLORIDO
CALIFORNIO
ALASKO
Using a boxed list raises a difficulty because it is necessary to do an “open” between successive indexing activities. To get round this an “amend” based verb can be defined to work at the item level and then applied to each of the items in the object:
MASCA=.[`(<:@#@])`] } NB. selector verb gives index of last character
'o' MASCA each USA NB. Change each last character to an 'o'
┌───────┬──────────┬──────┐
│Florido│Californio│Alasko│
└───────┴──────────┴──────┘
Item Amend¶
So far, the adverbially qualified verb “selector}” has been used dyadically. It can also be monadic in which case } is called “item amend”. The result has the structure of a single item of the right argument y., and its value is determined by selecting one of y.’s items to provide each item of the result. Each item in y. must therefore contain the same number of items. This sounds quite a mouthful, but a simple example which makes everything plain is the random answering of a multiple choice test. The data is five items each comprising 20 repetitions of the same character; the result of each execution is a further item, each of whose items comes from just one of the original five.
]mch=.|:20 5$'ABCDE' NB. Construct the character matrix which follows:
AAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEE
(?20$5) } mch NB. (?20$5)} is a monadic verb
ADDEBCEAACDABACDCEEC NB. Hopefully you got something different!
When selection depends on the data, for example if it is required to generate the multiple choice responses in strict sequence, use a verb as selector. A verb to obtain repeating integers is