Using quad-NA for Data Translation

VECTOR Vol.3 No.4

Using Name Association for Data Translation by David Piper

1. Introduction When used in a commercial data processing environment, systems implemented in APL will often be required to address significant volumes of data from external  (i.e. non-APL) files. Since these files are not generated by APL code, they may well contain data in formats not readily accessible by APL. Prime examples of such formats include packed decimal and the IBM S/370 floating point format. One of the supplied workspaces in both the VS APL and APL2 environments contains functions for translating external data into APL numeric form. Since these functions are coded in APL, making use of the most complex APL primitive functions (index-of and encode/decode), execution speeds tend to be somewhat slow. This poor performance is of especial concern when large volumes of data are being translated. 2. Name Association (JNA)

UNA is a system function provided as part of APL2 (Version | Release 2 and later). designed to allow routines coded in languages other than APL and stored outside the workspace to be invoked in exactly the same way as APL functions. The functions are invoked by use of an associated processor, one for each language type being used. In the CMS environment, two associated processors are provided; 10 allows use of the REXX language, 11 allows languages compiled into machine code to be called. TSO provides only processor 11, since there is no equivalent to the REXX language. 3. Properties of Associated Objects One of the key properties of associated objects is that they obey all the rules associated with normal  APL2 objects: m They appear in{JNL, )NMS etc. @ ONC returns the correct name class for the object. = Associated functions and operators can generate APL2 errors. m The association is preserved in a )SAVEd workspace. = Associations can be )COPYed from another workspace. !" Associations are dissolved by JERASE or EX. m (AT returns the correct information (even a timestamp can be defined.) An associated function appears identical to a locked function as far application programmer is concerned. 4, Interface Definition The interface between APL2 and the associated object is defined in a names  file. Amongst other factors, the names file contains descriptions of:

123

VECTOR Vol.3 No.4

The name of the associated function. The load library where the function can be found. The module name and entry point if different from the associated name. Interface type (object, fortran or function). Descriptions of the arguments, including structure and data types if these can be anticipated. @ A description of the function and time-stamp. The most complex feature to be defined is the pattern of the arguments. The pattern definition must include details of structure  that is the pattern of nesting in the argument. Details of rank, shape and acceptable domain must also be included. To facilitate the implementation of assembler coded functions, translation options can also be specified. These will cause an argument to be translated from APL format to S/370 on input to the function and the result to be re-translated on output. 5. Supplied Functions With APL2 several functions coded in assembler are provided. The lack of documentation makes these more difficult to use, several of are, however, extremely useful. Several of the functions allow a vector to be partitioned in various ways (see also my letter in VECTOR Vol.3, No.3). These include: CAN - partition on zeroes in a boolean vector. DAN partition on delimiters, removing the delimiters. SAN - partition on delimiters, retaining the delimiters. Other functions are more esoteric such as SVI which allows a systems programmer to investigate which associated processors are functioning. The functions used for converting data between APL and 8/370 formats and back again are RTA (record-to-array) and ATR (array-to-record). 6. RTA and ATR functions

These two functions translate S/370 data to APL formats and back again respectively. The left argument takes the form of a pattern specification of the data to be translated. The pattern specification is similar used to describe the left and right arguments in the associated functions name file definition.

The RTA and ATR functions are capable of translating an entire record, with fields of varying lengths and types at one call. The cover functions presented here are more limited, they are designed to replace the IBM functions in 1 UTILITY. Each function translates any number of occurrences of a given data type into APL format (or vice versa). For example, when read from file, a series of 6 byte packed fields will each be represented as 6 APL characters from [JAV. In this form, the data cannot be used by APL. Using the PDI (packed decimal input) function the 6 character representation is converted into valid APL numerics. Supplying an N by 6 array of packed decimal data will result numeric vector of length N. 124

VECTOR Vol.3 No.4

7. Pattern Arguments  The key factor in performing the data translation is the creation of the pattern argument. The pattern argument is of two separate forms, depending on the direction of the translation. When translating from $/370 into APL, the input data is always supplied to the RTA function as a vector, The rank and shape of the output is implied by the rank and shape of the argument to the cover function. The first part of the pattern argument defines the translation option, The option is in two parts, the data type (e.g. P  for packed, I  for integer) and the number of bytes per field (*P8’ will translate 8 byte packed decimal data). The next part specifies the rank (always 1) and length of the resulting APL vector. The resulting numeric vector is reshaped into an array that conforms with the input argument. When translating from APL to S/370, the input data is supplied as an array, the result is always a character vector. In order to be the inverse of the input  cover function, this vector is reshaped. The pattern argument is created slightly differently. The first item is an asterisk (*) stating that the count of items to be translated is unknown. The next item describes the translation option (as above). The last item is another asterisk, informing that the rank of the array being translated is variable.

8. Efficiency

The main aim of using the assembler-coded functions is to gain performance and execution speed. The following benchmarks show just how effective the assembler-coded functions are relative to the APL equivalents. The tests are performed by averaging over 1000 conversions of a specified array.

For the BIT conversion functions, 256 bytes of data are converted each time. For the PACKED and INTEGER conversions, integer arrays of 24 rows by 10 columns are converted. Both negative and positive data are included. Floating point conversion uses a 24 by 10 array of floating point numbers, of both negative and positive sign.

Tt

Converting from APL to $/370 (CPU milli-seconds per array conversion) Type DNa/aPL (2) Rank Logical (Bic) 2.68 20.13 13.31 2 Packed Decimal 3.48 23.28 14.95 3 Integer 2,58 11.39 22.65 4 Floating 2.56 210.58 1,22 1 Average 2.83 66.35 4.27

125

VECTOR Vol.3 No.4

Table 2

Converting from 8/370 to APL (CPU milli-seconde per array conversion) Type Ona APL QNA/APL (2%) Rank Logical (Bit) 2.23 20.52 10.87 4 Packed Decimal 3.68 72.88 5.05 3 Integer 2.09 101.35 2.06 1 Floating 77.24 3.24 2 Average 2.74 68.22 4.02

9. Function Changes The functions listed below are as identical as possible to those supplied in 1 UTILITY. A few changes forced by restrictions in the ATR and RTA functions:

= The IJ and IO functions will now only accept half or full word integers. m The FO function is capable of producing single precision as well as double precision floating point output. The integer processing restrictions imposed are not very serious. Nearly all integer fields are either half- or full-word, Additionally, the performance of the II/IO functions in ! UTILITY is so poor with integers greater bytes, that processing any volume of data is made impracticable. The FI function in 1 UTILITY is capable of converting full or double precision floating point into APL numerics. The FO function only produces double precision output. Single precision can be produced by dropping the last four columns of the output’ (quote from comments in the function). This technique is obviously inefficient. By default, the FO cover function will produce double precision output. The required precision can be specified in the left argument.

10. Conclusions

The requirement to translate data between $/370 and APL formats arises quite frequently in the commercial data processing environment. The use of assembler coded functions to speed up such translations makes the processing of significant volumes of data a realistic proposition. The functions given are intended to cover the the IBM supplied functions in 1 UTILITY. To this end, their syntax and functionality is mirrored as closely as possible. The RTA and ATR functions could easily be utilised in a different manner which would further facilitate data conversion work. The documentation of the supplied assembler coded functions is very poor. Improvements in this area would make use of the supplied functions much easier.

126

VECTOR

11. Sample Functions Caly 0} 1) 2] 3) 4) 3) 6) 7] 8]

10) FI.3 pr 1) 1987-02-19 13.04.32 R+FI DA;ND;O10 Vol.3 No.4

a FN: Convert from $/370 floating point to APL? numeric a DA ~ Character array aR ~ Numeric array Oro-t QEs(~(sDA)eO 19/5 4 DES(~(NDe" It, pDAdes 83/5 4 +€30 11 DAT 'RTA'}/ok Qes 1 2 OksRe'CES 9 5555559) teNDs1.x/14pDA R+("1l#pDA)pR RTA,DA

FO.3 2 13 1986-10-22 14.45.44 ReLE FO DA;DIO $/370 Floating point data APL2 equivalent data

Error not simple. Full or double words only Create/check association Unable to associate Conversion pattern Convert and form array aaraa>

FN: Convert APLZ numerics to $/370 floating point (ful1/double word}

Number of bytes per element APL2 data to convert $/370 representation

Dyadic call a Extension: Function will allow full word floating point A LE - Numeric scalar @ DA - Numeric array A.R - Character array O1o+1 +(2=0NC tLE')/dy LE+8 dy: DES(~(LEcS B)AORZLE*+*'pLE)/5 & (3 11 ONA SATR')/ok Des 1 2 ok:Re((pDA)LE)p(t(* ES *)'WLEJATR DA

IX.3 1987-02-19 13.05.27 ReIT DA;ND; O10 Default double-word LE scalar and LEe4 8 Check/create association Unable to associate Convert and form array

A FN: Convert $/370 integer data to APL integer array a Restriction! accept 2 or 4 byte a DA - Character array - Numeric array

=DA}€0 1)/5 4 CES(~(nD+"1t1,pDA)e2 4)/5 4 (3 21 DONA *RTA‘)/ok Ors t 2 ok Re (15 9 5555559) UND, x/ 1ltpDA R+CLépDA)pR RTA,DA

WO.3 pt 12 1986-10-22 14.45.44 ReLE 10 DA;Q10

Restriction: Output is LE - Numeric scalar DA - Numeric array R - Character array O1o+1 DES(~(LEe2 4)AQ=ELE*'*pLE)/5 & DES(v/.DAZLDA)/5 4 +(3-11 ONA 'ATR')/ok Des 12 oki R*((pDA},LE) po (t(« length

apevan

15 *)'¥LE)ATR DA

127 integers. Integers to be converted APL integer array Error

not simple Error not 1-7 bytes Create/check association Unable to associate Conversion pattern Convert and form array aaara>

FN: Convert APL2 integers to $/370 representation of integers limited to 2 or 4 byte integers Bytes per output integer Data to be converted 8/370 representation

LE scalar and LEe2 & Error, not integer Create/check association Unable to associate Convert and form array

Vol.3 No.4

Bit data converted ABL boolean array

Not simple Create/check association Unable to associate Conversion patterao Convert and form arcay pear

$/370 bit data. Data converted Output $/370 bit data

Not a simple array Check last dimension Create/check association Unable to associate Conversion pattern Convert and form array ppa2are

to APL numerics Packed data to be converted APL numeric data

8 Error not simple. A Error not I-16 bytes. a Create/check association a Unable to associate fA Conversion pattern a Convert and form array

packed decimal format Number of bytes per element Data array to be converted Packed decimal representation

LE scalar and 1SLES16 DA must be integer Create/check association Unable to associate VECTOR

Lalv L1.3 10 1986-10-22 14.45.44 Col R+LI DA;Q10 [1] A FN: Convert system/370 bit date into APLZ boolean array [2] a DA - Character array (33 AR ~ Boolean array C4] Dro+t £5] Des(~(spaje0 19/5 4 C6] +(3 11 ONA RTA) /ok 7] Ges 12 £8] ok: R+'(BS 9 5555559}'¥1 1,8xx/pDA C9] Re (C714¢pDA), 8x1 tpDA}pR RTALDA

Ca]v LO.3 ps Ll 1986-10-22 14.45.84 £ 0] R+LO DA;0I0 C3] # EN: Convert from APL2 boolean array to { 2] a DA - Boolean array C3] aR - Character array C4] £ 5] DA}e0 1)/5 4 £ 62 DEs(ReLR«(7 1th pDA}AB)/5 & 7) (3 11 DNA PATR')/ok { 8] Mes 12 C Fr ok:R+*(B5 9 5555559)" ¥I Lex/pDA a Re(C7LépDA).(71tpDA)#8)oR ATRYDA

{a]v PDI.3 pz 11 1987-62-19 13.04.36 0] R+PDI DA;ND; O10 [ i} 8 FN: Convert 6/370 packed decimal format C 2] a DA - Character atray C3] aR ~- Numeric array Cc 4] O1o+) C 5] DEs(~(=DA)eO 13/5 4 { 6] DES(~(ND+"1t]pDA)erl6)/5 4 C7) +(3 11 ONA RTA')/ok C 8] Qes 12 £91 okrR+*(P35 9 S555559) "ND, L.x/ b4pDA Clo] R+(~1+pDA)pR RTAsDA

Cale Pro.3 pz: 1} 2986-10-22 14.45.44 Co] R+LE PDO DA;QIO [C1]. FN: Convert AFL numeric array to S/370 { 2] a LE - Numeric scalar C 3) a DA - Integet array C 4} aR - Character array (5) Oro+1 C6] DES(~(LEc116)A0=sLE*? pLE)/5 4 C7 Des(v/,DAzlDA)/5 4 C8 *(3 11 ONA 'ATR')/ok C9] Ges 12 [10] ok: R+((pDA),LE}p(*(+ B35 *)*¥LE)ATR DA

128 Convert and form array