Using quad-NA to Eliminate WS FULL
Vector 4:4, page 107
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 107
Introduction¶
Much has already been written about using quad-NA to speed up APL2 function code. A lesser known (indeed undocumented) series of recent enhancements have been implemented which allow workspace size problems to be tackled using quad-NA. These enhancements are unsupported in the current release of APL2 (APL2 V1.R2).
Using Quad-NA¶
The Quad-NA system function is used to establish linkage with an object outside the APL2 workspace. Typically this object is a function coded in a language other than APL2. The right argument gives the function name with which linkage is to be established. The left argument specifies the name class and attached processor through which the linkage is made. The result is a boolean scalar, 1 if the association is established successfully, 0 otherwise.
The clue to the newly implemented features lies in the left argument. By changing the specified name class, objects other than functions can be associated with:
It must be emphasised at this point that the associated object takes only a minimal amount of workspace. When an association is formed with a function or operator, only about 12 bytes of workspace are used. This memory overhead is used to establish pointers to the correct code outside the workspace.
Saving Workspace¶
The largest objects in a given workspace are normally the variables to be used during processing. Clearly, opportunities for saving workspace will be maximised by storing variables outside the workspace, then associating with them as required:
⎕WA ⍝ Current work area
1987103
FIRST ⍝ Reference non-existant variable
VALUE ERROR
FIRST
^
2 4 ⎕NA 'FIRST' ⍝ Establish association
1
⍴FIRST ⍝ What is its shape
200 80 100
×/⍴FIRST ⍝ Number of items
1600000
⎕WA ⍝ Work area with variable
1987091
Note that in the above example, only 12 bytes of work area are used to store an array containing 1.6 million items.
Just as in the examples above with associated function, the associated variable acts exactly as a normal APL2 variable. The only difference is that the variable is stored externally to the workspace. Associated arrays can of course be nested:
Existing variables can be placed external to the workspace simply by associating the variable with the associated processor after it has been assigned. Once associated, any assignment to the variable updates the associated version rather than re-creating a copy in the workspace.
⎕WA
1987091
SECOND←(10 8 6⍴480?1000)(15 5⍴'ABCDE')
⎕WA
1985056
2 4 ⎕NA 'SECOND' ⍝ Externalise the array
1
⎕WA
1987079
The space saving that can be achieved using associated variables is obviously tremendous. The enhancement to Quad-NA which implements this facility is referred to as the Fully Integrated Reserved Storage Table (FIRST).
Saving More Workspace¶
Now that functions, operators and variables can be placed external to the workspace and accessed via name association, the only memory usage within the workspace is that required by the interpreter. In order to minimise memory usage, quad-NA has been further enhanced to allow the storage used by the interpreter to be stored externally also. These areas of storage are then accessed via name association.
Note the use of a ‘dummy’ name and name class on the call to quad-NA. The use of the name APL2 causes the entire interpreter storage to be removed from the workspace. Using other names allows different portions of the interpreter to be removed:
¯1 ¯4 ⎕NA 'APL2' ⍝ Get the interpreter storage back
1
⎕WA
2074824
1 4 ⎕NA '⎕FNS' ⍝ Only system functions moved
1
⎕WA
2215580
Note the use of the negative processor number in the above example. This is used to dissolve the name association, and relocate the interpreter storage back within the workspace.
The enhancement to Quad-NA which allows the interpreter storage to be moved out of the workspace is referred to as Associated Processor Remote Interpreter Linkage (APRIL).
Using No Workspace At All¶
Once all the defined objects have been removed from the workspace, and the interpreter storage is also relocated, the only remaining use of memory in the workspace is for the associated object definitions (12 bytes per object).
A third enhancement to quad-NA allows even these headers to be moved out of the workspace. Referred to as Full Operation Off-Line (FOOL), this allows your APL systems to run without using any memory at all.
Conclusions¶
The three enhancements to quad-NA described (FIRST, APRIL and FOOL) allow applications to be written with only very small memory overheads. The ability to store variables outside the workspace reduces the required workspace for a given application.
Memory usage is reduced still further when the interpreter is stored outside the workspace. If the quad-NA headers are stored outside the workspace, memory usage vanishes in a SPOOF of smoke.