Working with Compressed Data
Vector 14:4, page 117
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 117
I developed a Dyalog operator for working with compressed data. I am using a simple Boolean mask to compress out zeros from sparse data. The following function will produce a compressed data structure from a simple numeric vector:
∇ r←Compress vec;bo;values
[1] ⍝ Return vector <vec> in a compressed form
[2] ⍝ vec≡(1⊃r)\2⊃r
[3] bo←vec≠0
[4] values←bo/vec
[5] r←bo values
∇
The operator is:
∇ z←la(fn Compressed)ra;lab;lad;rab;rad;zab;bo
[1] ⍝Operator to use dyadic fns on compressed data
[2] lab lad←la ⍝ Split LHarg
[3] la←' ' ⍝ Save space
[4] rab rad←ra ⍝ Split RHarg
[5] ra←' ' ⍝ Save Space
[6] zab←lab∨rab ⍝ Create Boolean for result
[7] lad\⍨←zab/lab ⍝ Expand LH data to shape of result
[8] rad\⍨←zab/rab ⍝ Expand RH data to shape of result
[9] lad fn←rad ⍝ Apply function
[10] ⍝ Note the following is required if zero data
[11] ⍝ is to be treated differently from "missing data"
[12] :If ~∧/bo←lad≠0 ⍝ Check for zeros in result
[13] lad/⍨←bo ⍝ Compress out zeros from data
[14] zab∧←zab\bo ⍝ Adjust compression Boolean
[15] :End
[16] z←zab lad ⍝ Return compressed result
∇
Lines 7 and 8 could be re-written in ISO standard APL as