Hackers’ Corner: Six Megabytes in 640K?
Vector 14:4, page 112
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 112
This is unfinished work. I expect you will have to do some experimenting yourself if you need to use these techniques. I would be glad to hear from anyone who has solved more of the problems than I have.
I maintain a membership records system written in APL*PLUS/PC version 8.1. The organisation is growing fast and we have had problems with WS FULL, because it searches for a target name in a matrix of all names beginning with the target letter. Would you believe that there are more names beginning with M than any other letter? Anyway the size of the M matrix is about 42K, and it is appended to, which explains why my experiments are done with variables that take up 64K. I want headroom!
The author of the system had also had WS FULL problems and introduced a virtual workspace in an attempt to get over them. Because he only had 2 Mbytes of memory and was using a RAM disk to hold the virtual memory he had not explored the limits of the technique.
Although the system is running under Windows 95 it is necessary to have the DOS CONFIG.SYS and AUTOEXEC.BAT set up correctly or you don’t get the APL characters. When running under DOS it used to be possible to run VGAFONT.COM but this doesn’t work under Windows 95 so I have to resort to the codepage method. Here are my config.sys and autoexec.bat.
Config.sys¶
device=C:\WINDOWS\Himem.Sys
DOS=HIGH,UMB
devicehigh=emm386 noems
installhigh=c:\windows\command\nlsfunc.exe
devicehigh=c:\windows\command\ansi.sys
device=c:\windows\COMMAND\display.sys con=(ega,,1)
Country=044,850,c:\windows\COMMAND\country.sys
devicehigh=c:\windows\ramdrive.sys 76800 /E
The aim is to use as little of the base 640K memory as possible; even though Windows provides the DOS memory by emulation (so it needn’t really steal TSR stuff out of the 640K) it seems to emulate the DOS memory limitations too! The key items here are nlsfunc.exe, display.sys and the country line which are all essential for the APL font. The APL font supplied with APL*PLUS is an ega-compatible font which explains the display.sys setting.
Autoexec.bat¶
C:\PROGRA~1\MCAFEE\VIRUSS~1\SCANPM C:\
@IF ERRORLEVEL 1 PAUSE
SET TEMP=C:\WINDOWS\TEMP
SET PROMPT=$p$g
SET winbootdir=C:\WINDOWS
SET PATH=c:\windows;c:\windows\COMMAND;d:\utils
mode con codepage prepare=((850) c:\netapm\aplfont.cpi)
mode con codepage select=850
keyb uk,,c:\windows\COMMAND\keyboard.sys
The important lines here are the two lines beginning ‘mode’. They provide the APL font. It is essential that the properties of the batch program that starts APL are set to run FULL SCREEN. If you run in a window you won’t get the APL font.
APL.bat¶
REM APL.BAT SETS UP APL*PLUS 8 FOR NETWORK
rem amended Feb 1998 by AJC
C:
cd\
CD\NETAPM
lh aplprint
REM A+1 retain video attributes when scrolled off screen
REM | C=0 no comms buffer
REM | | F=40 file tie quota (probably not needed under Win 95)
REM | | | K=U unified keyboard
REM | | | | LIBS=C:\NETAPM where to look for APLLIBS.ASF
REM | | | | | M=2000 specifies top of memory (probably not needed)
REM | | | | | | S=16 sets scrolling screen memory to 16 K bytes
REM | | | | | | | W=APM2 workspace to load
REM | | | | | | | | VWS=&c in 4Kb granules allocate
REM | | | | | | | | | 5.938 Mb of virtual WS and
REM | | | | | | | | | swap out anything bigger
REM | | | | | | | | | than 32 bytes
REM | | | | | M=2000 | | |
APL A=1 C=0 F=40 K=U LIBS=C:\NETAPM S=16 W=apm2 VWS=f:\APL.VWS,1,5938,32
c:
CD\
I hope the the file is clear enough not to need explanation, except for the VWS parameter. VWS= sets the virtual memory options. First is the file ID and path, where the virtual workspace is to be held, second is the granule size in KB (which defaults to 1), third is the amount of disk space to reserve in KB (5938 is the maximum value and the manual says ‘*’ would also reserve the maximum value). 32 is the size in bytes of the smallest object that will be written to the virtual workspace when space is short. According to the manual 32 is the smallest permitted value.
When APL.BAT is double clicked the system should start and, if you have its properties set to full screen you will be able to type quad AV and see all the APL characters. Note that you cannot maximise the window to get to full screen. There may also be unexpected consequences of the screen driver; on my portable, which has an 800 by 600 pixel screen, the APL full screen comes up in a 640 by 480 patch in the centre and I guess this is because the portable cannot switch screen modes to vary the size of the pixels.
Now I have a workspace called HUGE. It contains so little that you will not suffer much by typing it in. There are two functions and a single variable. Here they are:
∇ ∆ ADD A;∆∆;∆∆∆
[1] ⍝ Adds or replaces variables ∆+A-Z with contents A then next letter
after ∆, and so on.
[2] ∆∆∆←¯1
[3] NXT:
[4] →(25<∆∆∆←∆∆∆+1)/END2
[5] ∆∆←⎕AV[⎕IO+65+∆∆∆]
[6] ⍎∆,∆∆,'←A'
[7] '⎕WA is ',⍕⎕WA
[8] SPACE
[9] →(1024<⎕WA)/NXT
[10] VWSINFO ⋄ (∆,∆∆),' added.',⎕TCNL
[11] END:'Please press enter or X enter to end.' ⋄ →(~'X'∊⍞)/NXT
[12] END2:'END2 of ADD reached. ',(∆,∆∆),' last variable created.'
[13] ∆∆∆←¯1 ⋄ ∆←⎕AV[1+⎕AV⍳∆] ⋄ →NXT
∇
The right argument started out as a 32750 reshape of a character string. I think anything will do. I believe that some interpreters keep only one copy of variables (e.g. a←b←c←d←65536⍴32↑⎕AV) which are exactly similar, making separate items only when one is changed. I don’t believe that APL*PLUS/PC is one of these. The size was chosen to make the quad-size of the variable 32768.
SPACE is necessary because ⎕WA returns the workspace available in the real memory, and if variables and functions are swapped out, their size is not included.
∇ SPACE
[1] ⍝ Reports the whole space taken by this WS
[2] 'The size of the workspace (⎕WA plus ⎕SIZE of VARS + FNS) is:
',⍕(+/⎕SIZE ⎕NL 2)+(+/⎕SIZE ⎕NL 3)+⎕WA
∇
As you can see ‘A’ADD A simply creates copies of A with names ‘AA’, ‘AB’, ‘AC’ and so on, running SPACE and ⎕WA after each to tell you how things are going. The reason for lines 10 and 11 is that I discovered that line 9 sometimes drops through when ⎕WA returns less than 1024 and it may do this well before the virtual workspace is full.
VWSINFO on line 10 is a function from the supplied workspace VIRTMODE and it returns the granule size in Kb, the maximum size of the virtual workspace in Kb, the number of granules, the number of granules free and the minimum size of object that would be swapped out. The VIRTMODE workspace contains several other useful functions, including some to prevent an object from being swapped or to reduce the likelihood of a swap. This dates from the time when PCs were slow. The variables to fill the six megabytes were created and swapped out on the test machine (a Dell D266) in under three seconds!
The value ⎕WA returns seems to vary and I don’t know why. I haven’t found any programmable way of forcing the garbage collection to run or forcing the system to swap anything out into the virtual workspace.
There is a ⎕POKE (121) which, according to the manual, forces a garbage collection in the workspace every time the keyboard unlocks for input. I hoped that it would work without actually insisting on a formal request for input, but it doesn’t seem to. Perhaps someone who provided support on this system when it was state-of-the-art will be able to tell us how to force the system to make space.
The system stops with WS FULL when the space used is over six megabytes. Here is a screen dump at the end of a run:
⎕WA is 30176
The size of the workspace (⎕WA plus ⎕SIZE of VARS + FNS) is: 6074432
END2 of ADD reached. GZ last variable created.
VWS FULL
ADD[6] HA←A
^
SPACE
The size of the workspace (⎕WA plus ⎕SIZE of VARS + FNS) is: 6107152
VWSINFO
1 5938 5938 19 32
⎕WA
62944
Y←16376↑A
'J'ADD Y
⎕WA is 46464
The size of the workspace (⎕WA plus ⎕SIZE of VARS + FNS) is: 6107152
VWS FULL
ADD[6] JB←A
^
VWSINFO
1 5938 5938 17 32
⎕WA
63104
2 HUGE Unified Caps Ins
As you see I tried reducing the size of the variable A to 16376 (to make it occupy 16384 bytes) but it only added ‘JA’ before the workspace was full again. I would have expected that with 17 granules not used and 63Kb of workspace available at least one or two more variables would be possible. Does anyone know why it behaves like this?
Erratum: Windows NT Registry Access¶
I tried (with some enthusiasm but little success) to use the John Sullivan’s code recently published in Vector. There are perhaps other APLers who experienced the same problem as I did, i.e.
- Reg.Open would happily create a new key and return a sensible handle, but
- it would fail to open an existing key, returning a handle of 0.
Those readers with the same experience will probably be users of Windows NT, as I am. The answer lies in the access variable. In the original code (Dyadic Systems’s function GetRegKeyHandle) this was assigned:
I quizzed them about this, and they admitted that this value was incorrect. It works fine on operating systems with inadequate registry security (e.g. Windoze 95 - in fact any old value would probably work) but not with NT. The correct statement should be:
With this adjustment, John’s code seems to work flawlessly. I hope this assists other confused readers.
Bob Hoekstra