APL in a Client-Server Environment BAA Workshop report

Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 62

Background¶

This year the BAA decided to try to do something a bit different with its program of events. The Friday afternoon talks were under-attended and often uninspiring events — the highlight being the tea-break when you could meet old friends and chat about current trends! The message coming through was that people were desperate to find out how to build Windows applications quickly and how to take advantage of the various nuts and bolts that the Windows operating system provides.

The committee decided, after good results/feedback from the BAA-GUI workshop a year and a half ago and the workshops at Swansea, that our time/money was better spent running a series of one-day workshops through 1995. The first workshop was held at City University in January. Unfortunately Duncan Pearson had to drop out due to work commitments, but thanks to Morten Kromberg for coming over from Denmark and Adrian Smith for covering for Duncan. Also thanks to Duncan for lending me his notes — the majority of which are reported below.

DLLs¶

DLLs are Dynamic Link Libraries. These are libraries of utility C-code on which most of the Windows Operating System is dependent — but are made available for any program to call. To a C programmer calling these functions is simple and analogous to using any other library of C functions. APL has had to come up with new low-level quad functions to interface with these libraries. The two main commercial PC/Windows APLs have come up with differing methods of calling these DLLs.

One of the most difficult things about DLLs even before you have worked out how to call one from your APL is finding out whether a DLL exists for what you want to do and then finding out the arguments required to call that DLL. The best place to start is the Microsoft Software Development Kit (SDK) and the associated help file which comes bundled with lots of Windows development tools. DLLs are documented using C Syntax and their arguments are therefore C datatypes (e.g. int char struct etc.). Many of these datatypes are defined in the header file of the library you are using or windows.h (e.g. LPCSTR UINT etc.) Hence, before you can even begin to call a DLL you need a reasonable knowledge of C and the Windows environment.

Once you’ve got to grips with some of this you’ll want to call one of these from APL. I’ll explain how to do this in Dyalog first — because it teaches you more about how the interface works. PLUS III avoids a lot of the tricky stuff — but as a result hides what’s really going on.

In Dyalog you must define the function (via ⎕na) you are about to call by declaring the C datatypes you are about to pass to it. Dyalog is then able to translate your APL arrays into C datatypes and successfully call the function

{aplfnname} ⎕na 'returntype dllname.P16|cfnname argtype1 argtype2 ...'

P16 is the calling structure — Pascal 16 bit — most Windows 3.1 DLLs execute in this mode.

Once defined you can call it like a normal function with APL variable arguments (r←aplfnname arg1 arg2 ...). Dyalog converts your arguments into the C data types that you defined for the function and then calls the C function with these arguments. If you define an argument incorrectly you will almost certainly find the function generates a General Protection Fault (GPF) when it is run or it returns junk information. This can be very frustrating!

Dyalog uses a special syntax to map APL variables onto C datatypes — the following table gives some common examples:

Examples¶

Opening, Closing and Reading files — Windows style! Although APL provides its own native file handling we can also use the Windows file-tie and read utilities.

_lopen is the Windows equivalent to ⎕FTIE. In the SDK documentation _lopen takes two arguments — a filename (pointer to a null terminated string — thus we declare it as <0T) and a mode in which to open it (int — I) — it also returns a file handle (int — I). To run the function we must use ⎕NA to associate the DLL function/library with an APL function name. Thus:

⎕NA'I kernel.P16|_lopen <0T I'

creates a function called _lopen (it defaults to the C function name if no left argument is supplied to ⎕NA). The SDK documentation tells us that the open modes are READ, READWRITE and WRITE — these are predefined C constants — unfortunately we must use the real integer values of these constants and hence look in the windows.h to find out what these constants really are in this implementation of Windows (presumably its done this way for portability of code) The value of READ is 0. Hence:

fh←_lopen 'c:\windows\system.ini' 0

gets us a read-only file handle. To read the file we need to find out how big it is first because we can only read 64K bytes at a time and hence we are going to have to loop if it’s bigger than that limit. The function _llseek moves a pointer through a file and we can use this to discover the size of the file. The SDK tells us it takes three arguments — a file handle (int-I) — a number of bytes to move (long-U4) and a position to move from (int-I) and it returns the new offset in bytes of the pointer from the beginning of the file (long-U4) — thus:

⎕NA'U4 kernel.P16|_llseek I U4 I'

associates the function and the SDK also tells us the possible values for the position to move from are 0,1 and 2. A 2 means that the position to move the pointer from is the end of the file. A 0 means move the pointer from the beginning of the file. (A 1 is move it from wherever it currently is), so:

size←_llseek fh 0 2
sink←_llseek fh 0 0

moves the pointer 0 bytes from the end of the file (i.e. to the end of the file!) and hence returns the size of the file. We then stick the pointer back to the beginning of the file which is where we want to start reading from. As long as size is less than 64K we can read the file in one glob — we will assume it was for simplicity! We can now read the file in.

_lread is the function which the SDK tells us to use to read files and takes three arguments — a file handle (int-I) — an address of a buffer to put the data in (huge*->U1[]) — and the number of bytes to read (uint-U2). It returns the number of bytes successfully read in (uint — U2) …

⎕NA'U4 kernel.P16|_lread I >U1[] U2'
ct bytes←_lread fh size size

… reads from the file fh and puts the result in the variable bytes — the number of bytes read is ct which in this case will be the same as size because we checked how big it was first. Notice how Dyalog handles arrays which are returned by the DLL. You simply pass the required size of the array (so Dyalog can create a buffer the correct size) and it then appears as an extra result of the function so that you can assign it into a variable. PLUS III handles this differently — I will show how later.

As in APL we have to free the file-handle once we’ve finished with it. _lclose does this and takes just a file-handle as its argument and returns 0 or an error code if unsuccessful:

⎕NA'I kernel.P16|_lclose I'
Done:sink←_lclose fh

That’s all there is to it! (How many Windows SDK programmers does it take to change a light bulb? ….. 1 to write GetBulbHandle, 1 to write GetBulbContext, 1 to write GetBulbStatus, ……. 1 to write FreeBulbHandle ………. etc.)

As I mentioned earlier the real problem with this stuff is not writing it but knowing/finding which functions to call or whether a function even exists for what you want. For example when faced with deleting a file I immediately looked in the SDK under D* but found nothing, then _l*, still nothing — finally I looked for anything with File in it and found OpenFile. OpenFile’s 3rd argument is a mode, one of which is OF_DELETE (512) which deletes the file! Hence:

⎕NA'I kernel.P16|OpenFile <0T ={U1 U1 U U1[4] T[128]} U'
file←⊃OpenFile 'c:\autoexec.bak'(136 0 0(4⍴0)(128↑' '))512

deletes the file and returns a file handle. The structure is for the file details and is filled with information about the file — we are probably not interested in this if we have just deleted it!

Structures¶

In the previous example the 2nd argument of Openfile was a pointer to C-structure which was both passed to the function and returned. This was represented by ={*}. Where the * is the individual elements of the structure. In Dyalog’s current implementation of this you cannot nest the structures, i.e. if a structure contains a pointer to another structure, i.e. you can’t do {*{*}*}. This means that you cannot call some of the DLLs in Dyalog. (You actually can call them but you have to allocate your own memory and pass your own pointers as I4s. This is probably beyond most mortal APLers.). Once again the PLUS III implementation gets around this…

Calling DLLs from PLUS III¶

Manugistics have a much more advanced interface to DLLs than Dyalog. As well as supporting ⎕NA they have an interface called ⎕wcall which gives you immediate access to all the more common Windows DLLs. So instead of spending half your development time with your head buried in the SDK manual you just have to know the function exists! For example:

⎕wcall '_lread fh size size'

would read a file of size size (already opened with _lopen etc.). They achieve this by defining the association of a DLL call in the aplw.adf. All the common calls have been pre-defined in this file and also you can add any new ones (you have to map all the datatypes as in Dyalog and then recompile the .adf file) and also document all the calls — so if you forget the arguments you can query the call, and it will return the documented calling syntax. Also in the .adf you can define not only function calls but you can also define C structures and other derived data types. These mappings can then be used by the function mappings themselves and allow you to define functions which refer to nested structures etc., hence overcoming the problem that Dyalog ⎕NA has.

⎕wcall also has the benefit of handling output arrays in a slightly more consistent manner. Rather than using Dyalog’s approach of having to pass the size of the output array and then returning the result as an extra returned element, PLUS III requires you to create a variable of the right size and returns this variable updated.

Not having a PLUS III interpreter/documentation this is as much as I can tell you. I’m sure that Vector readers would be very interested in reading an article that describes the PLUS III .adf in detail and how to add new definitions to it and also how to define structures, if there’s anyone out there who can write it — please do!

Common Problems Calling DLLs in Dyalog¶

The workshop highlighted some of the problems people have when trying to call DLLs from Dyalog. The exercise was to use WritePrivateProfileString to create an ini file called "dlltest.ini" in the windows directory with a section "[User]" and an entry "name" with the value of someone’s name, for example:

[User]
name=jon sandles

The function just takes four pointers to strings as arguments:

'wpps'⎕NA'I kernel.P16|WritePrivateProfileString <0T <0T <0T <0T'

Most people at the workshop managed this quite quickly. A few got the <0T confused with >0T which defines OK when ⎕NA’d but gives a domain error when called with the required arguments:

      wpps 'User' 'name' 'jon sandles' 'dlltest'
DOMAIN ERROR
      wpps'User' 'name' 'jon sandles' 'dlltest'

Because it was defined (incorrectly) as a >0T the interpreter returns a DOMAIN ERROR because it is expecting a buffer size for the returned strings, hence:

      wpps 0 0 0 0
0 K1ör

works fine — but returns 0 (for failure) and gobbledegook in the four null-terminated strings. This illustrates two possible areas for confusion.

1. If you define the data mapping incorrectly — the ⎕NA may well still work but when the correct arguments are passed it gives a DOMAIN ERROR (or maybe even GPFs).
2. If you define the data mapping incorrectly and pass the wrong arguments (that match the incorrect mapping) the function runs and returns data — but probably gobbledegook. This situation can be very tricky to debug!

A typical occurrence of (2) is when defining the mapping of integer datatypes — if you get the defined length of the integer wrong (double, long, unsigned?) then the results can be very confusing. As we have already seen the correct definition of _lread is:

⎕NA'U4 kernel.P16|_lread I >U1[] U2'

Run correctly it returns (fh is a file handle tied to c:\windows\win.ini):

      2⊃_lread fh 10 10
91 119 105 110 100 111 119 115 93 13
      asc[1+91 119 105 110 100 111 119 115 93 13]
[windows]

But if defined erroneously as (U2 rather than U1):

      ⎕NA'U4 kernel.P16|_lread I >U2[] U2'
      2⊃_lread fh 10 10
30555 28265 28516 29559 3421 30556 28265 26926 26990 0

you get data that is far too large to be ASCII! This problem can also be very tricky to debug. Of course to C programmers this probably all seems quite obvious — but to an APLer who typically only has to distinguish between integers and characters this can get most frustrating. A similar problem is when you define datatypes incorrectly so that the memory they take up is not as large as the function requires — the chances are the function will then write all over unreserved memory and the program will GPF.

Hopefully Dyalog can learn from PLUS III and provide a way of pre-defining the more common calls. One approach might be a utility workspace, allowing you to copy them in and learn from them if you need to add your own.

Overall¶

Apologies to people who found the overhead projection equipment inadequate — next time we will make sure the presentation facilities are better — also apologies to those who were expecting more on the DDE side — in the end we had very little prepared on the topic — but we hope to cover it very thoroughly in the next workshop. Well done to all those who completed all the exercises!

We learnt a lot from the day and the Committee will be laying down guidelines for events in the future. We need places with good PC facilities and presentation equipment; if anyone has any suggestions please let me know. We would be particularly interested in facilities in the Birmingham area.

We have also received details about the BCS’s IDPM Continuous Professional Development Programme. For a small overhead we could register our workshops for you to gain points towards your own development. If anyone is interested in gaining IDPM credits by attending APL workshops, then please contact me (see inside back cover for address details).