Interfacing APL to C
Vector 5:1, page 49
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 49
Why Interface?¶
At first sight, you may see no need to interface APL with other languages. APL is powerful and expressive, and modern implementations are very efficient. Fast hardware is affordable, and interpreters like the new APL*PLUS II system for the 80386 give the power of a dedicated mainframe to top-end PC users.
You may still want to consider using other languages with APL, however. There are several reasons for this; one of the most common is that your application includes an algorithm that is iterative or that works on scalar values. (This often happens when you are trying to optimise something, or simulating a complex system.) You can often see in advance of coding it that an APL version of that algorithm would run too slowly, but you want to write the rest of your application in APL. If you are a mainframe APL user, you might write the relevant code in APL and then compile it; on a PC, you would need to write it in a compiled (or assembled) language which you can invoke from APL.
A second need arises when your application is written, but working too slowly. A monitoring tool like Quad-MF can help you to analyse the reasons for poor performance, but you may not always be able to speed up the APL code that is causing the problem. This may be because the code is iterative or scalar – but it can also arise if APL cannot take advantage of some known characteristics of your data. Perhaps an example will clarify this problem. Suppose that you are consolidating financial data for an international sales organisation, in which you have sales for branches, and you want to total these by district, area and country. You may well use an APL inner product to do your calculation. If so, you will often use a boolean argument to specify which branches belong to which districts, and floating point numbers for your sales figures. APL will diligently unpack your boolean values and do floating-point multiplications with them, instead of just using a 0 or 1 to decide whether to include the number in a total. If you recode this one function in a lower-level language, the speed-up can be dramatic.
You may encounter a third reason to interface APL if you want to use existing code that has been written in another language. There must be many thousands of specialist libraries available; libraries of mathematical functions, interfaces to databases and spreadsheets, drivers for graphics display devices and many, many more. If you want to use them in your APL application, you will have to interface to the language in which they are written. You may also want to use an APL front-end to existing application code.
Which Language?¶
You are confronted with a huge choice of languages in which to develop your own software, and many of them are now supported by commercially available libraries. Which language should you use? If you are using STSC’s APL*PLUS/PC or II system for the 386, your choice will be strongly influenced by the interfacing capabilities of the interpreter that you are using. Although you can invoke application programs using quad-CMD, the most efficient way of interfacing an APL application with routines written in other languages is via the quad-CALL system function.
STSC’s quad-CALL allows you to invoke assembler code (held as an array) from within an APL workspace. The assembler code is subject to certain restrictions; in particular, it must be relocatable, since the array containing it may be moved within your workspace by the APL interpreter.
You could restrict yourself to assembler as your language for use with APL, but assembler is not always ideal. Most people find it slow to write, and floating-point calculations are particularly difficult to develop. If you wish to use pre-written software, you may have no choice but to use something other than assembler; the software may be written in another language.
For many people looking for a general language to use with APL, C is an attractive choice. Most APLers who have come across C like the language, and learn it quickly. They find that C compilers are readily available, as is a large volume of public domain and commercial software. C code is portable to many environments, and many interfaces to databases and spreadsheets are implemented in C.
If you have already coded your application, and the portion to be speeded up is written in ISO APL, you might even be able to use the SOFREMI APL compiler to generate C code directly.
Overcoming the Problems with Using C and APL.¶
Alas, the choice of C as your interface language is not without problems. The code produced by standard C compilers does not meet the restrictions that quad-CALL imposes. Moreover, C data structures differ from those used internally by APL. Worst of all, both APL and C have their own dynamic memory management methods, which can clash.
Fortunately, these obstacles have been overcome for us by a commercially available package, developed by Lauer Software. Their APL2C interface allows you to develop applications that run under APL*PLUS/PC using both APL and C. The package contains four main components – a manual, a copy of the Datalight C compiler, libraries of C code and an APL workspace. The workspace contains a source file editor (ED) which can also trigger the compilation and linking of your C code, so you can edit, compile, link and test from within the APL environment. In addition, registered users get a newsletter containing technical tips and news of new releases. Upgrades are available at a nominal charge.
The current version (4.11) runs under APL*PLUS release 7, and comes with v3.1 of the Datalight compiler. In addition, it can be used with version 3.2 of Lattice C. The remainder of this article deals with my experience of the interface in practice.
My situation is unusual, in that I have just upgraded from an earlier version of APL2C that did not include the Datalight compiler. Both the APL2C software and Lattice C were installed from scratch, however, as I was upgrading my hardware at the same time.
The APL2C installation instructions are comprehensive, and easy to follow. (They leave the APL2C workspace in an APL2C sub-directory, however, without telling you that you will have to gain access to this library in order to load it. I suppose the assumption is that anyone experienced enough to attempt interfacing knows enough to work this out for themselves.) The installation of Lattice C is also straightforward; when both products have been installed successfully, you can carry on with the introductory example in the manual.
I suspect that this example works fine if you are using Datalight, but the C code generates compiler errors with Lattice C. The reason is simple; return statements have been omitted from the end of each C function. Presumably the Datalight compiler inserts them for you. A corrected version of the code for Lattice users is included as Appendix A.
This is not the only setback that Lattice users will encounter. The sample compilation in the APL2C manual covers three situations: compilation and linkage of C software for the two compilers under DOS, and for the Datalight compiler using the ED APL-based editor. If you want to compile using Lattice C from within ED, you need to customise the compile and link commands; you can find out how to do this in a much later section of the manual (p98 et seq., on ED).
Two Sample Applications.¶
It took me about half a day to install the software and get everything properly set up; I then went on to use the interface for two sample applications. The first was a utility for use with STSC’s spreadsheet manager. The second was a utility to perform character set translation.
The first utility was a C version of code that I had prototyped in APL, but which ran too slowly. The utility took a LOTUS spreadsheet name and a range name; it returned the coordinates of the corners of the range within the spreadsheet, or an error code if something went wrong. It worked by reading through the spreadsheet file, record by record, until it came across the record that it required.
My APL original worked, but it ran rather slowly. It took about four seconds to locate a range that existed; if I gave a non-existent range name in a large spreadsheet, the APL version took about forty seconds to pass the file before giving up.
I used quad-MF to confirm that most of the time was spent reading records and checking their type. It looked as if this would run much faster in C; after half a day of coding, I was able to test the C code using the APL2C interface. It took less than two seconds to find a range, and less than four to establish that a range did not exist.
The speed-up of two to one on a four second operation was worthwhile, but the speed-up of ten to one on a forty second operation could mean failure or success for a commercial system. Encouraged by this, I took a look at a much simpler problem that I hoped C could solve.
Most of us have met the need to do character set translation. It may just be a matter of changing lower case text to upper case; it may be that we are moving data from one computer to another. This is very easy to do in APL; the idiom below takes W, a character array, and translates each character in it using a 256-byte translate table A.
Although this idiom is easy to code, it is slow. I thought that a C version would run much faster, and so it did. A large (5K) character vector took about 3 seconds to translate using the APL idiom, and less than half a second using C. The C code was easy to write, and (like the APL idiom) the code worked first time. A C listing is included as Appendix B.
I felt very pleased until Jonathan Barman (a colleague of mine from Cocking & Drury) gave me his assembler function to do the same job. It ran about ten times faster than the C code, and sixty times faster than the APL. He explained that this was as he expected, since the assembler code used a hardware instruction that did just what was wanted at hardware speed.
Looking at Trade-offs.¶
You may feel from all this that the trade-offs in using different languages are complex. Things are further complicated by some idiosyncrasies of C (or the APL2C interface). Firstly, C distinguishes between scalars and other arrays in a way that APL does not. This difference is echoed in the interface, which passes pointers to APL arrays, but values of APL scalars. If you want to write a function using the interface that behaves in the same way with an argument of any rank, you will have to do some un-natural juggling to achieve this.
A second problem occurs with memory management. Much C code uses dynamic memory allocation – a method by which space is requested when the program needs it, and then freed when it is no longer required. This happens explicitly in C, but it also happens implicitly in APL. The two languages use different methods of handling the problem, and although the interface tries to deal with this it does not do so completely.
Another thing that you can take for granted when using APL is data typing. Most of us do not need to worry about whether a particular array is integer or real. In C, we have to know this – and know it in advance. This causes two problems: if we do a calculation in C on integers, and produce an answer that is too big to hold as integer, nothing will tell us that we have done so; and if we do a calculation in APL on Real numbers that happens to give us an integer answer, APL will change the type without consulting or informing us. The trick for getting around the second problem is covered by the interface (see pp22, 97 of the manual), but I know of no general solution to the first.
C code may take less time, but it does take up a lot of room. The object code for my simple translate example takes 11K bytes! Almost all of this is due to the inclusion of library code that I do not need, but it would not be trivial to eliminate this. (By contrast, Jonathan’s assembler function is about 300 bytes.)
Finally, debugging C code from inside APL is not easy. We are spoilt by the excellent debugging facilities that form an inseparable part of the APL environment; these days, C programmers can use debugging aids that are comparable in convenience and power. Unfortunately, there seems to be no way to use these when running C code via the APL2C interface.
Conclusions¶
Hybrid systems can offer real beneifts, and C is a good choice if performance needs a speed-up, and the algorithm is complex or includes floating point calculations. Moreover, C allows you to use a lot of existing code. Assembler may be better for simple integer or char manipulation.
Appendix A: Sample C program from Lauer manual¶
… corrected for use with Lattice C.
/*********************************************************************/
/* Timestamp: Sat Jun 25 1988 05:52:18 PM */
/* Program Name: demo prog from Lauer Software manual */
/* Programmer: Romilly Cocking */
/* Copyright (C) Cocking & Drury Ltd 1988 */
/*********************************************************************/
#include <apl.h>
int csub1(),csub2(),csub3();
int (*rlookup[])() = {csub1,csub2,csub3};
int numfns = 3;
csub1() /* no arguments */
{
printf("Hello world!\n");
return(0);
}
csub2(n) /* one scalar argument */
int n;
{
int i;
for (i=0; i<n; i++)
printf("Hello from C level. i=%d\n",i);
return(0);
}
csub3(x) /* one array argument */
unsigned x;
{
int i;
printf("Rank of X = %d\n",RANK(x));
for (i=0; i<RANK(x); i++)
printf("Length of dimension %d is %d\n",i+1,peekw(x,8+i*2));
return(0);
}
Appendix B: Source for Translation Function¶
/*********************************************************************/
/* Timestamp: Sun Jun 26 1988 06:27:45 PM */
/* Program Name: TRANS */
/* Programmer: Romilly Cocking */
/* Copyright (C) Cocking & Drury 1988 */
/*********************************************************************/
#include <apl.h>
#include <aplobj.h>
APL2CFN(trans); /* insert interface code for function 'trans' */
char tab[256]; /* holds local copy of translate table, for speed */
trans(inseg,tabseg,outseg) /* inseg - segment address of APL input data
tabseg - same for translate table
outseg - same for result */
unsigned inseg,tabseg,outseg;
{
char far *ip;
char far *tp;
char far *op;
int i;
ip = apl_object[0].start; /* set ptr to data values in array */
tp = apl_object[1].start;
op = apl_object[2].start;
for (i=0; i<256; i++) /* copy each table entry into local array */
tab[i] = *(tp+i);
for (i=0; i<numelems(inseg); i++) /* for each element of array */
*op++ = tab[(int) *(ip+i)]; /* tranlate and store in result */
return(0);
}