The Genesis of ASL (2)

VECTOR VolL7 No.4

Deputy Chairman: Alan Sykes, Department of Management Science and Statistics, University College of Swansea, Singleton Park, Swansea SA2 8PP. Telephone: 0792-295296

Manager: Jake Ansell, Business Studies, University of Edinburgh, William Robertson Building, 50 George Square, Edinburgh EH8 9JY. Telephone: 031-667-1011

BAA Representative & Newsletter Editor: David Eastwood, MicroAPL Ltd, South Bank Technopark, 90 London Road, London SE1 6LN. Telephone: 071-922-8866

BAA Representative: John Searle, 13A Mount Ararat Road, Richmond, Surrey TW10 6PQ. Telephone: 081-948-6737 (home)

The Genesis of ASL (2): October 1988

by Tony O’Hagan

First Approaches to the BAA

The very earliest beginnings of the ASL idea were chronicled in the first article in * this series. The next key event was a meeting on 5th October 1988 at IBM South Bank, London. Present were Jake Ansell, David Eastwood, Norman Thomson and myself. This meeting drew up the following list of goals as part of establishing an APL service to statistics users”.

1. Define standard functions for basic statistical operations. Identify good implementations, possibly by soliciting code from the statistics/ APL community.

2. Identify standards for delivering functions/workspaces to different APL systems.

3. Draft standards for more complex functions, their syntax and means of verification.

4. Create a library for statistical functions, comprising the standard basic functions and others submitted by users.

36

VECTOR Vol.7 No.4

5. Prepare housekeeping procedures/ functions to administer this library. These should cover receipt, verification and storage of new functions/ workspaces, plus selection, packaging and delivery of requested functions / workspaces.

6. Document functions in book form, possibly loose-leaf.

Several ways of achieving these goals were discussed, but the favourite was for a student to do it as a one-year MSc project. I agreed to look into the possibility. Finally, the meeting considered ways of funding the project. The idea of an MSc student was attractive because it would be cheap, but still substantial money would be involved. It was here that involvement of the British APL Association was first suggested. Another possibility was funding by IBM.

My investigations concerning the MSc idea were not very successful, for similar reasons as caused us to drop the thought PhD project in our earlier discussions. But David Eastwood made contact with the BAA on our behalf at a BAA committee meeting in November. They expressed an interest in principle in providing support for our project. In particular, they suggested that the ball could be set rolling by having a conference on APL and Statistics’, to establish demand and collect ideas. This very positive response to our tentative enquiries was extremely encouraging. For the first time, it looked like the statistics library’ might be in business!

T have no record of what happened in the next couple of months. I believe that we were asked to clarify our thinking, but no written proposal seems to have been made then, nor any meeting held. The BAA committee met again in January 1989, Alan Sykes, who was the BAA Education Officer at the time, wrote me a letter dated 18th January reporting the committee’s conclusions. A substantial part of Alan’s Education budget for the year was to be given to the proposed conference. He had already made a booking for it at the University of Wales Conference Centre at Gregynog for that September. Furthermore, a firm proposal for funding of the project itself would be welcomed.

Another meeting took place on 24th February, again at IBM South Bank. This was publicised by Jake Ansell through the APL Statistics Users” Group, and attracted several interested people from industry. In all, the attendance was Dick Bowman, Mike Day, David Eastwood, Peter Lane, A MacGillivray, Robin Morphet, Alan Sykes, Norman Thomson and myself. A great deal was discussed. Dick Bowman suggested that Sig APL might also contribute to the ASL project. Dick later followed this up without success. It seems that the American group are very wary of supporting an initiative which might later compete (unfairly’ is implied) with commercial products.

37

VECTOR Vol.7 No.4

The main achievement of this meeting was to establish two interim committees. A Conference Committee, chaired Sykes, would meet to plan the conference in September. A Proposal Committee, with me in the chair, was charged with drawing up detailed proposals for funding of proper, for submission to Sig APL and the BAA.

Another interesting suggestion made at this meeting was due course the Royal Statistical Society might be prevailed upon to put some kind of seal of approval on the functions produced. Although this has not been pursued any further, we still hope to do so when sufficiently battle-hardened software is teady. The meeting ended with a discussion of a name for the library, and for the project. S-APL was tentatively agreed.

My records are hazy again here. It is clear that ] prepared a first draft proposal and sent it to members of the Proposal Committee only four days later, on 28th February. It seems likely that when the meeting on the 24th ended it metamorphosed Proposal Committee meeting, on to discuss detailed proposals at some length. The text of draft is no longer with me, either. It went through a series of modifications, including further feedback from the BAA in April, 5th as a substantial five-page document. The name had changed to ASL = APL Statistics Library (because S- APL was already used by 1 P Sharp), and that name has stuck. Since this document defines ASL in basically the form which BAA agreed formally to fund, itis worth setting down here the essential features.

1. The objectives were set out as to establish:

(a) a set of standards which will give coherence to the assembly of functions, make them easy to use singly and in combination, facilitate porting to a range of interpreters and hardware, and help with verifying their correctness;

(b) a bottom shelf’ of functions comprising the most fundamental statistical routines (descriptive statistics, calculations on standard distributions, etc.), plus utility functions (data handling and I/O, graphics, numerical procedures, etc.);

(c) various other shelves  of functions for specific statistical applications, such as regression (linear models), categorical data analysis, multivariate statistics, reliability, variance, sample survey analysis, time series and forecasting, all taking functions from the bottom shelf, and possibly other shelves (e.g. a shelf for generalised linear models may take functions from the regression shelf).”

38

VECTOR Vol.7 No.4

2. The benefits were described as follows. One of the best-selling categories of software is statistics packages... We want ASL to be a flagship for APL in this important market.” But it was stressed that ASL would not be just another statistics package. ... it will not be a closed package, allowing the user to do only what the programmers have thought to include. The full power of APL will always be available, ASL merely providing a sophisticated set of functions, which the user will then combine with each other, the APL primitives and his/her own functions. For the lay user, the documentation will provide idioms for operations such as selection of variables and cases, and passing data between functions. To such a user ASL will appear as a complete package, as general and user-friendly as conventional packages. To the APL user it will be much more.”

3. It was also boldly claimed that APL is the for doing statistics. Data analysis is essentially exploratory. It is not a simple matter of applying a single standard analysis to the data, but of trying different kinds of analysis, looking at the data in different ways, and drawing together a range of conclusions. The APL environment is ideal for this activity. The APL language also uniquely supports the processing of arrays of data neatly and simply.”

4, Management of ASL was to be by a Management Committee, to be elected at the September conference. In addition to a Chairman, Manager (responsible for day-to-day running of the ASL effort), Treasurer and Secretary, it was hoped to have BAA involvement through its Projects, Technical and Education Officers.

5. A programme of work was described, culminating with a general release of the first shelves  in September 1990, These were to comprise a bottom shelf as above and a regression shelf. Although both first and second generation APLs were to be supported, APL2  implementation was not the first priority.

6. Funding was requested for a programmer, working for about 2 days per week for a year, for a PC for the programmer, and a smail amount for expenses and consumables, The BAA was asked to commit money on a phased basis, payment at the second phase being subject to satisfactory progress in the first phase (to March 1990).

Since then, there minor changes in the formal objectives, the management and the timescale, which will be described in later episodes of this series ... (watch this space!), but the ideals of (2) and (3) have kept ASL going ever since.

39

VECTOR VolL7 No.4

The ASL Regression Shelf

by Alan Sykes

Introduction The ASL project outlined initial shelves  to be produced in the first year. One of these was a Regression Shelf. An exploratory workspace was produced and discussed by the ASL committee. That workspace built on the work of Jake Ansell and Alan Sykes over recent years in using APL routines that behaved similarly to those implemented in the Royal Statistical Society's Software Package GLIM . It was decided that the Regression Shelf should contain a set of functions that performed GLIM-type calculations in a way that allowed them to be incorporated into user functions.

Introduction to Regression Most users of Statistics, or for that matter users of APL, understand the basic principle of Simple Linear Regression’ where a response variable’s average value depends on a predictor variable linear way. The equation of that line is estimated from bi-variate data by the method of least-squares, which in APL means the use of the dyadic version of domino .

In this introduction, some basic terminology is explained and the simple linear regression model is extended into the concept of a statistical linear model.

There are three major attributes of statistical linear model. First we have the RESPONSE variable, often referred to as the y-variable . This is usually random, with distribution assumed to be NORMAL or GAUSSIAN. Secondly, we have one or more PREDICTOR variables, usually regarded as non-random, which predict the average value of the response variable in a linear way. If E(Y) denotes the expected  value or mean value of the response variable Y, and U,V,..W are predictor variables then we assume that E(X) = a+bUscv+....aw ,

where a,b,...d are unknown constants.

So much for the mean of Y; what about its variance? The simplest assumption to make is that the variance is constant, given by k say. Hence y has a Normal Distribution with mean at+bU+cU+...tdW and variance k. Regression Analysis

40