Namespaces in Causeway

Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 94

Causeway is an application development environment for Dyalog APL that has been described in other articles in Vector and demonstrated at various conferences. The latest version which is in beta test is wholly integrated with the Dyalog APL namespace extensions, taking advantage of them in its own internal organisation and being sensibly tolerant of namespace changes within application code developed in Causeway. This article introduces the concepts upon which Causeway is based and describes how it is used in application development. It then describes the rôle of namespaces in the internal organisation of Causeway and the way in which the use of different namespaces in application code affects the behaviour of Causeway.

Overview of Causeway¶

Causeway is a tool for generating Windows interfaces for APL applications. The basic unit of interface is a form, represented by a window on the screen. It contains controls with which the user may interact. Controls can do two main things, display application data such as the content of edit fields, and allow the user to trigger actions in the application such as on the pressing of buttons. Of the controls which display data some will allow the user to change the data.

Data¶

Each control may have more than one aspect of its visual appearance. For a list box one aspect might be the names in the list, another might be the selection, those items that have been highlighted. These aspects are called properties. The principal concept behind Causeway is that a property of a control may be linked to data in the application, typically a variable. When the user changes the property of the control then the variable is updated. When the application changes the variable the control is updated to reflect the new data. Both aspects of this interchange occur automatically, managed by Causeway.

The data in the application need not be just a variable. For properties of a control that the user cannot change, the data can be defined by any APL expression that returns an array of the correct type for the property. If the user can change the property then that change must be passed to the data. The way that Causeway does this is to surround the expression supplying the data with parentheses and use the resulting phrase to the left of an assignment. Thus any variable, list of variables, indexed variable or selective specification of a variable can be used.

Each expression providing data for a property will depend on a number of variables. The developer needs to specify these variables as they cannot be derived reliably from the expression, for instance when the indices of an indexed variable are provided by a function that calls another function etc.

Events¶

In general the developer can define for a control a set of APL expressions to execute on each event that can occur to a control. A typical event for a button might be being pressed, for a form being resized, for an edit field a key press.

If the APL expression changes application data then it is a good idea for the developer to tell Causeway that this has happened. This is the only way that Causeway can know that the data has changed. It needs to know so that all controls with properties dependent upon those variables can be updated. When specifying the expression to run on an event there is space for putting just this information.

Controls as Classes¶

Each control on a form is an example or "instance" of a particular "class" of control. It has appearance, behaviour and properties that are defined by the class of which it is an instance. Dyalog APL itself has control classes such as EDIT and BUTTON. In Causeway the need to link controls to data and to extend or modify the functionality of the Dyalog classes has encouraged us to define our own classes. Each control on a Causeway form is an instance of one of these Causeway classes. In the version of Causeway distributed with Dyalog APL the definitions of the classes together with all the functions managing the behaviour were held in a single global variable called ∆classtab. In the new version each class is defined by the functions and variables in a namespace of the name of the class.

Form Definitions¶

The definition of a form and its controls in Causeway is held in a five-column APL matrix. The first row defines the form and each subsequent row defines one control. The first element of each row is always the control’s class. The next three are caption, position and size and the final element is itself a two-column matrix of property-value pairs.

The form is created by running a Causeway function, passing in the definition variable. There are two functions that do this. Call creates the form, waits while the user interacts with it and returns when the user closes it. Make creates the form and returns immediately leaving the form on the screen for the user to interact with at his leisure.

In the previous version of Causeway the functionality of Make and Call were provided by Gui_make and Gui_call. The Gui_ prefix is no longer necessary as the functions are in a separate namespace and the initial letter is capitalised to indicate that they are functions. Similarly Gui_reject is now Reject and Gui_return is Return. The conversion routine for form definitions makes these and other replacements in property and action expressions. A routine is provided to make the replacements in application functions.

Namespace Dependence¶

The evaluation of application expressions and resolution of application variable names is central to the work that Causeway does. Traditionally the functions Gui_make and Gui_call lived in #, the root namespace, and thus all names were resolved relative to # and all expressions were evaluated there also.

If the Causeway functions were placed in a different namespace, e.g. #.Gui, then names and expressions would be resolved and evaluated in that namespace. If this approach was adopted then all variable names, in dependence lists for properties or in lists of changed variables for actions, would need to be fully qualified (beginning with #.). Furthermore all names in all expressions, whether they be for properties or actions, would need also to be fully qualified.

It seemed to us when designing the new version that the most desirable namespace in which to resolve names and evaluate expressions would be the namespace from which Make or Call was called. Thus Make (which is a sub-function of Call) picks the first namespace in ⎕NSI which differs from its own and marks that as being the namespace for that instance of the form. All names are taken as relative to that namespace and all expressions are evaluated there.

The way in which this is achieved is explained below in Name Resolution and in Expression Evaluation.

Expression Evaluation¶

In order to execute the developer’s code in the namespace from which the form was called, Causeway collects all the expressions, whether they be property expressions or event actions, and fixes them in a function the first line of which changes into the relevant namespace. The following line branches to the line to be executed. If there is more than one expression to be evaluated it does them in the same call, visiting each required line in turn. The list of lines to visit is held in a local variable in the calling function, for reasons that will become apparent. After each line has been executed the list is beheaded and branched to. In the simple case 0 is appended to the list before the first branch, so that the final branch exits the function. This is complicated if the form has local variables, about which more later, but the simple case is illustrated below. The function executes the expressions for form 1 of which there are two, on lines 15 and 19 below.

     ∇ Exec1;clicks
[1]    #.CPro._dummy clicks←1,#.CPro.loc_val[⎕IO+0]
[2]    ⎕CS #.CPro._lcl⊃#.CPro.lcl_spc
[3]    :If 1<⍴#.CPro._fns
[4]      ⍎⎕IO⊃#.CPro._fns←1↓#.CPro._fns
[5]    :EndIf
[6]    →#.CPro._next←#.CPro._next,⍬⍴⎕LC+1
[7]    #.CPro.loc_val[⎕IO+0]←1↓1 clicks
[8]    :Return
[9]   ⍝ Set #.CPro._obj to the control name and refresh controls
[10]   #.CPro._ref←0 2⍴0 ⋄ #.CPro._obj←(⎕IO+1)⊃#.CPro.ctl_name
[11]   →#.CPro._next←1↓#.CPro._next
[12]   :If 0<1↑⍴#.CPro._ref ⋄ #.CPro._ref #.Class.FM.Refresh #.CPro._obj ⋄ :EndIf
[13]   →#.CPro._next←1↓#.CPro._next
[14]  ⍝ Execute action expressions
[15]   :If ~#.CPro._rej ⋄ clicks←clicks+1 ⋄ #.CPro.Touch 1 ⋄ :EndIf
[16]   →#.CPro._next←1↓#.CPro._next
[17]  ⍝ Evaluate property expressions
[18]   #.CPro._rej←0
[19]   :If ~#.CPro._rej ⋄ :AndIf
#.CPro._force∨((⎕IO+0)⊃#.CPro.prp_val)⍸#.CPro._val←(⍕clicks),' clicks so far' ⋄
((⎕IO+0)⊃#.CPro.prp_val)←#.CPro._val ⋄ #.CPro._ref⍪←'Data'#.CPro._val ⋄ :EndIf
[20]   →#.CPro._next←1↓#.CPro._next
[21]  ⍝ Read properties and execute property assignment expressions
     ∇

The first five lines and also the seventh implement variables local to forms as described below. Lines 6, 11, 13, 16 and 20 implement the branching mechanism and the remaining lines do various bits of housekeeping. The many references to #.CPro. in the body of the function are to avoid name pollution in the execution namespace.

Name Pollution¶

One of the problems with all application development tools is name pollution, a problem that namespaces are supposed to solve. You wish to import some functions of your own but the names clash with some functions in your development tool. The method of placing all the form’s application code in special functions that execute in the application’s space removes all the Causeway management code and working variables from the application namespace. However when a Dyalog function changes space all the local names are taken into the new space. Local names include not only local variables but also labels, arguments and the result. Therefore the execution function must neither have locals, arguments nor return a result. Thus the list of lines to execute must be global to the function and in the Causeway namespace. In the assembly of the function the ∇ character is used in place of a direct path to the Causeway namespace. In the fixing of the function all ∇s that occur in the executable code are replaced by the absolute path of the Causeway namespace. The function above started life looking marginally more readable as the vector of text vectors:

Exec1;clicks
∇_dummy  clicks ←1,∇loc_val[⎕io+0]
 ⎕cs ∇_lcl⊃∇lcl_spc
 :if 1<⍴∇_fns
  ⍎⎕io⊃∇_fns←1↓∇_fns
 :endif
 →∇_next←∇_next,⍬⍴⎕lc+1
∇loc_val[⎕io+0]←1↓1  clicks
 :return
⍝ Set ⍺ to the control name and refresh controls
∇_ref←0 2⍴0 ⋄ ⍺←(⎕io+1)⊃∇ctl_name
→∇_next←1↓∇_next
:if 0<1↑⍴∇_ref ⋄ ∇_ref #.Class.FM.Refresh ⍺ ⋄ :endif
→∇_next←1↓∇_next
⍝ Execute action expressions
:if ~∇_rej ⋄ clicks←clicks+1 ⋄ ∇Touch 1 ⋄ :endif
→∇_next←1↓∇_next
⍝ Evaluate property expressions
∇_rej←0
:if ~∇_rej ⋄ :andif ∇_force∨((⎕io+0)⊃∇prp_val)⍸∇_val←(⍕clicks),' clicks so
far' ⋄ ((⎕io+0)⊃∇prp_val)←∇_val ⋄ ∇_ref⍪←'Data' ∇_val ⋄ :endif
→∇_next←1↓∇_next
⍝ Read properties and execute property assignment expressions

This substitution has the side effect that any ∇ in the application expressions will automatically refer to Causeway. Thus in a form definition a reference to #.CPro.Make could be replaced by ∇Make which would be converted on form creation to #.CPro.Make. If at a later stage in the development of the application the Causeway namespace was moved or renamed (e.g. to #.Causeway) then code in form definitions need not be changed as the ∇s would then automatically be converted to #.Causeway..

Local Variables¶

A Causeway form can contain local variables. Each instance of the form will have its own data associated with the local names so that if a form definition is used more than once to create separate forms, each can have its own copy of certain variables invisible to the other instances. The names of the variables are given as the "Locals" property of the form and any references to those names in any data expressions or event actions of the form or its controls will resolve to data specific to the form. The values of the variables are actually held in a table in the Causeway namespace and on execution of the application code they are assigned to local names in the execution function. At the end of the execution then the values of all local variables must be placed back in the appropriate places in the value table. This is done on the seventh line of the function, after which the function ends. Thus on the sixth line 7 is added to the end of the list of lines to execute. Effectively the function is a header followed by a set of subroutines and ∇_next is the list of subroutines to execute.

If a Causeway form is created from an action on another form then that form is its parent. If the parent form has any locals then these are visible to all actions on the child form, except of course those that the child form has shadowed with locals of its own. This is very similar to the way in which a called function inherits the locals of its calling function. Causeway needs to match this behaviour. The easiest way to do this is to have a function in the stack for each parent form, each function localising, setting and finally saving the locals for that form. This is what lines 3 to 5 of Exec1 are doing.

For each form we hold its context which is the list of those forms in its list of parents that contain local variables. When we need to execute an action on a form we execute in turn each line of the function for each form in the context. They each set up their local variables, execute the next function in the list and then save the values of their local variables. Because we are implementing form locals with functions then everything that is true of function locals, with respect to changes of space, is also true of form locals. Thus, if in an action on a form I change space, then my form locals are still available in the new space.

The function that executes the context functions for a form is ExecLines which takes a set of lines to execute and a form index.

     ∇ _lcl ExecLines _next;stack;_fns;_dummy
[1]   ⍝ Execute the lines <_next> in the locale <_lcl>
[2]   ⍝ <_dummy> is used in the assignment of locals in the ExecNN functions
[3]   ⍝ <_stack> is the list of environment functions currently on the stack
[4]
[5]   ⍝ Save the value of <_stack>
[6]    stack←_stack
[7]
[8]   ⍝ Create the environment function list
[9]    _fns←(⊂(⎕CS''),'.Fns.Exec'),∘⍕¨(_lcl⊃lcl_ctx)∪_lcl
[10]
[11]  ⍝ Remove any already running
[12]   _fns~←_stack
[13]
[14]  ⍝ Add the number of new functions to <_stack>
[15]   _stack,←_fns
[16]
[17]  ⍝ Execute the new functions
[18]   ⍎↑_fns
[19]
[20]  ⍝ Restore <_stack> to its previous value
[21]   _stack←stack
     ∇

Name Resolution¶

When a name occurs in a list of dependence for a property or of variables changed by an action it must be matched against the dependence lists for each property, not only on that form but on any form. The name in the list is both relative to the namespace in which the form runs and also the localisation affecting the form. In order to avoid confusion all dependence between variables and properties is held in Causeway as a correspondence between indices into a properties table and indices into a name list. Each fully qualified global name appears in this list once and each local variable appears once per instance fully qualified by the namespace of the form of which it is the local.

When a name is given as being relative to a form it goes through a two-stage process. First it is qualified by the function Qualify.

     ∇ r←space Qualify names;dots;spaces
[1]   ⍝ Fully qualify <names> given relative to the namespace <space>
[2]    :If '.'∊∊names
[3]      dots←⍳∘'.'∘⌽¨names
[4]      spaces←(-1+dots)↓¨names
[5]      names←(-dots)↑¨names
[6]      spaces←space⍎(⎕CS''),'.spaces⍎¨⊂''⎕cs'''''''''''
[7]      r←spaces,¨'.',¨names
[8]    :Else
[9]      r←(⊂space⍎'⎕cs'''''),¨'.',¨names
[10]   :EndIf
     ∇

If the name contains no dots then the space name is prefixed to it. If it has a dot in it then it has a path part and a name part. These are separated and the phrase ⎕CS '' is run in the space named in the path part. ⎕CS '' will always return the shortest full path to the space in which it is run. So even if the same variable is referred to using two different paths they will resolve to the same entry.

Next the qualified name is compared to local names for the form and then to known global names. Any new global names are added to the list of known global names.

     ∇ vids←locale Resolve names;unames;locals;lvids;linx;global;ginx;new
[1]   ⍝ Resolve <names> relative <locale> into name indices
[2]
[3]   ⍝ Ensure names are fully qualified
[4]    names←2↓¨(locale⊃lcl_spc)Qualify names
[5]
[6]   ⍝ Work on the unique set
[7]    unames←∪names
[8]
[9]   ⍝ Set up the result
[10]   vids←(⍴unames)⍴¯1
[11]
[12]  ⍝ Retrieve the set of name indices for the
[13]  ⍝  locals of the context of <locale>
[14]   locals←∊lcl_locs[⌽locale⊃lcl_ctx]
[15]   lvids←loc_vid[locals]
[16]
[17]  ⍝ Look for <names> in the list of local names
[18]   linx←vid_name[lvids]⍳unames
[19]
[20]  ⍝ If not local, a name is global
[21]   global←linx=↑⍴locals
[22]
[23]  ⍝ Place indices for local names into the result
[24]   ((~global)/vids)←lvids[(~global)/linx]
[25]
[26]  ⍝ Look for the remaining names in the known global names
[27]   ginx←vid_name[glb_vid]⍳global/unames
[28]
[29]  ⍝ Any not found are new
[30]   new←ginx=↑⍴glb_vid
[31]
[32]  ⍝ Place indices of found names in the result
[33]   ((~new)/global/vids)←glb_vid[(~new)/ginx]
[34]
[35]  ⍝ If there are any new global names then add them to
[36]  ⍝  the name list and the global list, placing the new
[37]  ⍝  indices in the result.
[38]   :If ∨/new
[39]     (new/global/vids)←glb_vid,←AddVids new/global/unames
[40]   :EndIf
[41]   vids←vids[unames⍳names]
     ∇

The resulting index distinctly identifies either a global name in a namespace or a local variable on an instance of a form.

Control Library¶

One of the main criticisms of the previous version from the point of view of advanced users was that it was difficult to add a new control class or to change the behaviour of an existing one. The class definitions were held in a deeply nested matrix in the global environment. The functions that implemented the behaviour of a control were held within this matrix as ⎕ORs and it needed a set of special maintenance tools to edit these functions. The structure of the matrix was not readily accessible and this reflected the fact that nearly all class development was done by either Adrian or myself, both of whom have a deep understanding of the internal workings of Causeway.

Another disadvantage of the single control definition matrix was that if a new version of a control class was sent out inclusion of it would override any changes to the class that had been made by the developer. These disadvantages are catalogued in admirable detail in Jon Sandles’ article (in Vector 12.3 p.51) and it is mostly due to his experience that we have avoided many of the pitfalls in the new version.

In the new Causeway a control class is implemented by a namespace of the same name which contains a specification, which also documents the class, some required variables and functions and any other functions or variables used by the class. The required variables and functions and the reasons for the requirement are given later. The individual control class namespaces are held in a parent namespace that can contain common code for dealing with e.g. bitmaps and icons or ⎕NA calls.

In my test version my parent namespace is called #.Class and the control classes are #.Class.AC, #.Class.TM etc. However the parent namespace could be called anything and indeed there could be more than one class parent namespace. The variable ∆classpath in the Causeway namespace is a list of namespaces in which can be found control classes. The initialisation of Causeway traverses this path and collects a list of all the classes found therein. If a class of the same name appears in two places on the path then Causeway will use the first and ignore all subsequent occurrences. Thus the class developer can have two or more versions of the same class and determine which is used by the order of the search path.

The main advantage of the namespace approach is in the accessibility of the class to the developer. The Dyalog namespace explorer can be used to browse the class tree and to show and edit the components of a particular class. The session space can also be set to the class. If while an application is running an error occurs in a class function then the session will suspend in the class and the error can be fixed. The application can be restarted and the bug will be fixed not only for the instance in which the error had occurred but also in any other active instances of the same class. This makes for much more dynamic application development, something to which APL has made us accustomed.

Class requirements¶

A control class namespace must contain a number of variables and functions without which Causeway will fail. These are for the management of instances of the class. Causeway must manage the instances but from the simplest (the label) to the most complex (the text grid) it knows nothing of the internal workings. It needs to be able to create and destroy instances, to pass data for the control to display, to know when the control has changed and to read new data from the control. To do each of these tasks there must be a function in the class with a set name that the Causeway engine can call to perform these actions.

When a new form is run from a definition variable each row defines a control (instance of a class). For each row Causeway runs the Create function in the class passing it the remainder of the row which contains the name of the parent, the position and size and caption and a table of other properties that are needed for the creation (e.g. colour, hint, caption position). The Create function will create the instance, normally but not always as a child of the parent, and return the actual name of the new instance. In all subsequent dealings with the class regarding the control, that name will be given for identification of the specific instance. As the data which the control is displaying changes, Causeway will run the Refresh function in the class, passing the name of the instance to be refreshed and the new data. The task of presenting the data to the user in the instance is given to the class.

The class specification, a variable in the class namespace, contains a list of the events that will be triggered if the application user changed any aspect of the data of the instance (property). If one of these events occurs then Causeway is alerted and it calls the Read function in the class with the name of the property that has changed and the message reporting the event, which contains the name of the instance. The Read function returns an APL array corresponding to the changed property, which Causeway assigns to the expression associated with the property in the form definition. Thus the class knows how to represent data in and retrieve data from the control, but not the application expression giving that data nor the namespace in which it is run. Conversely the Causeway engine knows how and where to get and put the application data, but not how it is represented in the control.

Finally when the user closes the form Causeway must call the Destroy function in the class of each control on the form. It cannot simply expunge the control because associated with the instance, and created by the Create and Refresh functions, there could be other resources that might then remain.

As each of the above functions is called, the current namespace changes to the class namespace for the duration of the call. This is normal (and unavoidable) in calling functions in Dyalog namespaces. Thus any other utility functions and variables needed by the main functions are there ready to be used. Classes that use ⎕NA functions need only associate them once and they will then be available for use by all instances. Bitmaps, such as the VCR button faces and the tick box for the Yes/No control, can be shared between instances, thus saving space and reducing instance creation time. As mentioned above, utility functions can be shared between classes by including them in the parent namespace. The class functions need only refer to them e.g. as ##.Label or ##.Uppercase.

Instances as Namespaces¶

The fact that all Dyalog GUI objects are themselves namespaces is not used by Causeway. It is possible for class functions to use the instance as a namespace but the decision is that of the implementor of the class.

An alternative approach is to copy the class functions into the instance namespace on creation of the instance, or to create the instance from the ⎕OR of a previously created "proto-instance". This has the advantage that any data relevant to the instance is immediately available to the class functions. I have not taken this approach for a number of reasons.

Firstly, as mentioned above, if each instance uses the class functions directly then fixing a bug in a class function will fix it for all instances of the class. If the functions are copied into the instance or appear there from a ⎕OR then, although all the instances of the function begin identical, they are in fact separate and a change in the source will not affect the instances.

Also mentioned above is the sharing of data or resources between instances of a class. Although this is possible with functions existing and executing in the instance, by recording in the instance the location of the class definition and keeping the shared data there, it is easier using functions in the class namespace which have direct access to the data.

The third objection is that of space efficiency. Due to the mutable nature of Dyalog namespaces each one must carry its own symbol table, and all functions defined in the namespace must have their internal symbols matched against the symbols of the namespace. Thus when a function is copied from one namespace to another, although Dyalog APL is able to share the memory for the actual function itself, it must allocate memory for the correspondence of the function’s symbols to those of the namespace. Furthermore any of the function’s local symbols that are not already in the symbol table of the namespace must be added to it. This is entirely due to the dynamic nature of the namespace. In C++ a class definition is static for the running of the application, as it must be for a compiled language. Thus an instance of a class only takes enough memory for the data of its members. In a language as flexible as APL some added overhead is unavoidable.

Conclusion¶

It is clear that for the writer of an application development tool, namespaces are a two-edged sword. On the one hand they allow great simplification of the class management code and the structure of the class definitions. On the other hand to give this same flexibility to the user more work needs to be done before executing the application code.

There is no doubting however the benefits to the application developer. He can organise his application in the way that suits him. He can start a Causeway form from anywhere in the namespace tree and it will behave sensibly. He can change namespace willy nilly in the actions of a form: any new forms will run from the new namespace, but will still see local variables from their parent form.