The JADS/SMS Utility Management System
Vector 12:4, page 47
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 47
Introduction¶
In the last Vector I wrote about problems I had maintaining objects within the Causeway environment for Dyalog APL. There seemed to be a need for some sort of object management system and version control management system to allow maintenance of objects across different releases of Causeway.
Well over a year ago Vector was sent a copy of JAD SMS – "System Management Software for APL" – to review. For some reason, which now escapes me, I found problems getting the software going, but then I decided to look again into the whole topic of "Object Management", and as a result I mailed the creator of JAD SMS - David Crossley - and asked him for the latest version. Fortunately I had no problems getting the software going this time, so I began to explore what it could do for me .....
What Does JAD SMS Do?¶
JAD SMS is written in and used to manage functions/variables (referred to in the rest of this article as objects) in APL*PLUS II v5, although there are plans to port it to other APLs. The evaluation copy came bundled with the Toronto toolkit, which is a useful place to start in seeing how the software works. The system is made up of a bunch of APL functions (unfortunately locked) which can be )copy’ed into your workspace when you need them. They can be run interactively or embedded into your own functions. Also supplied is a DOS-based pop-up windows interface which calls all the lower-level functions for you. In this review I concentrate on this full screen interface, but bear in mind a great strength of this system is that all the lower-level code is well documented and you can build your own interface/routines based on it.
Why do we need JAD SMS? You’ve probably tried to organise your workspaces before. Namespaces have been helpful for this purpose. Indeed, the way JAD SMS stores functions in a hierarchical tree structure is much the same as storing your functions in a bunch of namespaces - so what more does it give us? Having stored your functions in namespaces, you’ve also probably used that same structure in many different workspaces – indeed all you needed to do was )copy the namespace tree to proliferate whole chunks of working systems from one application to another. But what happens when you find a bug right in the core of one of these proliferated functions? All of a sudden you need to update all the different occurrences of the function – remembering where they all are could be quite tricky. In a multi-developer environment the problem is even worse – there’s always someone who likes to ‘fiddle’ the utilities to work the way they want them to. If you do a wholescale update of every workspace you can lay your hands on with the fixed function you could create three different problems .... (and probably more).
- You fix the bug – but maybe introduce new problems since the original developer had already worked around the problem – the work-around could now have undesirable effects.
- You fix the bug - but you lose any fudges other developers have built into specific applications over time.
- You fail to fix the bug – none of your workspaces work anymore!
Thus, implementing the bug-fix could be very time consuming, and testing the bug-fix could be even harder. An object management utility should enable the development team to work together and share the same utility set – but be flexible enough to allow developers to adjust them in a controlled manner. It does this by providing information about the status of utilities in different applications, and the status of the master utilities. This information allows developers to update common code across a number of applications in such a way that they can have a high degree of confidence that the change will not introduce new bugs.
A Test Example¶
To illustrate how far JAD SMS takes us towards this goal I developed a test example. I have a set of functions which provide reasonably advanced print capabilities from DOS – allowing the user to select printers and define the printer type and select the paper orientation etc. (if the printer is capable). The functionality is quite simple – OPEN 'FRED' opens a channel for printing, ∆PR MAT sends data to the open print channel – and CLOSE 'FRED' closes the print channel and spools the print to the printer. The functions – although quite simple – rely on quite a lot of standard utility code and when I tried to isolate them from the rest of the workspace I was surprised that I actually needed a lot more than just the 3 functions to make them run. In fact I needed a monstrous 37 functions! Only two of these were specific to the print – all the others were utility functions which are used hundreds of times in hundreds of workspaces.
This should illustrate how dependent code is on the standard utilities that you take for granted. Not only that, but if a bug was to crop up in one of these utilities (it does happen) how important it would be to identify where you need to upgrade your workspaces.
To help illustrate further here is the calling tree of the function OPEN .....
OPEN
┌────────┬───┬───────┬──────────┬──────┴────┬───────┬───────┬────────┬─────┐
CAT CHK∆LPTS IF NEXIST WIN∆PICK WIN∆UNDO ∆DOIF ∆IOTA ∆SETUP ∆TAB
Some of these functions have their own calling trees calling other lower-level utilities.
JAD SMS allows us to store our utilities in a structure very similar to the DOS directory structure. A top-level utility library is in effect an APL sharefile (called a database in JAD SMS) storing all the utilities, but within that you can break the utilities up into a hierarchical structure made up of directories of objects. This is not the only way of grouping utilities in JAD SMS but it is the most important as directories are the level at which security is defined – and hence the level at which most of the JAD SMS utility functions work.
Security is an important aspect of JAD SMS and it works using an access matrix approach similar to many APL sharefile systems. It means you can give read-only access to utilities to all but the few who are actually responsible for the utility code, but also create new ‘development’ areas where you give access to completely different people. You may even want to prevent people from accessing areas of the utility set completely. Security is very well defined in JAD SMS and it seems to provide all the functionality you would want from such a system.
So what does it look like ....?
(Remember you don’t have to use the full screen interface – all the lower-level functions are provided and fully documented – but I found the full screen interface very useful to get the hang of what the system can do.)
TEST Tree
Esc Quit F1 Help F2 Users F3 Files F4 Search F5 PermTab f6 ⎕Fdup
F8 Print F9 Export F12 Ver Pg ↑/↓
JS Dir Actions: Make Delete Rename Move Attributes Enter
──────────────────────────────────────────────────────────────────────────
TEST GEN
PRINTER
WIN
Here I have created a database file called TEST in which I created 3 directories called GEN, PRINTER and WIN. This was simple enough to do - press <F3> ‘Files’ and add the file ‘TEST’. Then press <M> ‘Make’ and type in the name GEN etc. I made these 3 directories to store my printing utility code. In GEN I was going to put any general utilities that come in useful all the time, in PRINTER the utilities specific to printing, and in WIN the utilities specific to windowed output. I then loaded my printer workspace and copied in JAD SMS and populated the directories with the appropriate functions.
This was also very simple to do - I selected the directory I wanted to populate and hit Enter - this takes you into the directory screen (shown later) - hitting F2 <Add> brought up the additions screen. Here you can manually type in the names of the objects you want from your workspace adding into this directory. Far easier, as long as the workspace is not too big, is hitting F4 <AllWs> to get all your functions/variables from the workspace and deleting all the ones you don’t want in this directory. (It also only picks up those objects that aren’t already in the library.)
I did this for all three directories and the PRINTER directory screen looked like this ....
TEST.PRINTER Directory OPEN edited
Esc Cancel F1 Help F2 Add F3 OK F4 Search F5 Action F6 Restore
F7 Sort F8 Print F9 Attribs F10 Select F11 Cmpre F12 Tree Pg ↑/↓
1 of 10 JS Keys: ALT+A..Z, B C D E H L O P R S
──────────────────────────────────────────────────────────────────────────
Name Type Date Time Class # of Ver Rel User
Ver # #
CHKLPTS f 96-02-29 21:16 1 1 0* JS
CLOSE f 96-02-29 21:16 1 1 0* JS
OPEN f 96-02-29 21:16 1 1 0* JS
OPTIONS f 96-02-29 21:16 1 1 0* JS
∆PDESC v 96-02-29 21:16 1 1 0* JS
∆PR f 96-02-29 21:16 1 1 0* JS
∆PTYPE v 96-02-29 21:16 1 1 0* JS
∆SETUP f 96-02-29 21:16 1 1 0* JS
⍙⍙⍙prdest v 96-02-29 21:16 1 1 0* JS
⍙⍙⍙preset v 96-02-29 21:16 1 1 0* JS
This screen shows all my functions and variables that are directly related to printing - the v and the f for functions and variables. You then have the timestamp of when that object was last changed. On the right hand side is the user who last made a change to the object. The 3 columns of 1s and 0s represent the number of versions - the version number and the release number of that particular object. "Why do you need all those?" you are probably thinking.
Remember what we are trying to do here - to manage a bunch of functions that are in use in a variety of systems. When the utility developer upgrades/fixes a utility you don’t always get your system administrator to pull in the new version immediately. So, although you have fixed the bug, some of your users may well be running old versions. If you do not keep a copy of the old version you might get very confused when they re-discover the bug you have just fixed in 5 years’ time. JAD SMS allows you to maintain both old production versions of code and new development versions of code all in the same database directory.
By Way of Illustration ...¶
To illustrate let’s assume I have need to fix the top-level function OPEN. After editing, JAD SMS reports the following for that function ...
There are now two versions of the function (I elected not to overwrite the original but create a new version). The current version is 2. (These are always the same unless you delete old/unwanted versions - version numbering is always absolute.) The previous version is kept unless you elect to overwrite - and is read only for all versions bar the most recent. You can easily scroll between versions to compare any changes etc.
The release number is still 0*. What does the * mean ? Well - the release numbering scheme is a way of protecting versions of code that you have released to your application developers. Once you release a set of utility code from JAD SMS you apply a release number and all the latest versions of code get the * removed and the new release number is applied (unless it already is released status in which case it would already have had its * removed and it would keep its original release number). Once the latest version of an object is released it cannot be changed any more. To change the function definition you must create a new version of the function. Application developers must then retrieve either the latest versions of the functions (if they are developing new code) or the latest releases of the functions (if they are updating production systems).
To show this in action I applied a release number of 0.1 to all my printer functions (I would then communicate to all teams that the latest release is 0.1). This clears all the *s from the latest versions of each object (indicating that they can no longer be edited). To check this I then edited OPEN again. When I saved it, JAD SMS automatically created a new version and re-added the * to the release number (to indicate that it is an unreleased object).
Name Type Date Time Class # of Ver Rel User
Ver # #
CLOSE f 96-02-29 21:16 1 1 0.1 JS
OPEN f 96-03-06 20:26 3 3 0.1* JS
Notice it is still part of the 0.1 release set - that is because JAD SMS always applies the latest release number to any new versions - and because I hadn’t yet defined the next release number it used the most recent existing one. I’m not sure I like the way it does this - I think it is confusing. To get the functionality I was expecting you always need to create a development release level that gets applied to all changes following the last release you did.
In JAD SMS applying a release protects the latest versions of all code - but allows you to create new versions of code that still get attached to that release number. Hence application developers might get different sets of code depending on when they pull in the latest release of the code. It would be better if code changes after a release were flagged as ‘unreleased’ until the code is next released. Thus developers have the choice of pulling in the released set of code or alternatively all the latest versions of the code if they want the development set. In reality it does this with the * - the latest released code is the highest releasenumber/version combination that does not have an *. (But this is a bit too confusing!)
The JAD SMS version/release mechanism illustrates that you need quite a complex method of tracing utility code effectively.
Associating Objects¶
The final aspect of JAD SMS I’m going to talk about (and I’m skipping over quite a lot of functionality) is how you associate objects with each other to ease retrieval. Going back to my PRINTER example: the OPEN function had quite a complex calling tree and some of the lower functions had calling trees themselves. When a developer retrieves OPEN he would ideally like the option to retrieve all the required sub-functions to make that function work - rather than having to call the function 20 times until he has copied all the required functions identified by 20 VALUE ERRORs!
JAD SMS provides a method of achieving this by defining associated objects. Each object has an optional list of associated objects - these are objects that will also be retrieved/printed if the base object is selected. The association lists have to be set up manually - so if you add a call to a new function in an object you need to remember to add it to the association list. Some object management systems provide association automatically by lexically analysing the function every time you save it/retrieve it.
There are other aspects of the JAD SMS method of associating that are fairly crude. When you retrieve an object (whatever version) which has associated objects you only ever get the latest versions of the associated objects - this could lead to inconsistencies - in reality it ought to search for the latest version from the same release as the retrieved object. Also associated objects need to reside in the same directory - if you look back at the calling tree for OPEN it called functions that resided in all three of the directories that I defined. Fortunately JAD SMS provides a second mechanism for this ...
Object linking allows you to reference objects in multiple directories whilst only really storing them once. Thus, going back to the OPEN function again, we need all the functions it calls linked into the PRINTER directory. To do this you have to highlight the functions you want to link to the PRINTER directory in their source directory, and then press <F5> and select ‘link to another directory’. (I then tried changing the source of one of the linked functions and the link appears to be to the latest version always rather than to a specific version - again this could cause inconsistencies?)
I was now in a position to associate all the sub functions of OPEN - I did this - a simple case of typing in all the names on the association screen that comes with every object/version. (This isn’t validated so typing errors could be costly!) My PRINTER directory now looked like (first page only) ...
TEST.PRINTER Directory OPEN edited
Esc Cancel F1 Help F2 Add F3 OK F4 Search F5 Action F6 Restore
F7 Sort F8 Print F9 Attribs F10 Select F11 Cmpre F12 Tree Pg ↑/↓
1 of 18 JS Keys: ALT+A..Z, B C D E H L O P R S
Name Type Date Time Class # of Ver Rel User
Ver # #
CHK∆LPTS f 96-02-29 21:16 1 1 0.1 JS
CLOSE f 96-02-29 21:16 1 1 0.1 JS
+OPEN f 96-03-07 20:46 4 4 0.3* JS
OPTIONS f 96-02-29 21:16 1 1 0.1 JS
∆PDESC v 96-02-29 21:16 1 1 0.1 JS
∆PR f 96-02-29 21:16 1 1 0.1 JS
∆PTYPE v 96-02-29 21:16 1 1 0.1 JS
∆SETUP f 96-02-29 21:16 1 1 0.1 JS
⍙⍙⍙prdest v 96-02-29 21:16 1 1 0.1 JS
⍙⍙⍙preset v 96-02-29 21:16 1 1 0.1 JS
CAT f l 96-03-07 22:39 2 2 0* JS
IF f l 96-02-29 21:12 1 1 0* JS
NEXIST f l 96-02-29 21:12 1 1 0* JS
∆DOIF f l 96-02-29 21:12 1 1 0* JS
∆IOTA f l 96-02-29 21:12 1 1 0* JS
Classes: none defined
Notice the ‘+’ beside OPEN - this indicates that there are associated objects. Notice the ‘l’ against the functions towards the bottom of the screen - these are the objects I linked in from other directories - they are read only when viewed from this directory - but notice that CAT has had its version incremented even though I originally linked version 1 of CAT. I subsequently created a new version of CAT just to see what happened to the linked version.
To illustrate (a) the power of some of the low-level functions provided with JAD SMS and also the point of all this messing about ....
)CLEAR
CLEAR WS
)FNS
)COPY 2 SMS23
SAVED 04/12/1995 15:38:37
)FNS
jcompare jdelete jfile jprint jselect jsms jsurrog
jcopy jedit jget jretrieve jsetup jstore
'+A' jretrieve 'PRINTER; OPEN'
11 objects retrieved
OPEN CAT IF CHK∆LPTS NEXIST WIN∆PICK WIN∆UNDO ∆DOIF
∆IOTA ∆SETUP ∆TAB
The ‘+a’ in the left argument requested all the associated functions - you can see that this worked fine - but some of these functions had sub-functions as well - to pull back all these I would have to repeat the process I went through for OPEN. This would be very time-consuming and difficult to maintain for a large utility set. However, in the list of possible to extensions to the system there is a mention of being able to retrieve called functions automatically - this would remove the need for typing in all the associated function names. (Although I expect you’d still need to do the object linking bit?)
Summary¶
JAD SMS provides a sound multi-user environment for sharing multiple versions of utility code and allowing multi-user development and retrieval. It also provides a good method of defining the status of code via the release codes and status symbol (*). There are many features I haven’t reviewed - printing, code searching, multi-user access and function creation, and the lower-level functions that are available. The lower-level functions could be embedded within your workspaces to automatically retrieve latest versions of utility code. Also developers can use them to automatically add their own changes to utility code. The full screen interface was reasonably good and the online help also pretty good. The only real gripe I had is that I found the manual exceptionally difficult to dip into - you really have to read the whole thing from cover to cover and even then I found it very difficult to look things up quickly.
JAD SMS is currently being ported to Dyalog APL’s GUI world. A limited-functionality prototype will be available this summer. The main visual difference is that the first two screens shown in this article will be combined in the style of Windows File Manager/Explorer. Further information can be requested via email to the author of JAD SMS or mail the address for JAD Software in the product guide.