PATTIE: a Practical Expert System(2)
Vector 1:4, page 98
Transcribed from the printed issue; not yet reviewed. Read it in the PDF of the issue, page 98
Part 2
Ed: The first part of Christopher’s paper can be found in VECTOR Vol.1 No.3.
Problem Areas Highlighted¶
Knowledge Engineering¶
One of the most interesting things to arise was the concept of Knowledge Engineering.
It is fair to say that most experts are reluctant to admit to the amount of knowledge they have. A second party is generally required to try to extract all the knowledge from the expert. This second party is the Knowledge Engineer.
We found this to be a highly rewarding task, although it was not particularly easy. The problem is that the expert does not know exactly what you are after. Some background knowledge of the expert’s domain is undoubtably of great use. As is the ability to “ferret out” information from the (willing) expert.
As well as acquiring the knowledge the Engineer must also organise the knowledge into a form that can be used by the system. This must always be in the Engineer’s mind as he ‘interrogates’ the expert. The Engineer must therefore have the ability to convert the ‘rough’ data he obtains from the expert to, possibly, rules that the system can understand and use.
On the user side of the system, it is the Knowledge Engineer’s responsibility to see that the system appears ‘comfortable’ to the human user and also to make the system’s inference processes understandable to him.
Knowledge Acquisition and Refining¶
An expert generally becomes more of an expert. The system should really be able to do the same. There must also be a means of changing and amending the knowledge held within the Knowledge Base since the knowledge held will rarely be perfect; there will be inconsistencies and it must be possible to correct these.
Ideally these functions should be able to be performed by the user rather than having to call on the services of the Knowledge Engineer. We were therefore aware that this facility had to be made as easy as possible to operate.
However, a new problem arose: the ‘power’ of an Expert System derives from the knowledge within the Knowledge Base, not from how the inferencing mechanism operates on that knowledge. There is a point of view here that says that one does not let the ‘naive’ user enter knowledge into the Knowledge Base. The reason: when we say that the power lies with the Knowledge Base, we mean with its structure. In the case of a rule-based system, it is the interaction of rules which is important. This implies that the user need have some knowledge of the structure of the Knowledge Base if he is to be able to alter the data held there to achieve anything near optimum performance from the system. This problem can be helped, however, by the inclusion of “knowledge about the knowledge”, or “meta-knowledge”, in the system. This is mentioned elsewhere herein.
A more far reaching, and powerful, feature of the Knowledge Refining process is that of the system being able to maintain and update its own Knowledge Base automatically, i.e., without the intervention of humans at all. Due to the continual updating of the knowledge held by the system, the Knowledge Base will become more complex. Various problems, such as duplication and even conflicts of knowledge will arise. Therefore, it is necessary that the system is able to detect, and correct, these problems. It should also be able to update itself when, during normal operation, it discovers some piece of information that it decides it should keep a note of permanently. But, how does it know when to add something to its knowledge and even then, how does it formulate the rule around which to store the information?
These problems will become more important as systems increase in size. Present systems are of the order of hundreds of rules but when Knowledge Bases hold 20,000 rules …..?
System and Implementation Language¶
Although, perhaps, it was not relevant to the operation of the system, the language that the system was written in provided problems which were most unwelcome.
We cannot understand why languages such as C and Pascal are popular for writing systems such as these. Obviously, it can be said that (most) languages have particular areas of application in which they are ‘better’ than other languages but what slowed this work up more than was appreciated was that so much time was spent worrying about implementation details of the language; “You can’t do that in C …..”. Languages (and their associated environments) exist which are far superior for a considerable number of problem areas, especially areas such as Expert Systems where “brain power” ought to be going into design problems not language implementation problems. Examples of such (fairly) widely available languages are Prolog, Lisp and APL. System writers who use one (or more) of these rarely move back to a ‘conventional’ language with any degree of enthusiasm!
The Operating System used was UNIX. There is a considerable amount of excitement presently about this system. A very complex file directory system had to be written for our Expert System. Compare this with, say APL’s component file system! The point is made that the Operating System should do the work not the application system designer.
Another area worthy of mention was the (comparatively) poor run-time debugging available for the design of this system. Again, a considerable amount of time was wasted taking care of details that could have been handled by good programming aids.
Languages are now being developed with Knowledge-Based Systems in mind. These will be most welcome.
Whilst on the question of language, and for the record, PATTIE consists of 6000 lines of C, that being 120K bytes of source code which translated into some 20K of 68000 object code.
Pattern Matching¶
One of the great restrictions of our system was that of the pattern matching of rule clauses. Again, due to the time constraint involved, in the time available it was only possible to include a simple matching mechanism. This means that the sytem would not be able to recognise (for the purposes of processing) that the following clauses were identical:
- i)
- IF the pipe site is buried
- IF the pipe site is buried underground
- ii)
- IF the pipe environment is near to a road
- IF the pipe environment is close to a road
When one considers this problem it is by no means simple. Our system merely compares the text strings “buried” and “buried underground” with each other and then says that they are not the same. As the second example shows it is not just a case of matching parts of strings.
One way of solving this problem that we are currently working on is that of using “meta-knowledge”. In the second example, it is the concept of “being in the vicinity of” that is important, not the actual wording used. A secondary level of “knowledge about knowledge” could be of considerable use here.
Another example of where this could be of some use is, for example, in the case of:
IF the pipe site is buried
IF the pipe site is not buried
At present, the system treats these two clauses as entirely distinct from each other. But, it should know that one is the ‘opposite’ of the other.
A further example, and another constraint, was the fact that our system has no ‘relational’ comparison ability. E.g. given the clause:
IF the pipe diameter is greater than 1 m
the system took “greater than 1 m” as a text string, as in previous examples. Thus if a comparison was made with the clause:
IF the pipe diameter is 4 m
the required result would not be obtained because the system would see no connection between the two clauses. This clearly depends on the type of information the sytem is handling but in our case it turned out to be a very important point. Not least because the system would ask the user what appeared to be daft questions, e.g.
At this point, the system does not appear to be an ‘Intelligent’ Knowledge-Based System at all!
Calculations¶
Whilst on the point of the system holding numeric information; the system could, at certain stages of the processing, perform calculations based upon information gained up to that stage. The results of these calculations (on, say, pressures in pipes) could then be used to answer further system queries and/or be returned to the user. There would appear to be great scope here for the system to ask for certain information and then perform quite complicated mathematical calculations.
User Friendliness¶
The help facility was vital. It is all too common for systems to refer the user to a manual. We found that heavy use was made of this facility.
Also important is the general view the user has of the whole system. Items of relevance in this area are now mentioned.
It was considered that the system operating in terms of ‘I’ and ‘You’ (as mentioned previously) was an interesting approach that proved popular.
A strong criticism made was that the user is asked a lot of questions without actually knowing why they were asked. The inclusion of “I’m now going to ask you a few questions about …..” would be helpful. This may aid the user to accept what may appear to be ‘odd’ questions. Although the user always had full use of the Explanation Facility, it was considered better for the system to take the initiative and tell the user something rather than the user having to ask.
Another criticism was that all results from the system were presented “all in one go” at the end of the session with the system. This is a debatable point and depends upon the domain in question, the preferences of the user and the structure of the information returned.
The concept of ‘noise’ words was a quite successful idea which made the text output to the user much more readable. This point emphasises how important we feel the user’s view of the system is.
Miscellaneous Areas¶
It was found that the operation of the system was fairly slow at times due to the vast amount of rule comparison (and its associated file accessing). Our system had about thirty rules; had it had of the order of hundreds, it would have been unusable. Not until the system is re-written and further tests carried out will we be able to say whether this is a major problem. Up to a point, we consider that slow speed of operation is not all that bad. There is something un-nerving about a computer that comes up with an answer too quickly. Especially in the case of Expert Systems it is felt that the system taking a few seconds over a question would instil more credibility in the belief attached to the answer.
Our system allowed the user to reply DON’T KNOW to a question but then did not actually do anything with the parts of the processing that were thus avoided. DON’T KNOW had to be taken as an answer of NO since it would not have been safe to proceed as if the reply had been YES. In this sense it was merely an additional aid to the user interface. It would be a good idea for this to affect the processing in some less drastic manner than to stop it altogether in the affected areas, i.e. to introduce some degree of fuzziness.
Our initial realisation that the Explanation Facility was important was re-emphasised. It is vital. Again, the question of credibility is raised. The system will have none unless the user can see exactly what the system did in reaching the conclusions it did. The extra work required in this area was well worth the effort.
It has been shown that the knowledge that an expert has is largely heuristic, i.e., good guesses and good practice rather than fact and rigour. This usually finds its way into how the system processes knowledge and this leads to the system needing to handle uncertain knowledge or information with some degree of confidence. We simplified the design of our system by choosing rules from a domain that did not require this kind of thought. This idea is however an area of research and debate and almost certainly (sic) needs to be taken into account in most system implementations.
Due to the way our system was designed it was found that the user would have to know a certain amount about the knowledge that the system had before he could use it. Careful thought has to be given to just where the dividing line is between the system being all-knowledgeable (in its own area) and the user having, to an extent, ‘previous’ knowledge. The domain expertise in question will be a deciding factor. In our case however, it did prove to be a problem.
Our system is specific to the domain of pipes and, indeed, was designed around rules about pipes. We did this to gain experience in building an Expert System. Our next system will be designed with the concept of a ‘shell’ in mind. That is, a system not specific to any one particular domain. This is important if systems are not to be re-written to some extent before being applied to a different subject area.
Conclusions¶
The aims of the project were more than satisfied. Developing our own system has been an invaluable experience and has highlighted the problem in many fascinating areas in which research is being pursued.
We can see a very exciting future for Intelligent Knowledge-Based Systems of one form or another. The number of these systems in operation can only multiply many fold over the coming years.
Specifically on our system, we have learnt in the best way; encountering problems as we went along. The final system is by no means an effective solution to the original design objectives. We view the system as a prototype. It requires redesign, bearing in mind what we have now learnt, and rewriting. We intend to do this in two languages, Prolog and APL.
In summary, we can say of specific areas:
Knowledge Engineering — This is one of the most interesting, fascinating and important aspects of the whole exercise. If this stage is not perfomed correctly then the system is almost certainly doomed to fail.
Knowledge Acquisition and Refining — Very important if the system is not going to remain ‘static’. An Expert System must be able to grow and to learn. For the system to be able to do this automatically …
Explanation Facility — Absolutely vital. Without this, it will have no credibility whatsoever in the eyes of the user.
Knowledge Representation — There are many ways of storing knowledge and many associated inferencing mechanisms. Considerable research is being done in these areas.
Natural Language Interface — At one end of the scale is a simple Question/Answer system, at the other is free-format speech input. This part of the system must not be considered an ancillary add-on. Whether the system will be actually used or not could depend upon this interface. The most important part of the system is the user.
On the whole, a thoroughly worthwhile experience and one worth recommending.