CLIPS: A Tool for Building Expert Systems
clipsrules.net
clipsrules.net
> A fact such as (duck) or (quack) is said to consist of a single field [...] As a simple analogy, you can think of a field as a picture frame. The frame can hold a picture, perhaps a picture of your pet duck (For those of you who are curious what a picture of a “quack” looks like, it could be (1) a photo of an oscilloscope trace of a duck saying “quack”, where the signal input comes from a microphone, or (2) for those of you who are more scientifically inclined, a Fast Fourier Transform of the “quack” signal, or (3) a TV-huckster selling a miracle cure for wrinkles, losing weight, etc.).
There are a few moments in my life as a young programmer that I remember as mind-expanding. CLIPS was one of those moments (along with arrays, files and recursion).
It's forward-chaining (not backward, as Prolog is) so you can see the database of facts build up step by step. A nice bridging point between procedural and declarative programming.
And a foot in the door for parens-based syntax (it looks a bit like LISP) without the s-expression model.
If I recall correctly, a summer intern in the AI section under the Mission Planning Directorate, whose name I have forgotten, developed a prototype or a proof-of-concept of CLIPS. At the time, 1984-ish, the AI group at JSC had many expensive Symbolics and Lisp Machine computers running ART. The intern's project showed that a pattern matching rules based engine could be developed in a conventional programing language for distribution to traditional cost effective hardware.
In 1984, C was considered a new programming language. The VAX first came out in 1977, and the Sun 1 in 1982. Traditional hardware in 1984 would be an IBM mainframe.
It is used in medicine as a diagnosis tool: https://pdfs.semanticscholar.org/64c0/672703724bd32e04b94c10... http://www.jatit.org/volumes/research-papers/Vol4No10/5Vol4N...
In software security for malware detection and categorization: https://ieeexplore.ieee.org/abstract/document/4724664
In gaming for driving AI: https://www.youtube.com/watch?v=-S1CkfzEHSU&t=489s
And plenty of other fields.
It's a very good solution for complex problems where a large amount of knowledge or expertise is required in order to reach a solution. Its documentation and REPL are excellent for those who want to approach fundamentals of AI by solving concrete problems.
It also comes with several language bindings allowing to integrate it with other AI/ML platforms such as tensorflow, numpy etc.. In the past I helped few students which were trying to bring together machine learning models and expert systems to benefit from both worlds.
More recently, I was tempted to make a tool for the designers on Storm Casters to specify game and meta-game logic like this, but had to abandon that due to higher priorities.
I didn’t do much with Clips but OPS5 was a large part of my working life for years (supported commercial versions on Macintosh and Xerox Lisp Machines, extended the Rete algorithm to support multiple worlds for an internal project).
Given that ART was written in Common Lisp and exposed a full Common Lisp inside, it would also be trivial to get large amounts of information about its implementation (data structures, functions, arglists, documentation, traces, macro expansions, disassemblies, ...). I'm pretty sure ART itself had extensive debugging&tracing tools for the rules engine.
Not having source to some Common Lisp software, says very little, when the full Common Lisp environment is available for introspection.
I wouldn't expect that ART was shipped in source to its customers. But I would expect that both ART and the underlying Lisp had excellent introspection capabilities.
http://clipsrules.sourceforge.net/WhatIsCLIPS.html
> The prototype version of CLIPS was developed in the spring of 1985 in a little over two months. Particular attention was given to making the tool compatible with expert systems under development at that time by the Artificial Intelligence Section. Thus, the syntax of CLIPS was made to very closely resemble the syntax of a subset of the ART expert system tool developed by Inference Corporation.
Since ART was what they were using.
> Although originally modelled from ART, CLIPS was developed entirely without assistance from Inference or access to the ART source code.
Sure, it wouldn't say anything else in their position.
I would guess to be compatible one would need to know both syntax and semantics (what the constructs are doing and how a rule system executes the rules) of the constructs of ART. Having access to documentation and actual implementations helps with that.
I never did say anything about source code or performance. I would also not think that people were stupid - especially since they were smart enough to develop expert systems.
Inference Corp was then Brightware, which then was bought by Firepond, then ???
The book "Expert Systems: Principles and Programming" [1] is also good though now out of print.
[0] http://www.clipsrules.net/Documentation.html [1] http://www.amazon.com/exec/obidos/tg/detail/-/0534384471/ref...
Is this not the case?
As for expert systems in terms of ai, I do think that's gone by the wayside, but probably not entirely gone (ditto for decision trees).
I'll have to look into Answer Set Programming more as I havn't seen it before. Thanks!
Not really, no. This whole industry is very fad driven, and there is a tendency to always aggressively jump on the "new shiny"... and once that happens, anything deemed "old" gets treated - by some members of the community - as though it is completely dead, obsolete, verboten, whatever. But rarely is that actually the case. Quite often "old" tools and techniques are quite useful for specific things.
As an example, the box-end wrench didn't become useless because the air-ratchet was invented. As another commentor said, "use the right tool for the job".
All of that said, sometimes the older techniques do legitimately have major shortcomings, or limitations, or may simply be superseded by better tools. As somebody else said, one of the biggest limitations to some older approaches to "expert systems" was making them scale. Some of that we got for free thanks to Moore's Law. In other cases, we may still be waiting for some new algorithmic breakthrough or whatever.
Anyway, things like CLIPS, OPS5, etc. still have uses, even if they didn't turn out to be the be-all end-all of AI as they may once have been perceived to be.
On the other hand, if you have a system of case statements or if thens that is a bit opaque (or very opaque) refactoring them to use an expert system is a very good idea.
And, if you have a homebrew knowledge store then using a rule system is also a good idea - but now-a-days you might want to try building a knowledge graph instead, mostly because that's v.trendy.
Hmmm. I think I just figured out what I'm going to do for my next game jam.
It seems to me that HN might not be too frequented hy people who work with symbolic AI, but expert systems and semantic tools are very, very deeply connected. In fact, semantic technologies are derived from work on expert systems, and using JESS with RDF store was part of my coursework at university.
Amazing!
Caveat first off: it's 7+ years since I looked at this, so take "modern" in that context. I used SWI Prolog specifically [1].
At that point, CLIPS and Prolog occupied similar spaces - but not the same. Prolog is backward chaining. So it's really good at answering questions that can be derived from a base set of facts. For example. Let's say:
There's a road from City A to City B, and a road from City B to City C.
It's then very easy in Prolog to say "there's a route from "Start" to "Finish" if either:
1. there's a road from Start to Finish, or
2. There's a road from Start to City X, and a route from X to Finish.
The beauty of prolog is that you don't need to explicitly code iteration over all the available roads; that's what the backward chaining algorithm does for you.
In arriving at the answer, it'll exhaustively test all permutations. Importantly though, it won't add to the list of facts you started with (unless you specifically tell it to). Once the question is answered, the intermediate results are discarded. In that sense, Prolog - at least in purest form - is somewhat stateless. Assert some facts, then ask questions about inferences that can be drawn from them.
CLIPS is different. It's a forward chaining engine, from a class of related products known as Production Systems. It's designed to be used where new facts are being asserted; when that happens, it figures out the consequences as they ripple through. As new facts are asserted - and their consequences computed - the underlying state evolves. So intermediate results are retained (in fact, that's key to operation).
CLIPS (and Production Systems in general) were used, for example, in medical diagnosis applications. As new symptoms were asserted, the rules rippled through, suggesting the most likely diagnosis.
The key to making this efficient is the forward chaining algorithm, known as Rete [2] (pronounced "Rait").
Again: these are examples, and the reality is likely much more nuanced now. But at least in their purest original form, CLIPS and Prolog are more complementary than overlapping.
[1] http://www.swi-prolog.org/ [2] https://en.wikipedia.org/wiki/Rete_algorithm
EDIT: minor grammar changes + clarified naming in Prolog example.
Why is it inherently better to use a bakeards chaining system under all circumstances?
Is the resurgence of expert systems finally a thing after statistics-all-the-things fatigue?!
Can't wait till marketing folks run with "3rd wave AI".
It looks like they were able to make expert systems useful.
Our work on KnowledgeTool might be equally useful!
After all, it appears that their work was stuck with C, and our work on KnowledgeTool was based on, was a pre-processor to -- sorry to mention this -- PL/I. But, gee, they based on C, so mentioning PL/I should not be so bad!
Our use of KnowledgeTool was to monitor server farms and networks. There in particular we were trying to do problem detection. So, in expert systems,
When I see A and B and
5 or more cases of C
without a D, then it
looks really bad --
raise an alarm.
And this statement of a symptom, test, for something wrong, was supposed to come from operator experience. So, broadly the idea of KnowledgeTool for that work was not to find better ways to monitor but just to encode what operators had already learned, mostly just from experience.
Hmm .... In such monitoring, we have two ways to be wrong, (1) a false alarm where we say that the system is sick when it is well and (2) a missed detection where we say that a system is well when it is sick.
So, we want the false alarm rate, to be called the probability of Type I error, and the missed detection rate, to be called the probability of Type II error.
Quite broadly there is a trade-off between these two rates: If we accept a higher false alarm rate, then we stand to get a lower missed detection rate.
Not all means of detection are equivalent: Some poor detectors have really high rates for both false alarms and missed detections, and some good detectors have really low rates on both of the errors.
A problem with expert systems was that we had no idea what rates we were getting or if our detectors were poor or good.
Commonly at least in principle we can adjust the rate of false alarms. That is good because (1) a false alarm sends the bridge staff on wild goose chasing and (2) false alarm rate too high is a standard complaint.
Looking at the problem in a little more detail, we easily had wide, deep rapidly flowing oceans of data. So, we could easily have data on 1000 variables with the data for each variable arriving at 100 times a second. So, write some expert system rules? No thanks!
So, I dreamed up a solution, basically, right, as guess from Type I/II, a statistical hypothesis test but one that is both distribution free and multi-dimensional, maybe the first such. False alarm rate can be set in advance of collecting data, and is in a useful sense for each selected false alarm rate has the detection rate the highest possible (for that false alarm rate).
I did this work to improve on expert systems for monitoring. Yes, I published the work, in Information Sciences.
Ah, beating expert systems! What's in the paper is some applied math based on some advanced pure math prerequisites, especially an idea from ergodic theory. What's in the paper really SHOULD be deployed. I suspect that more could be done.
Well, the OP starts with
> Developed at NASA’s Johnson Space Center from 1985 to 1996, the C Language Integrated Production System (CLIPS) is a rule-based programming language useful for creating expert systems and other programs where a heuristic solution is easier to implement and maintain than an algorithmic solution.
Gee, I thought that my "algorithmeic", really original mathematical statistics, was easier than doing as well with heuristics.
From you, too, we want to hear what utility you see in the paper!
"A Real-Time System-Adapted Anomaly Detector", Information Sciences, volume 115, pages 221-259.
The core math is some measure preserving transformations of the data -- get to move data around without changing probability distribution -- for any distribution. The usual place see measure preserving is in ergodic theory. Recall, my work is distribution free which means need make no assumptions at all about probability distributions. In the context of computer and network performance data, distribution-free is nearly essential. After moving the data around, get to do some counting that leads to calculating false alarm rate.
So the paper is some applied probability, that is, does some probability calculations in the service of some, call it, mathematical statistics.
The simplest view is that it's all just nearest neighbors detection -- if get a point too far from the old points, then raise an alarm. Since I wrote the paper, that idea has become common. But apparently what has not become common is how to adjust and calculate false alarm rate, and in practice that is really important. Indeed, for any detection, my work can report the lowest false alarm rate for which the (real time) observation would still be a detection -- so get to have not just a detection but, intuitively, a <i>seriousness</i> measure.
"Joe, this is a detection, and it will remain a detection with a false alarm rate of one in a billion. Put down your pizza, and let's investigate that." So, right, could display a strip chart with such data.
For calculating false alarm rate there is a simple approach that gives the right answer, but the simple approach makes an impossible assumption of independence. Well, the right answer is still true; it's just that an actual proof of false alarm rate, where don't assume independence, is a bit tricky mathematically.
This math is a close cousin of not independence but a weaker assumption exchangeability where often can get the same results as if had independence.
My math may be something of a rationalization of part of resampling as pursued by P. Diaconis and B. Efron at Stanford.
In an intuitive sense, we almost have independence. Hmm -- approximate independence is not a big field. There is a paper by M. Talagrand, A New Look at Independence that may be able to shed some light on my paper, find another way to do the calculations and show that the same calculations hold approximately, maybe with some bounds, in other situations. Talagrand is one heck of a good mathematician, a student of Choquet, a member of Bourbaki.
It would be nice to have the most powerful statistical hypothesis tests as from the classic result of Neyman-Pearson; alas, in the context we don't have enough data to use Neyman-Pearson. But I did come up with a weaker but still useful sense in which my techniques do yield the most powerful test. In practice, the test will likely be seen as nicely powerful, that is, even when false alarm rate has been selected to something small, the detection rate is still, likely in practice, about the highest can hope for for that selected false alarm rate.
There is an issue of how to make the computations fast -- so, need something out of computational geometry. I worked up a technique that should be reasonably fast. The solid state drives of today should do wonders for this technique.
The computational geometry and the most powerful test derivation are not in the paper.
There's more that can be done.
Might also use the work for other anomaly detection problems. One nice point is, although might be using lots of variables, don't encounter "over fitting".
What we were doing with expert systems was mostly just thresholds on one variable at a time. Well, in some cases, we know so much about the variable that that is okay. But there can be thousands of variables, collected by HP, Microsoft, etc., and we can't know just from personal knowledge or experience just what thresholds make good sense. And we have nothing solid on false alarm rate or how to adjust it -- yes, we could get an approximation by going back through the data. And, then we have still less on detection rate. If want a server farm that is reliable and secure, about the best you can hope for, then maybe my work would be one of the important tools and means.
This stuff about "multi-dimensional" is serious and important. E.g., suppose we have three threshold detectors. Then whether we wanted to or not, we just assumed that the 3D region of normal, healthy, not an anomaly or sick, performance is a 3D box. The box does not promise to fit reality well. Make the box too small, and get too many false alarms. Make the box bigger and can reduce the false alarm rate, but since we still have a box that does not fit reality well we are stuck with detection rate too low and a "poor" detector.
Well, can think a little and intuitively convince yourself that the region of normal (healthy) performance could be fractal, e.g., the Mandelbrot set, and still the math should work.
To me, for monitoring server farms and networks, my little paper totally blew out of the water with the doors blown off nearly everything we were doing with expert systems.
Let us all know what utility you find in the paper!
but there are no PDFs that I could download without payin 37.95 $USD at ( https://www.sciencedirect.com/science/article/pii/S002002559... )
I used K-means clustering in various domains (access control, fraud control, network data plane quality controls), so wanted to check your work.
One problematic area for K-means, at least from my point of view, is that it is not really possible to easily trace what features/variables are mostly responsible for the anomaly.
Explanation is a problem, also for my work.
Broadly, for a first cut, pick a target system want to monitor. Then use variables that are for the target, the whole target, nothing but the target or some such. Then if get a detection, start diagnosis, looking for cause, with the target.
For two targets that interact, have detectors for each and then one more for variables from both of them.
Broadly, but crudely, have a hierarchy of detectors and then when start getting detections chase down the tree of the hierarchy, like the patient is sick so, look at the major measures and the major organs, etc. When find a suspicious organ, zoom in, drill down, etc. Finally discover the problem is a USB cable that is loose from vibrations from a cooling fan or some such??
In this tree, might want to have somewhat coordinated false alarm rates.
One motivation for my work was a cluster for transactions. One day one computer in the cluster got a little sick in the head and was throwing away all its incoming transactions. So, to the load leveling it looked not very busy and was getting nearly all the transactions and, thus, essentially ruined the work of the whole cluster. And that's not the only case of a cluster getting totally sick due to just one computer in the cluster getting sick. So, I was hoping that getting data from all the computers in the cluster would raise an alarm from the cluster and data from each of the computers in the cluster would essentially do the diagnosis of which computer in the cluster was sick.
Likely more could be done. E.g., might want to do some data scaling.
I don't yet have a big server farm, and I don't know anyone who does who cares enough about anomaly detection to use my work. Expert systems? At one time, yup. My work? Nope. If my startup does well, then, sure, I'll deploy my detector, lots of instances.
My startup isn't anomaly detection.
At one time, I wrote lots of VCs about my anomaly detection work, that it was from IBM's Watson lab, that it was published in Information Sciences, that I had work on fast algorithms, that I had good results on real data, etc. I got back just nothing.
I guessed that the target customers would be high end shops so that I would have to come in with a highly polished product, with lots of good data handling utility tools, some first class hand holding, etc., all of which would be expensive before the first sale. So, I gave up on the idea that I could do a startup from my anomaly work. There might be a way; if I ran a large, really serious server farm, I have at least a little project pursuing my work and/or related work. But, apparently mostly that's not the way server farm management works.
So, I picked another problem for a startup, one where I could get a good solution and bring it to good revenue with just my own efforts as sole, solo founder. I did that and am about to go for an alpha test.
Somehow anomaly detection just doesn't get people very interested.
Once I gave a talk, and some in the audience mentioned that could use my work for fraud detection in, say, credit cards. Well, maybe, but that audience had nothing to do with credit cards.
At one time there were some Soviets in England watching some government offices and keeping track of lights, comings, goings, etc. They guessed that if a war was on the way, this data would show anomalies and early warning. Yup, the Soviets saw the broad issue. I don't know if they used anything like my work or not.
So, maybe the secret is not just some good work in applied probability with some useful results but publicity, hype, fads, group think, a movement, etc., even if what are selling is just total nonsense.
My work is on the shelves of the research libraries. I did my part. If people want anomaly detection, there's some good work there. I've told the Sand Hill Road people, the Hacker News audience, several organizations, e.g., the main NASDAQ server farm at Trumbull, CT, etc.
My experience is that people will start to act, react, if they have a problem that, like a tight shoe, really hurts, and then start to work on the problem. If some early work has the shoe not pinch so much, then they will f'get about that work and that problem and concentrate on something else, even if there's lots of money to be made in better solutions to the original problem.
There is a time lag problem: A lot of the math is on the shelves of the libraries, but only much more recently has suitable computing been available. Only a tiny fraction of people in computing now, where the computing has the potential, know that old math that's been sitting there waiting on the computing.
E.g., when I first did my work on anomaly detection, some people said that a lot of computing would be needed. Well, yup. The response was so obvious I didn't have to make it: The needed computing was coming. Well, now, say, with solid state disks, big server farms, lots of system monitoring data gathering, etc., now the time is right for actual usage. But, nope, interest is really low!
The startup I'm pursuing now is much more promising. I don't have to get a few bureaucratic, cautious, conservative, CIOs all fired up. Instead I just have to please a lot of Internet users a little bit each, and that should be much easier.
The CIOs are running big server farms, and they are working. So, they don't have a shoe that pinches very much. So, they are not motivated to do anything new or different. Okay, lesson learned.