Page 1 of 1

Compiled clauses

Posted: 19 Jul 2019 12:31
by Rangarajan
Hi,
I am new to Visual Prolog. It is going to take me a while to get used to the strong typing, etc. I have a quick question: If I have many "Clauses" files, each containing hundreds of clauses (the standard Prolog clauses), is there way to load them all and save the combined set as "compiled" binary file? The idea is that the compiled binary file can be "restored" much faster compared to "consulting" the files. Is this possible in VP 9?

Regards,
Rangarajan

Re: Compiled clauses

Posted: 19 Jul 2019 15:24
by Martin Meyer
Hi Rangarajan,

short answer is: It depends on whether they are fact or predicate clauses.


A longer answer:

A strength of Visual Prolog compared to other Prolog systems is speed. The main reason, why VIP programs are running fast, is, that VIP compiles the program code to executable machine instructions. Thus it does not need to interprete the program code at run time unlike most other Prolog systems.

The VIP compiler performs its work however solely before you execute the program. It cannot be used to add clauses of predicates at runtime. In that sense the answer to your question is "no".

However you can write and read facts at run time to/from binary files. Hence regarding (the clauses of) facts the answer is "yes".

For a very basic example create a new console application and paste this code into main.pro:

Code: Select all

implement main   domains     bind = bind(string Name, integer Value).     environment = bind*.   class facts     myFact : (environment Environment) nondeterm.   clauses     myFact([bind("X", 7), bind("Y", 13)]).     myFact([bind("A", 203), bind("b", -173)]).   class predicates     showMyFacts : (). clauses     showMyFacts() :-         foreach myFact(Environment) do             stdIO::write(Environment, '\n')         end foreach,         stdIO::write("---\n\n").   constants     myFileName : string = "myFile".     myFileMode : stream::mode = stream::binary.   clauses     run() :- %--- % Write the facts to a file         OutStream = outputStream_file::create(myFileName, myFileMode),         foreach myFact(Environment) do             OutStream:write(Environment)         end foreach,         OutStream:close(),         showMyFacts(), %--- % Clear the facts database         retractAll(myFact(_)),         showMyFacts(), %--- % Read the facts from the file         InStream = inputStream_file::openFile(myFileName, myFileMode),         foreach InStream:repeatToEndOfStream() do             Environment = InStream:read(),             assert(myFact(Environment))         end foreach,         InStream:close(),         showMyFacts().   end implement main   goal     console::runUtf8(main::run).

Re: Compiled clauses

Posted: 20 Jul 2019 10:06
by Rangarajan
Thanks Martin!

Assuming I have a text file that contains just "facts" (something like your "myFact"), I would like to load them all into the program, save them in a binary format, and then when needed, load them all in one go. In your example, I see that you are saving and reading the facts one by one.

In SICStus Prolog, I can "consult" several Prolog fact files, and then "save" them (all combined) as a single "image" file. Then, I can read back this image file using "restore". This is usually much faster than processing individual text files. I am wondering if there is something similar in ViP. Maybe internally, "save" and "restore" do what you are doing, but I want to be sure that this is done as efficiently as possible to get good performance.

Regards,
Rangarajan

Re: Compiled clauses

Posted: 20 Jul 2019 14:29
by Martin Meyer
Yes, there are save and consult predicates in VIP. They are not first language constructs but part of the PFC ("Prolog Foundation Classes").

The PFC comes with a help file. If you have installed Visual Prolog to folder
"C:\Program Files (x86)\Visual Prolog 9 PE\" then the help file will be
"C:\Program Files (x86)\Visual Prolog 9 PE\appData\doc\vip.chm". You find the save and consult predicates (as well as the write and read predicates) in the outputStream and inputStream interfaces.

A variation of the example which uses save and consult is:

Code: Select all

implement main   domains     bind = bind(string Name, integer Value).     environment = bind*.   class facts - myFacts     myFact : (environment Environment) nondeterm.   clauses     myFact([bind("X", 7), bind("Y", 13)]).     myFact([bind("A", 203), bind("b", -173)]).   class predicates     showMyFacts : (). clauses     showMyFacts() :-         foreach myFact(Environment) do             stdIO::write(Environment, '\n')         end foreach,         stdIO::write("---\n\n").   constants     myFileName : string = "myFile.txt".   clauses     run() :- %--- % Write the facts to a file         OutStream = outputStream_file::create8(myFileName),         OutStream:save(myFacts),         OutStream:close(),         showMyFacts(), %--- % Clear the facts database         retractAll(myFact(_)),         showMyFacts(), %--- % Read the facts from the file         InStream = inputStream_file::openFile8(myFileName),         InStream:consult(myFacts),         InStream:close(),         showMyFacts().   end implement main   goal     console::runUtf8(main::run).
When you have a look into the generated file myFile.txt, you get an example of the clauses' syntax which is required when you are loading your clauses by consult:

Code: Select all

clauses myFact([bind("X",7),bind("Y",13)]). myFact([bind("A",203),bind("b",-173)]).
save and consult are but only working with text files. I.e., you cannot write to a binary file by save.

The VIP compiler produces fast machine code. The performance of I/O is generally dominated by the speed of the I/O devices (i.e., the speed of your hard disk or SSD). The performance difference (if there is one at all) of using a built-in (resp. PFC) predicate or a user programmed loop is completely neglectable.

Re: Compiled clauses

Posted: 20 Jul 2019 14:43
by Harrison Pratt
Rangarajan,
In most cases you should not need to resort to binary manipulations to read/write facts from/to disk storage. The consult/2 and save/2 predicates are remarkably fast.
And, yes, you can save, for example, one set of database facts in one file, another set in another file and then consult them cumulatively into the same database. You can try the code below in a console application by editing (copy/paste) it into main.pro.

Harrison

Code: Select all

implement main     open core   clauses     run() :-         test(),         succeed. % place your own code here   class facts - eavDB     eav : ( string EntityTag, string AtributeTag, value Value ). % see core.cl in PFC directory for definition of value   class predicates     test:(). clauses test():-     ListDB = { ():- stdio::nl, foreach eav(E,A,V) do stdio::write("\n", E, "\t",A,"\t",V) end foreach},     assert( eav("Person1","Name",string("George Washington"))),     assert(eav("Person1","AgeInYr",real(65.2))),     ListDB(),     file::save("GeoWash.db",eavDB),     retractFactDb(eavDB),     assert( eav("Person2","Name",string("Marth Washington"))),     assert(eav("Person2","AgeInYr",real(62.2))),     assert(eav("Person2","Sex",string("F"))),     ListDB(),     file::save("MarthaWash.db",eavDB),     retractFactDb(eavDB),     file::consult("GeoWash.db",eavDB),     ListDB(),     file::consult("MarthaWash.db",eavDB),     ListDB(),     _=stdio::readLine().   end implement main   goal     console::runUtf8(main::run).

Re: Compiled clauses

Posted: 22 Jul 2019 12:33
by Paul Cerkez
Rangarajan,
While I haven't been using VIP 9 (yet!), in earlier releases of VIP, I was quite literally using a few thousand facts, reading (consult) and writing (save) them in a couple of my applications. In each case, these facts were spread across multiple different files.

There was no noticeable speed impacts (except the usual OS issues mentioned by others).

Re: Compiled clauses

Posted: 22 Jul 2019 15:01
by Harrison Pratt
I have the same experience with 20,000+ facts in a consultable database. Speed optimization of database access via consult/2 should be deferred until it becomes an actual problem to be solved.

Re: Compiled clauses

Posted: 25 Jul 2019 11:30
by Rangarajan
Thanks to all who have responded. I get the idea.

One reason why one might want to store the facts/clauses in a binary format is to protect IP. Suppose these facts have sensitive information (e.g. business logic related data) that I don't want the end user to know, I would not want to keep it in a text format. Of course, even if it is compiled but not encrypted, another Visual Prolog developer might be able to use it, but that is a different issue.

Regards,
Rangarajan

Re: Compiled clauses

Posted: 25 Jul 2019 20:12
by Harrison Pratt
Nothing in VP prevents you from storing your facts in an encrypted form (e.g., as binary structures, scrambled strings, etc.) in an internal database which can be saved and consulted. This thread could have been much shorter had you told us in your original post that your concern was intellectual property protection.

Re: Compiled clauses

Posted: 28 Jul 2019 13:45
by Rangarajan
Hi,
I meant no offence in my post. What I asked first was a genuine question. There was no need to disguise my intentions! The follow up was just another use case for binary representation.

Regards,
Rangarajan

Re: Compiled clauses

Posted: 5 Aug 2019 13:56
by Thomas Linder Puls
When going from one language to another, you may of course consider how things you faced in the old language is treated in the new one. But often you will find that the "things" in the new language is not the same as those in the old language.

You should notice that Visual Prolog is a fully compiled language. So your clauses will be compiled into a binary executable program.

In ancient time SICStus Prolog (and other Edinburgh/ISO Prolog variants) have clauses that can be asserted/retracted/saved/consulted/etc during runtime. So the program can be updated dynamically during runtime. The clauses are treated as data that can be updated at runtime. This is for example used to deal with knowledge bases. I.e. your knowledge base is represented as clauses which is dynamically asserted/retracted/saved/consulted/etc.

The dynamic behavior of ISO Prolog comes at a certain cost. Namely that it may be too expensive (and/or difficult) to compile and/or optimize the clauses if they often change. Furthermore even if you use the dynamic features for knowledge bases and the like, 98% (figuratively speaking) of your predicates will never be changed dynamically; these predicates are a static part of your program which you have spend months/years developing, debugging, etc. You don't need (or even want) them to update dynamically. On the other hand, you want these predicates to perform at best possible speed.

So at some time in the history of ISO Prolog, it was made possible to have non-dynamic predicates. This made it possible/easier for ISO Prolog languages like SICStus to deal with optimizations and compilation.

I guess it is this feature you are referring to.

Visual Prolog is quite different in this respect, a Visual Prolog program is the 98% static part of your program, and this part is optimized in many ways and fully compiled into an executable file (exe or dll).

So your question is somewhat incompatible with Visual Prolog so to speak. The "really good" question in this respect is not about the 98% static code in your program it is about the 2% dynamic code.

Dealing with those 2% can range from trivial to very complicated.

It is trivial if your 2% is just what we call facts (i.e. a grounded/variable free clause without body). Such facts can be asserted/retracted/saved/consulted/etc during runtime, and as the other users mention above save and consult does not pose any significant performance problem. At runtime they are handled linearly and that can be a source of algorithmic inefficiency (I do not know whether SICStus have a more efficient behavior in this respect).

It can be very complicated if your 2% contains "real clauses" with variables and bodies, because then you will have to implement something which eliminates this need. And that may be very complicated.

Finally, I have a little twofold general advice on performance:
  • Do not solve fictive performance problems that you don't actually have.
  • Instead learn to consider algorithmic complexity of your code and use efficient data structures by default.
The first part of the advice both has to do with not wasting your time on solving non existing problems and what is even worse make your code unnecessarily complex and hard to maintain.

The second part of the advice has to do with the real source of (software) performance problems algorithmic complexity. While you can possibly make a program/algorithm 20% more efficient by "fiddling" with the code, nobody will really be happy because 20% is almost always "nothing". When you have a real performance problem you will normally need at least to make the code twice as efficient but more likely 10 times as efficient. Such an efficiency improvement cannot be solved by "fiddling", it will require a reduction of the algorithms complexity.

Facts as mentioned above are arranged linearly and that can easily become a source of inefficiency. Those parts of your code which need to traverse all your data will not suffer from this, but those parts that need to access a single piece of data will/may suffer dramatically because in average you will have to search through half your data to find a certain piece.

Therefore it is a good idea to (by default) consider access to your data:
  • Does this code search linearly for a piece of data among a (potentially) large amount of data
  • Should I use a more efficient data representation (e.g. a map from the collection library) for this kind of information
  • ...
If you do a good job in this respect you will by default make your code perform much better without decreasing the readability of it. The downside is that when you then do get performance problems they will be much harder to solve, because you have already done what will normally help ;-).