Problem reading large VIP 5.2 database file as text strings
Posted: 19 Jan 2017 18:05
A simple routine to read and write some saved facts from a 20,000 KB VIP 5.2 database into another file stops unexpectedly in the middle of the data source file. I assume it is at an endOfStream condition. I looked at the file in both Notepad++ and a hex editor and there are no spurious control characters in the region where it stops reading. I opened the database file in NNP, saved it as a plain text file, did a text comparison using CodeCompare and the files are identical.
Then I had the 5.2 application purge some redundant facts from "above" the break point and the 7.5 read clause still stops a the same line, but at a different point in the 5.2 database file. The file position of the break is 15,494,135, at the 285,435th line.
Maybe there's something strange about the file that I'm reading as lines of string data. If it were a memory issue then the stopping point (last record read) would change after the purge, but it didn't.
I haven't tried having my 5.2 application export this data yet because I'd really like to understand this 7.5 problem.
The data around the breakpoint looks like this with the last line successfully read indicated:
A test predicate to copy the 5.2 database file lines to another file is:
I experimented with file5x clauses and had the same result. I've done this sort of thing with different file types for many years and have to admit that I'm baffled on this one!
Then I had the 5.2 application purge some redundant facts from "above" the break point and the 7.5 read clause still stops a the same line, but at a different point in the 5.2 database file. The file position of the break is 15,494,135, at the 285,435th line.
Maybe there's something strange about the file that I'm reading as lines of string data. If it were a memory issue then the stopping point (last record read) would change after the purge, but it didn't.
I haven't tried having my 5.2 application export this data yet because I'd really like to understand this 7.5 problem.
The data around the breakpoint looks like this with the last line successfully read indicated:
Code: Select all
dr(1603151030,"03/15/2016","10:30:00",35.16,2,305,2,1010,46.4,44.4,-99.9)
dr(1603151036,"03/15/2016","10:36:00",36.02,1,301,3,1010.1,46.6,44.4,100)
dr(1603151042,"03/15/2016","10:42:00",36.02,1,281,3,1010.1,46.4,44.4,-99.9)
dr(1603151048,"03/15/2016","10:48:00",37.32,1,312,2,1010,46.5,44.4,-99.9)
dr(1603151054,"03/15/2016","10:54:00",37.36,0,326,1,1009.9,46.8,44.4,-99.9) % <== last line successfully read
dr(1603151100,"03/15/2016","11:00:00",37.13,0,159,1,1009.8,46.9,44.4,-99.9)
dr(1603151106,"03/15/2016","11:06:00",37.13,1,317,1,1009.9,47,44.4,-99.9)
dr(1603151112,"03/15/2016","11:12:00",36.69,1,286,2,1010.1,46.8,44.4,-99.9)
dr(1603151118,"03/15/2016","11:18:00",36.5,1,302,2,1010.3,45.7,44.4,-99.9)
dr(1603151124,"03/15/2016","11:24:00",36.69,2,335,3,1010.3,44.5,44.4,-99.9)
Code: Select all
class predicates
copyData : ().
clauses
copyData():-
file::existExactFile( dataFile52 ),
stdio::write( "\nREADING FILE ... " ),
_ = vpi::processEvents(),
IS = inputstream_file::openFile8( dataFile52 ),
OS = outputStream_file::create8( "TEST.FILE" ),
IS:repeatToEndOfStream(),
S = IS:readLine(),
frontToken(S,_Tok,_),
OS:write(S,"\n"),
IS:endOfStream(),
!,
OS:close(),
IS:close(),
stdio::write("\nSUCCESS ", predicate_fullname() ).
copyData():-
stdio::write( "\nFAILED ", predicate_fullname() ).