Page 1 of 1

openFileBOM/1 and extra empty strings on readLine()

Posted: 21 May 2018 20:48
by Harrison Pratt
From the documentation, I expect openFileBOM/1 should behave the same as openFile8/1. However, when reading a simple text file with no byte-order marks successive readLine() statements return text and an empty string when the file is opened with openFileBOM/1.

Is this how openFileBOM/1 should behave?

Code: Select all

Output using openFileBOM(FN): zero   one   two   three   Output using openFile8(FN): zero one two three
The output above is generated by this test code:

Code: Select all

    test() :-         FN = @"SimpeTestFile.txt",         In1 = inputStream_file::openFileBom(FN),         stdio::write("\n\nOutput using openFileBOM(FN):"),         In1:repeatToEndOfStream(),         stdio::write("\n", In1:readLine()),         In1:endOfStream(),         In1:close(),         In2 = inputStream_file::openFile8(FN),         stdio::write("\n\nOutput using openFile8(FN):"),         In2:repeatToEndOfStream(),         stdio::write("\n", In2:readLine()),         In2:endOfStream(),         !,         In2:close().     test().

Re: openFileBOM/1 and extra empty strings on readLine()

Posted: 22 May 2018 8:57
by Thomas Linder Puls
I have not been able to reproduce this problem, can you attach the input file you use?
Which Visual Prolog version do you use for this.

(Also notice that repeatToEndOfStream works particularly well with foreach:

Code: Select all

clauses     run() :-         FN = @"..\test.txt",         In1 = inputStream_file::openFileBom(FN),         stdio::write("\n\nOutput using openFileBOM(FN):"),         foreach In1:repeatToEndOfStream() do             stdio::write("\n", In1:readLine())         end foreach,         In1:close(),         In2 = inputStream_file::openFile8(FN),         stdio::write("\n\nOutput using openFile8(FN):"),         foreach In2:repeatToEndOfStream() do             stdio::write("\n", In2:readLine())         end foreach,         In2:close().
)

Re: openFileBOM/1 and extra empty strings on readLine()

Posted: 22 May 2018 10:51
by Harrison Pratt
The test file I used is attached.

I am using VP 802.

Re: openFileBOM/1 and extra empty strings on readLine()

Posted: 22 May 2018 11:21
by Thomas Linder Puls
If I had used the correct version I would have reproduced it.

The openFileBom should look like this (last call is added):

Code: Select all

clauses     openFileBom(Filename, Access) :-         openFile(Filename, binary, Access),         checkBom(),         setCRLFconversion(true).
Alternative (to updating PFC) you can do the change in your own part of the code In1:setCRLFconversion(true):

Code: Select all

clauses     run() :-         FN = @"..\SimpeTestFile.txt",         In1 = inputStream_file::openFileBom(FN),         In1:setCRLFconversion(true),         stdio::write("\n\nOutput using openFileBOM(FN):"),         foreach In1:repeatToEndOfStream() do             stdio::write("\n", In1:readLine())         end foreach,         In1:close(),         In2 = inputStream_file::openFile8(FN),         stdio::write("\n\nOutput using openFile8(FN):"),         foreach In2:repeatToEndOfStream() do             stdio::write("\n", In2:readLine())         end foreach,         In2:close().

Re: openFileBOM/1 and extra empty strings on readLine()

Posted: 22 May 2018 19:37
by Harrison Pratt
I understand -- and prefer option #2 rather than tweaking PFC code.

Question: what would happen if setCRLFconversion(true) was called in the 2nd clause of checkBOM/0 which is executed when there is no BOM?

Code: Select all

    checkBom() :-         setPosition(0),         setMode(stream::ansi(threadAnsi)),         setCRLFconversion(true).  % <== why not here?

Re: openFileBOM/1 and extra empty strings on readLine()

Posted: 23 May 2018 14:22
by Thomas Linder Puls
CR LF conversion should be performed for any text file regardless of it contains a BOM or not.

Re: openFileBOM/1 and extra empty strings on readLine()

Posted: 23 May 2018 14:44
by Harrison Pratt
Silly question ... thanks.