Increase buffer size
Posted: 5 Dec 2019 8:58
I am streaming (reading) a 2GB utf8 file in binary mode; doing some processing; and then streaming out the processed file.
Input "record size" is 1024 (8 bit) bytes as this suits my application.
Everything works, and the processing time is reasonable (294 seconds), though I think I can do better with minor tuning. Larger read and write buffers would probably help as a 2000 byte default buffer size does not fit well with my chosen "record" size.
See code below;
I have tried searching for a predicate to increase the default buffer size, and I can see alternate file open predicates (non utf8) that can allocate custom buffer sizes (at least for the reading).
Can anyone point me to a way to increase my read and write streaming buffer sizes?
Regards,
Loffy
Input "record size" is 1024 (8 bit) bytes as this suits my application.
Everything works, and the processing time is reasonable (294 seconds), though I think I can do better with minor tuning. Larger read and write buffers would probably help as a 2000 byte default buffer size does not fit well with my chosen "record" size.
See code below;
Code: Select all
InStream = inputStream_file::openFileUtf8("File1Test.txt"),
InStream:setMode(stream::binary),
stdio::inputStream := InStream,
OutStream = outputStream_file::createUtf8("File9Test.txt"),
OutStream:setMode(stream::binary),
stdio::outputStream := OutStream,
processing .......
Can anyone point me to a way to increase my read and write streaming buffer sizes?
Regards,
Loffy