Mikio Hirabayashi has built a very nice API called 'QDBM'.
http://qdbm.sourceforge.net/
see also http://qdbm.sourceforge.net/benchmark.pdf
QDBM is well known in the Unix world, but has been ported to Win32. Practically, it is a disk-based Hashtable + Binary tree + inverted list package. QDBM offers many solutions called 'depot', 'curia', 'villa', 'odeum' and is well worth a look if you want to manage large amount of data.
In the enclosed project, you'll find a small 'depot' wrapper for Visual Prolog.
The example creates 1 000 000 (key, values) and then ask for 1 000 000 keys at random. All the key/values are stored on disk in a single file.
Using a collection of instances of the visual prolog class 'depot', you can manage a very large hashtable on disk.
I do not provide a full documentation since the real documentation can be found on the above web site. Only the port of depot is functional in this lib.
I tried to simplify the use of this lib as much as possible, and tried not to get too far away from the original API syntax :
Code: Select all
NewDepot = depot::new("dtest"), % create a depot db
NewDepot:dbopen("create"), % open it (create mode)
NewDepot:dbput_string(Key, Value), % put a key,value
NewDepot:dbclose(), % close the file
Code: Select all
ReadDepot = depot::new("dtest"), %
ReadDepot:dbopen("read"), % open it in read mode
Value = ReadDepot:dbget_string(Key), % tries to find the value for a key, fails if no value
ReadDepot:dbclose(), %
The project doesn't include a copy of qdbm (because of the allowed posting size); I encourage you to download the most up-to-date version on the site above. It is provided under the GNU Lesser GPL.
Just copy the three following files into your exe directory :
libiconv-2.dll
mgwz.dll
qdbm.dll
you will find the package there :
http://qdbm.sourceforge.net/win/
Best Regards
Gildas