Page 1 of 1

Memory corruption in finalizer

Posted: 20 Apr 2018 21:41
by Martin Meyer
Hello Thomas,

please have a look at below example (in VIP 8 build 801). It creates a number of instances of an object type. The object has a nondeterm fact named keyValFact. In the object's constructor one exemplary key-value pair is asserted to the fact. In the finalizer it is checked whether the pair is still there. The outcomes of the checks are recorded in class facts okCount and errList.

My tests revealed random behavior. Please try in 32/64 bit modes and with various values for ObjCount.

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyValFact : (unsigned Key, unsigned Val) nondeterm.   clauses     new() :-         Key = 1,         Val = 2,         assert(keyValFact(Key, Val)).   clauses     finalize() :-         KeyValList = [ tuple(Key, Val) || keyValFact(Key, Val) ],         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main
I have tested replacing the nondeterm fact by several kinds of collections. The result is always the same. It seems to run into some memory corruption. Three variations of the example:

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsignedNative Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsignedNative Key, unsigned Val}** := [].   facts     keyToValDict : radixTree::dictionary{unsigned Val} := radixTree::empty.   clauses     new() :-         Key = 1,         Val = 2,         keyToValDict := radixTree::insert({ (K1, _K2) = K1 }, keyToValDict, Key, Val).   clauses     finalize() :-         KeyValList = [ tuple(Key, Val) || radixTree::get_nd(keyToValDict, Key, Val) ],         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyToValMapM : mapM{unsigned Key, unsigned Val} := mapM_redBlack::new().   clauses     new() :-         Key = 1,         Val = 2,         keyToValMapM:set(Key, Val).   clauses     finalize() :-         KeyValList = keyToValMapM:asList,         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyValList : tuple{unsigned Key, unsigned Val}* := [].   clauses     new() :-         Key = 1,         Val = 2,         keyValList := [tuple(Key, Val) | keyValList].   clauses     finalize() :-         if keyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [keyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main
Furthermore I have noticed that exceptions in the finalizer are not raised to the outside of the finalizer. Is that intended? I used this code to test it:

Code: Select all

interface obj end interface obj   %---   class obj : obj   properties     finalizeCount : unsigned (o).   end class obj   %---   implement obj   class facts     finalizeCount : unsigned := 0.   clauses     finalize() :-         finalizeCount := finalizeCount + 1,         if finalizeCount > 0 then             exception::raise_error()         end if.   end implement obj   %===   implement main   clauses     run() :-         foreach _N = std::cIterate(10) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         stdIO::write(obj::finalizeCount).   end implement main

Re: Memory corruption in finalizer

Posted: 23 Apr 2018 10:52
by Thomas Linder Puls
Thank you. We will investigate these problems.

It is correct that exceptions in finalizers are not reported anywhere. A problem is that finalizers can be run by any thread in many different situations. So to avoid that exceptions are reported/handled in strange contexts, they are trapped and thrown away. I agree that this is a drastic behavior, but the opposite is also problematic.

In general you should use finalizers as little as possible, but they can be handy for "closing/releasing" external handles/resources.

Re: Memory corruption in finalizer

Posted: 23 Apr 2018 15:12
by Martin Meyer
Thanx for the info, Thomas!

By now I got the feeling you have said before that exceptions are not raised to the outside of finalizers. But maybe it's déjà vu. I will try hard to not forget it again.

Re: Memory corruption in finalizer

Posted: 23 Apr 2018 15:52
by Martin Meyer
I have an add-on question. Probably you can answer it too:

Is it legal in a finalizer to store the object (i.e. This) in a class fact and thereby abandon the disposal of the object?

Re: Memory corruption in finalizer

Posted: 23 Apr 2018 19:06
by Thomas Linder Puls
Martin Meyer wrote:
23 Apr 2018 15:52
Is it legal in a finalizer to store the object (i.e. This) in a class fact and thereby abandon the disposal of the object?
No.

Re: Memory corruption in finalizer

Posted: 24 Apr 2018 0:19
by Martin Meyer
I see. Thanks again for answering!

Re: Memory corruption in finalizer

Posted: 24 Apr 2018 19:47
by Thomas Linder Puls
I think we have solved this problem, and will create a new build soon (I believe this build also solves the other problems you have reported).

Re: Memory corruption in finalizer

Posted: 14 May 2018 11:18
by Thomas Linder Puls
This problem is solved in Visual Prolog 8 - Build 802.

Re: Memory corruption in finalizer

Posted: 12 Apr 2019 18:04
by Martin Meyer
Hello Thomas,

please check again the initial example code in build 901. It looks like the fix has not been taken to version 9.

Re: Memory corruption in finalizer

Posted: 23 Apr 2019 9:27
by Thomas Linder Puls
Well, actually the problem is different now.

When I run the program the keyValFact looks wrong in the finalizer. That could be because they have already been garbage collected. That is however an undesirable behavior. I will investigate the problem.

Re: Memory corruption in finalizer

Posted: 26 Apr 2019 11:44
by Thomas Linder Puls
This bug is solved with build 902.