Memory corruption in finalizer

Discussions related to Visual Prolog
Post Reply
Martin Meyer
VIP Member
Posts: 235
Joined: 14 Nov 2002 0:01

Memory corruption in finalizer

Post by Martin Meyer » 20 Apr 2018 21:41

Hello Thomas,

please have a look at below example (in VIP 8 build 801). It creates a number of instances of an object type. The object has a nondeterm fact named keyValFact. In the object's constructor one exemplary key-value pair is asserted to the fact. In the finalizer it is checked whether the pair is still there. The outcomes of the checks are recorded in class facts okCount and errList.

My tests revealed random behavior. Please try in 32/64 bit modes and with various values for ObjCount.

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyValFact : (unsigned Key, unsigned Val) nondeterm.   clauses     new() :-         Key = 1,         Val = 2,         assert(keyValFact(Key, Val)).   clauses     finalize() :-         KeyValList = [ tuple(Key, Val) || keyValFact(Key, Val) ],         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main
I have tested replacing the nondeterm fact by several kinds of collections. The result is always the same. It seems to run into some memory corruption. Three variations of the example:

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsignedNative Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsignedNative Key, unsigned Val}** := [].   facts     keyToValDict : radixTree::dictionary{unsigned Val} := radixTree::empty.   clauses     new() :-         Key = 1,         Val = 2,         keyToValDict := radixTree::insert({ (K1, _K2) = K1 }, keyToValDict, Key, Val).   clauses     finalize() :-         KeyValList = [ tuple(Key, Val) || radixTree::get_nd(keyToValDict, Key, Val) ],         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyToValMapM : mapM{unsigned Key, unsigned Val} := mapM_redBlack::new().   clauses     new() :-         Key = 1,         Val = 2,         keyToValMapM:set(Key, Val).   clauses     finalize() :-         KeyValList = keyToValMapM:asList,         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyValList : tuple{unsigned Key, unsigned Val}* := [].   clauses     new() :-         Key = 1,         Val = 2,         keyValList := [tuple(Key, Val) | keyValList].   clauses     finalize() :-         if keyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [keyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main
Furthermore I have noticed that exceptions in the finalizer are not raised to the outside of the finalizer. Is that intended? I used this code to test it:

Code: Select all

interface obj end interface obj   %---   class obj : obj   properties     finalizeCount : unsigned (o).   end class obj   %---   implement obj   class facts     finalizeCount : unsigned := 0.   clauses     finalize() :-         finalizeCount := finalizeCount + 1,         if finalizeCount > 0 then             exception::raise_error()         end if.   end implement obj   %===   implement main   clauses     run() :-         foreach _N = std::cIterate(10) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         stdIO::write(obj::finalizeCount).   end implement main
Regards Martin

User avatar
Thomas Linder Puls
VIP Member
Posts: 2341
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Post by Thomas Linder Puls » 23 Apr 2018 10:52

Thank you. We will investigate these problems.

It is correct that exceptions in finalizers are not reported anywhere. A problem is that finalizers can be run by any thread in many different situations. So to avoid that exceptions are reported/handled in strange contexts, they are trapped and thrown away. I agree that this is a drastic behavior, but the opposite is also problematic.

In general you should use finalizers as little as possible, but they can be handy for "closing/releasing" external handles/resources.
Regards Thomas Linder Puls
PDC

Martin Meyer
VIP Member
Posts: 235
Joined: 14 Nov 2002 0:01

Re: Memory corruption in finalizer

Post by Martin Meyer » 23 Apr 2018 15:12

Thanx for the info, Thomas!

By now I got the feeling you have said before that exceptions are not raised to the outside of finalizers. But maybe it's déjà vu. I will try hard to not forget it again.
Regards Martin

Martin Meyer
VIP Member
Posts: 235
Joined: 14 Nov 2002 0:01

Re: Memory corruption in finalizer

Post by Martin Meyer » 23 Apr 2018 15:52

I have an add-on question. Probably you can answer it too:

Is it legal in a finalizer to store the object (i.e. This) in a class fact and thereby abandon the disposal of the object?
Regards Martin

User avatar
Thomas Linder Puls
VIP Member
Posts: 2341
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Post by Thomas Linder Puls » 23 Apr 2018 19:06

Martin Meyer wrote:
23 Apr 2018 15:52
Is it legal in a finalizer to store the object (i.e. This) in a class fact and thereby abandon the disposal of the object?
No.
Regards Thomas Linder Puls
PDC

Martin Meyer
VIP Member
Posts: 235
Joined: 14 Nov 2002 0:01

Re: Memory corruption in finalizer

Post by Martin Meyer » 24 Apr 2018 0:19

I see. Thanks again for answering!
Regards Martin

User avatar
Thomas Linder Puls
VIP Member
Posts: 2341
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Post by Thomas Linder Puls » 24 Apr 2018 19:47

I think we have solved this problem, and will create a new build soon (I believe this build also solves the other problems you have reported).
Regards Thomas Linder Puls
PDC

User avatar
Thomas Linder Puls
VIP Member
Posts: 2341
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Post by Thomas Linder Puls » 14 May 2018 11:18

This problem is solved in Visual Prolog 8 - Build 802.
Regards Thomas Linder Puls
PDC

Post Reply