Discussions related to Visual Prolog
Martin Meyer
VIP Member
Posts: 328
Joined: 14 Nov 2002 0:01

Memory corruption in finalizer

Unread post by Martin Meyer »

Hello Thomas,

please have a look at below example (in VIP 8 build 801). It creates a number of instances of an object type. The object has a nondeterm fact named keyValFact. In the object's constructor one exemplary key-value pair is asserted to the fact. In the finalizer it is checked whether the pair is still there. The outcomes of the checks are recorded in class facts okCount and errList.

My tests revealed random behavior. Please try in 32/64 bit modes and with various values for ObjCount.

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyValFact : (unsigned Key, unsigned Val) nondeterm.   clauses     new() :-         Key = 1,         Val = 2,         assert(keyValFact(Key, Val)).   clauses     finalize() :-         KeyValList = [ tuple(Key, Val) || keyValFact(Key, Val) ],         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main
I have tested replacing the nondeterm fact by several kinds of collections. The result is always the same. It seems to run into some memory corruption. Three variations of the example:

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsignedNative Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsignedNative Key, unsigned Val}** := [].   facts     keyToValDict : radixTree::dictionary{unsigned Val} := radixTree::empty.   clauses     new() :-         Key = 1,         Val = 2,         keyToValDict := radixTree::insert({ (K1, _K2) = K1 }, keyToValDict, Key, Val).   clauses     finalize() :-         KeyValList = [ tuple(Key, Val) || radixTree::get_nd(keyToValDict, Key, Val) ],         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyToValMapM : mapM{unsigned Key, unsigned Val} := mapM_redBlack::new().   clauses     new() :-         Key = 1,         Val = 2,         keyToValMapM:set(Key, Val).   clauses     finalize() :-         KeyValList = keyToValMapM:asList,         if KeyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [KeyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main

Code: Select all

interface obj end interface obj   %---   class obj : obj     open core   properties     okCount : unsigned (o).     errList : tuple{unsigned Key, unsigned Val}** (o).   end class obj   %---   implement obj     open core   class facts     okCount : unsigned := 0.     errList : tuple{unsigned Key, unsigned Val}** := [].   facts     keyValList : tuple{unsigned Key, unsigned Val}* := [].   clauses     new() :-         Key = 1,         Val = 2,         keyValList := [tuple(Key, Val) | keyValList].   clauses     finalize() :-         if keyValList = [tuple(1, 2)] then             okCount := okCount + 1         else             errList := [keyValList | errList]         end if.   end implement obj   %===   implement main   clauses     run() :-         ObjCount = 10000,         foreach _ = std::cIterate(ObjCount) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         ErrListLen = list::length(obj::errList),         %stdIO::write(obj::errList, '\n\n'),         stdIO::writeF("objects:      %10u\n", ObjCount),         stdIO::writeF("still alive:  %10u\n", ObjCount - (obj::okCount + ErrListLen)),         stdIO::writeF("finalized ok: %10u\n", obj::okCount),         stdIO::writeF("finalized err:%10u\n", ErrListLen).   end implement main
Furthermore I have noticed that exceptions in the finalizer are not raised to the outside of the finalizer. Is that intended? I used this code to test it:

Code: Select all

interface obj end interface obj   %---   class obj : obj   properties     finalizeCount : unsigned (o).   end class obj   %---   implement obj   class facts     finalizeCount : unsigned := 0.   clauses     finalize() :-         finalizeCount := finalizeCount + 1,         if finalizeCount > 0 then             exception::raise_error()         end if.   end implement obj   %===   implement main   clauses     run() :-         foreach _N = std::cIterate(10) do             _ = obj::new()         end foreach,         memory::garbageCollect(),         stdIO::write(obj::finalizeCount).   end implement main
Regards Martin
User avatar
Thomas Linder Puls
VIP Member
Posts: 1398
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Unread post by Thomas Linder Puls »

Thank you. We will investigate these problems.

It is correct that exceptions in finalizers are not reported anywhere. A problem is that finalizers can be run by any thread in many different situations. So to avoid that exceptions are reported/handled in strange contexts, they are trapped and thrown away. I agree that this is a drastic behavior, but the opposite is also problematic.

In general you should use finalizers as little as possible, but they can be handy for "closing/releasing" external handles/resources.
Regards Thomas Linder Puls
PDC
Martin Meyer
VIP Member
Posts: 328
Joined: 14 Nov 2002 0:01

Re: Memory corruption in finalizer

Unread post by Martin Meyer »

Thanx for the info, Thomas!

By now I got the feeling you have said before that exceptions are not raised to the outside of finalizers. But maybe it's déjà vu. I will try hard to not forget it again.
Regards Martin
Martin Meyer
VIP Member
Posts: 328
Joined: 14 Nov 2002 0:01

Re: Memory corruption in finalizer

Unread post by Martin Meyer »

I have an add-on question. Probably you can answer it too:

Is it legal in a finalizer to store the object (i.e. This) in a class fact and thereby abandon the disposal of the object?
Regards Martin
User avatar
Thomas Linder Puls
VIP Member
Posts: 1398
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Unread post by Thomas Linder Puls »

Martin Meyer wrote: 23 Apr 2018 15:52 Is it legal in a finalizer to store the object (i.e. This) in a class fact and thereby abandon the disposal of the object?
No.
Regards Thomas Linder Puls
PDC
Martin Meyer
VIP Member
Posts: 328
Joined: 14 Nov 2002 0:01

Re: Memory corruption in finalizer

Unread post by Martin Meyer »

I see. Thanks again for answering!
Regards Martin
User avatar
Thomas Linder Puls
VIP Member
Posts: 1398
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Unread post by Thomas Linder Puls »

I think we have solved this problem, and will create a new build soon (I believe this build also solves the other problems you have reported).
Regards Thomas Linder Puls
PDC
User avatar
Thomas Linder Puls
VIP Member
Posts: 1398
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Unread post by Thomas Linder Puls »

This problem is solved in Visual Prolog 8 - Build 802.
Regards Thomas Linder Puls
PDC
Martin Meyer
VIP Member
Posts: 328
Joined: 14 Nov 2002 0:01

Re: Memory corruption in finalizer

Unread post by Martin Meyer »

Hello Thomas,

please check again the initial example code in build 901. It looks like the fix has not been taken to version 9.
Regards Martin
User avatar
Thomas Linder Puls
VIP Member
Posts: 1398
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Unread post by Thomas Linder Puls »

Well, actually the problem is different now.

When I run the program the keyValFact looks wrong in the finalizer. That could be because they have already been garbage collected. That is however an undesirable behavior. I will investigate the problem.
Regards Thomas Linder Puls
PDC
User avatar
Thomas Linder Puls
VIP Member
Posts: 1398
Joined: 28 Feb 2000 0:01

Re: Memory corruption in finalizer

Unread post by Thomas Linder Puls »

This bug is solved with build 902.
Regards Thomas Linder Puls
PDC
Post Reply