Object Disposal

Discussions related to Visual Prolog
Post Reply
Martin Meyer
VIP Member
Posts: 241
Joined: 14 Nov 2002 0:01

Object Disposal

Post by Martin Meyer » 26 Nov 2018 19:44

Hello Thomas,

the garbage collector has a problem to dispose the objects in below code (in VIP build 802). Please have a look.

To begin with I take a working code version which creates a number of objects, calls garbageCollect/0, and in doing so it outputs the number of alive objects.

Code: Select all

interface someObj end interface someObj   %---   class someObj : someObj   properties     count : unsigned (o).   end class someObj   %---   implement someObj   class facts     count : unsigned := 0.   facts     objReference : someObj [used].   clauses     new() :-         objReference := This,         count := count + 1.   clauses     finalize() :-         count := count - 1.   end implement someObj   %===   implement main   constants     count = 1000.   clauses     run() :-         stdIO::write("<1>\n", someObj::count, " objects exist\n"),         foreach _ = std::cIterate(count) do             _ = someObj::new()         end foreach,         stdIO::write(count, " objects created\n"),         stdIO::write("<2>\n", someObj::count, " objects exist\n"),         stdIO::write("garbageCollect()", '\n'),         memory::garbageCollect(),         stdIO::write("<3>\n", someObj::count, " objects exist\n").   end implement main
It outputs:

Code: Select all

<1> 0 objects exist 1000 objects created <2> 1000 objects exist garbageCollect() <3> 1 objects exist
Thus the garbage collector has disposed all objects but one.

Now I change the domain of the objReference fact:

Code: Select all

interface someObj end interface someObj   %---   class someObj : someObj   properties     count : unsigned (o).   end class someObj   %---   implement someObj   class facts     count : unsigned := 0.   domains %%% new %%%     objReference = objReference(someObj). %%% new %%%   facts     objReference : objReference [used]. %%% changed %%%   clauses     new() :-         objReference := objReference(This), %%% changed %%%         count := count + 1.   clauses     finalize() :-         count := count - 1.   end implement someObj   %===   implement main   constants     count = 1000.   clauses     run() :-         stdIO::write("<1>\n", someObj::count, " objects exist\n"),         foreach _ = std::cIterate(count) do             _ = someObj::new()         end foreach,         stdIO::write(count, " objects created\n"),         stdIO::write("<2>\n", someObj::count, " objects exist\n"),         stdIO::write("garbageCollect()", '\n'),         memory::garbageCollect(),         stdIO::write("<3>\n", someObj::count, " objects exist\n").   end implement main
The changed version outputs:

Code: Select all

<1> 0 objects exist 1000 objects created <2> 1000 objects exist garbageCollect() <3> 1000 objects exist
The garbage collector does not dispose these objects.
Regards Martin

User avatar
Thomas Linder Puls
VIP Member
Posts: 2351
Joined: 28 Feb 2000 0:01

Re: Object Disposal

Post by Thomas Linder Puls » 27 Nov 2018 10:16

Hi, Martin. I can reproduce the problem. Actually, it has already been solved in internal versions.

You should notice that the reason the objects are not garbage collected is strongly related to the finalize predicate. If you remove that predicate the objects will indeed be garbage collected (but unfortunately you also lose the possibility of measuring it).

The main issue is that finalize code is a bit strange, because the idea is that it should run when the memory is no longer alive/reachable. But that contains a contradiction, because the finalize code has access to the memory, so actually is it still alive/reachable.

This gives rise to a lot of "problems". In this case it has to do with the memory that is reachable from the "semi-dead" object. In Vip8 the strategy is that memory that is reachable from a semi-dead memory is considered alive (except for direct self references).

In your first code objReference is stored directly in the objects own memory, and subsequently you have created a self reference. In the second code objReference points to a separately referenced piece of memory, which the therefore considered alive. And since this piece of memory contains a reference to the object the object is also alive, so in your second code the object is also considered alive.

We have recently switched to another strategy for semi-dead memory, where semi-dead memory doesn't keep anything alive at all. That will solve this kind of problems, but then you should definitely not bring memory back to live from the finalize predicate (which also seems like a daft idea, but maybe it happens indirectly without your awareness).


I don't know what your real problem is, but if you have only added finalize predicates to detect a problem, you may actually have introduced the problem (and then detected it).
Regards Thomas Linder Puls
PDC

Martin Meyer
VIP Member
Posts: 241
Joined: 14 Nov 2002 0:01

Re: Object Disposal

Post by Martin Meyer » 28 Nov 2018 0:19

Thomas, thank you for the detailed explanation! It is that I am actually using a finalizer in my real code:

In my real program there are objects with positive ID numbers. I use these IDs as keys of an array. My program keeps creating and disposing objects as long as it is running. The size of the array equals the highest ID (plus 1) which has been inserted so far. To not let the ID numbers grow too large and to keep the array's size within limits, I am reusing the IDs of disposed objects. Thus in the finalizer I am inserting the object's ID in a list of vacant IDs.

The objReference fact in my real program is of a compound domain with functor alternatives because the object is a node in a union–find data structure. The fact either points to a parent object, or, when the object is the representative of its component, it holds some data of the component.

When I was running my program on a relatively small input, I was surprised that it generated rather high IDs. Tracking down the reason for that, I found that it is not disposing the objects.
Regards Martin

Martin Meyer
VIP Member
Posts: 241
Joined: 14 Nov 2002 0:01

Re: Object Disposal

Post by Martin Meyer » 28 Nov 2018 0:42

Reading it again a question is comming to my mind: Is inserting the object's ID in a list of vacant IDs bringing memory back to live from the finalize predicate? What exactly counts as bringing memory back to live?
Regards Martin

User avatar
Thomas Linder Puls
VIP Member
Posts: 2351
Joined: 28 Feb 2000 0:01

Re: Object Disposal

Post by Thomas Linder Puls » 28 Nov 2018 20:46

A piece of memory is reachable, if it can be reached from a call stack, or from a class fact. So if you insert a semi dead piece of memory in a data structure which is reachable from a class fact, then it will be brought back to live.

You can solve you problem by creating a separate object that has a finalizer that insert the ID in the vacant list. I.e. your main object create an object with the ID and a finalizer, and store that object in a fact variable. Now the finalizer predicate is in a piece of memory that is not part of any cycles.

You should also notice/realize that as long as an object is in your array it is reachable and will therefore not be garbage collected.
Regards Thomas Linder Puls
PDC

Martin Meyer
VIP Member
Posts: 241
Joined: 14 Nov 2002 0:01

Re: Object Disposal

Post by Martin Meyer » 1 Dec 2018 11:50

Would it be cheaper in terms of runtime to create a physical copy of the ID instead of creating an additional object? The semi dead memory will not be reanimated by making a copy. Does memory::getInteger/1-> create a physical copy and is the below code a legal solution?

Code: Select all

interface myObj       open core   properties     id : positive (o).   end interface myObj   %---   class myObj : myObj end class myObj   %---   implement myObj       open core   class facts     idCount : positive := 0.     vacantIDlist : positive_list := [].   facts     id : positive.   clauses     new() :-         if [ID | RestVacantIDlist] = vacantIDlist then             id := ID,             vacantIDlist := RestVacantIDlist         else             id := idCount,             idCount := idCount + 1         end if.   clauses     finalize() :-         ID = memory::getInteger(memory::pInteger(id)),         vacantIDlist := [ID | vacantIDlist].   end implement myObj
I wonder however whether it is neccessary to use getInteger/1-> here to create a physical copy. When I look into memory.pro, I see that getInteger/1-> just resolves to

Code: Select all

get4(pInteger(V)) = V.
So, why not just do it like:

Code: Select all

    finalize() :-         vacantIDlist := [id | vacantIDlist].
I guess that copies the 4 bytes of the value of the id fact rather than to hand over an address where the (semi dead) value resides. Is that true?
Regards Martin

User avatar
Thomas Linder Puls
VIP Member
Posts: 2351
Joined: 28 Feb 2000 0:01

Re: Object Disposal

Post by Thomas Linder Puls » 2 Dec 2018 20:52

Your ID is completely irrelevant, the ID is just a number. The important thing is your object(s) and the references to it/them.

This is the essesce of the problem:

Code: Select all

class myObj : object end class myObj   %---   implement myObj     open core   facts     ref : optional{object}.   clauses     new() :-         ref := some(This).   clauses     finalize().   end implement myObj

The object has a finalizer, and the object has a pointer (i.e. ref) to another piece of memory (i.e. some(...)) and this piece of memory has a reference to back to the object (i.e. This). So we have an object with a finalizer that also is part of a cyclic reference (not a self reference).

Such an object will never be garbage collected (in Vip 8.0 and earlier).

You can avoid the problem by moving the finalize predicate to another object which does not participate in any cycles.

So we create an object whose only task is to handle the finalization:

Code: Select all

class myObjFinalize : object   constructors     new : (integer ID).   end class myObjFinalize   %---   implement myObjFinalize   facts     id : integer.   clauses     new(Id) :-         id := ID.   clauses     finalize() :-         idManager::releaseId(id).   end implement myObjFinalize
This object does not participate in any cycles, so it will be garbage collected even though it has a finalize predicate.

Now we update you main object:

Code: Select all

class myObj : object   constructors     new : (integer ID).   end class myObj   %---   implement myObj     open core   facts     ref : optional{object}.     myObjFinalize : object.   clauses     new(ID) :-         myObjFinalize := myObjFinalize::new(ID),         ref := some(This).   end implement myObj
This object still participate in an cycle, but it no longer has a finalize predicate. Therefore it will be garbage collected (when relevant), and when this happens there will no longer be any references to the myObjFinalize object, and therefore its finalize predicate will run.

That should solve the finalize problem.

But as mentioned you should be aware that you do not keep your object alive in some other way. I especially have worries about the array you mention. You do not say what you put into that array, but if you have a reference from the array back to the object, then the object can be reached as long as it is in the array. I.e. you will remove the object from the array when it is finalized, but the object will not be finalized before it is removed from the array (catch-22).
Regards Thomas Linder Puls
PDC

Martin Meyer
VIP Member
Posts: 241
Joined: 14 Nov 2002 0:01

Re: Object Disposal

Post by Martin Meyer » 3 Dec 2018 1:29

Yes, thank you, I understand.

My preceding post was insufficient. The solution, which I have in mind, in full, with both object reference and id facts, is like:

Code: Select all

interface someObj       open core   properties     optRef : optional{someObj}.     id : positive (o).   end interface someObj   %---   class someObj : someObj end class someObj   %---   implement someObj       open core   constants     nilPointer : pointer = uncheckedConvert(pointer, 0x10).   class facts     idCount : positive := 0.     vacantIDlist : positive_list := [].   facts     ref_fact : someObj.     id : positive.   clauses     optRef(none) :-         ref_fact := uncheckedConvert(someObj, nilPointer).     optRef(some(Ref)) :-         ref_fact := Ref.   clauses     optRef() = if uncheckedConvert(pointer, ref_fact) = nilPointer then none else some(ref_fact) end if.   clauses     new() :-         ref_fact := This,         if [ID | RestVacantIDlist] = vacantIDlist then             id := ID,             vacantIDlist := RestVacantIDlist         else             id := idCount,             idCount := idCount + 1         end if.   clauses     finalize() :-         vacantIDlist := [id | vacantIDlist].   end implement someObj
That way the ref_fact does not point to a separately referenced piece of memory and thus should not impede the disposal of the object (like in my initial code piece). I suppose this solution runs faster, because it omits creating an additional object.

I was concerned however, whether I am bringing semi-dead memory back to live in the finalizer here and I need some more elaborated construction to copy the value of the id fact. So, is the solution in the above form OK?

To your justified worries that I am keeping objects alive unintentionally: Yes, I store objects in the array, but only for intermediate calculations. After an intermediate calculation is finished, I clear (i.e. zero) the array and keep it to re-use it in the next intermediate calculation.
Regards Martin

User avatar
Thomas Linder Puls
VIP Member
Posts: 2351
Joined: 28 Feb 2000 0:01

Re: Object Disposal

Post by Thomas Linder Puls » 3 Dec 2018 9:27

The code you have presented here will work.

There is no problem in referencing the id or anything else for that matter in the finalizer.

Also types like integer are not by themselves subject of "life and death", such values are copied around and stored where needed. The memory they are stored in may die and as such that "copy" of the value will also be "dead". But when you insert the id into a list somewhere, the integer value itself is copied into a new storage and the old storage is completely unaffected.

All the basic number types, handles and pointers are copied between storage places, it is the things that pointers points to that can be heap allocated and therefore subject of garbage collection.

All compound/functor types, strings, string8 and objects are represented by a pointer pointing to the "actual data". The pointer is copied around, but the "actual data" stays in place and is subject of life, death and garbage collection.
Regards Thomas Linder Puls
PDC

Martin Meyer
VIP Member
Posts: 241
Joined: 14 Nov 2002 0:01

Re: Object Disposal

Post by Martin Meyer » 3 Dec 2018 10:48

OK, fine, and thanks again for explaining everything so well!
Regards Martin

Post Reply