More robust Reference tracking information in Debug

[twobitadder]

Describe the project you are working on

A small hobby project that makes use of Resources to track world and object states.

Describe the problem or limitation you are having in your project

My project has a memory leak and I suspect it's due to some unreleased references - tracking it down involves looking "around" the problem (things like checking signal connections, nulling out variables to be explicit about referencing, etc), since they are not full-fledged nodes. I made a custom build that exposed the function that describes how many reference counts that a Reference has but I would greatly appreciate more information on what exactly might be holding on to it.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Having some sort of information that denotes either who holds references or (perhaps more feasible) what references an object is holding for debug would go a long way towards helping with an issue like this, as it appears there's currently no way to do it.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Maybe an API call within Object that can provide a list of References that it has increased the refcounter on but has not decreased the refcounter. Something like object.get_references() that provides an array of those items. More useful might be something like reference.get_active_references() that would work in the opposite direction, but I don't know if it's worth introducing that kind of data redundancy.

It would be great if there was some information on why it's being held onto, but for now I would be okay with just what an object is holding onto.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not that I'm aware of - it might be possible to create an array that you have the names of references that you add/remove from, but that can't really account for the automatic work that would be happening engine-side.

Is there a reason why this should be core and not an add-on in the asset library?

There aren't any API exposures that do anything like this, so it can't really be an add-on either.

[Calinou]

Try building Godot with the rids=tracked_handles SCons option. This is new in Godot 3.5 if I'm not mistaken.

cc @lawnjelly

[lawnjelly]

Try building Godot with the rids=tracked_handles SCons option. This is new in Godot 3.5 if I'm not mistaken.

godotengine/godot#54907
You can do that if building the engine yourself. Although it may or may not help, it depends whether the Resources concerned are themselves creating RIDs.

Despite the name "RID" (which I assume stands for resource ID) these are objects in the servers, rather than Resources as referred to in the engine.

I have no idea what tracking Godot 4 has for Resources / References, there may be more than in 3.x (A quick look at 3.x source code I didn't see any tracking facilities, but it might be non-obvious). More tracking could theoretically be added, but it would likely have to be a specialized build of the engine, because tracking can use extra performance / memory.

The proposal probably needs an example MRP.

[twobitadder]

No problem! I'll get an example MRP going. I did also read over the rids=tracked_handles PR and it might not be exactly what I need since these are not created through server use.

I'll get a project attached later today, and maybe hopefully discover my own bug in the process, haha

[twobitadder]

And, of course, I seem to be unable to replicate it with an MRP. But! I did at least create and attach something that sort of illustrates my point. Running this should create a hundred children, and it does - it should be possible to check that pretty quickly with a print of get_children().size(). You can also see the number of nodes and resources - pretty expected behavior there.

That being said, the number of objects in the debugger hovers around 1000 items not counting those children. Other than comparing numbers after changing the number of children I create, I have no real way of knowing how many of those are references that I've created vs. things the engine has, automatically.

I realize this has maybe drifted a bit from my original suggestion - this is speaking more on Objects listed in the debugger count vs. References, but the number has directly correlated with my self-created refcount issue, which is why I thought to tackle this by analyzing Reference connections, and thus ran into my wall.

To illustrate the issue I am still quasi-having in my own project:

This is an insane stress test, but the number of Objects listed here does not at all jive with Nodes and Resources, and I was having trouble with items that inherit Resource still hanging on in memory and listening to signals despite me ostensibly clearing them out from anything that should be tracking them. All in all, it feels like trying to debug something like this is a little more opaque than I would like it to be, but I'm not sure if there's a good path forward to helping with that.

Anyway, a small little example project is below but it's still not terribly well representative of what's going on with my specific issue, but hopefully it's a good illustration of the issue I'm having tool-wise.

memtestmrp.zip