Unify the semantics of Nodes and the Parent / Child relationship

[andrueandersoncs]

In Godot, as in most game engines, the world is represented as a tree often referred to as the "scene graph" or "scene tree" (tree being the more technically correct due to the acyclic nature of the structure).

The problem with representing the world as a tree comes in the implementation of systems such as the Node management system, the positioning system, the rendering system, the UI system, and the physics system. Each of these systems uses the parent/child relationship and adds meaning to it. This overloading of the meaning of the relationship causes problems.

For example, I would have to make a node the child of another node in order to implement the behavior "Hide yourself when this other node is hidden"

For example, I would have to make a node the child of another node in order to implement the behavior "Translate yourself when this other node is translated"

For example, I would have to make a node the child of another node in order to implement the behavior "Instantiate yourself when this other node is instantiated"

For example, I would have to make a node the child of another node in order to implement the behavior "Resize yourself when this other node is resized"

And so on..

Likewise, in the opposite direction, I cannot opt-out of any of these behaviors if the parent/child relationship is present, for example, I cannot prevent a child from being translated when its parent is translated.

This is due to the fact that a given node can only have a single parent, and the parent is the driver of these behaviors. This is directly analogous to the problems with single-inheritance in class-based OOP languages, commonly referred to as the "diamond problem".

My suggestion is that we directly embrace composition in all systems. This looks like:

1. All built-in behaviors facilitated by the parent/child relationship are refactored except for the compositional behavior "Has a": meaning if a given parent node is created, its child nodes are also created. This reduces the meaning of "being a parent of a node" down to one single meaning: "a parent is a node that instantiates its children when it's instantiated" (as opposed to the current meaning of being a parent of a node, which is something like "a parent is a node that instantiates its children, translates its children, hides and shows its children, possibly resizes its children, and so on")

2. All other built-in behaviors facilitated by the parent/child relationship are implemented as Nodes, for example: TranslateNode, VisibilityNode, LayoutNode, and so on, which can be added as a child to a given Node and will drive the associated behavior for the parent (as opposed to the current implementation in which the parent drives the behavior of the child) - this is much more compositional and as far as I understand it, is much closer to the intended architecture of Godot in which you add children to add functionality.

I'm missing a lot of context on implementation details, so I'll need feedback to help you work through how exactly the refactoring would look in order to maintain a consistent semantics. I know there aren't a lot of game engines that work this way, and that is a fast heuristic for determining if it's a good idea or not, but I'm humbly asking for a bit of deeper consideration here. Absorb the idea, riff on it, I think you might like some of the implications of this architecture that I haven't had the time to expand on here.

Cheers, thanks for reading!

[AThousandShips]

You can accomplish this by various means, you can use Top Level to ignore parent translation, and use RemoteTransform to share position outside of the hierarchy

Generally I'd say the need to break this structure is limited in most cases, and should be considered an exception, and you should question rather if you genuinely need it, rather than adding new code to compensate

[AThousandShips]

Remove some code, but you'd mostly add code, moving code from one element to another isn't removing code, it's adding it

We don't do the component based structuring of nodes, nodes are self contained, Godot does not use ECS which is what this boils down to

The fact that we have one exception to the rule doesn't prove the need for it in general, it shows that some times we need a slightly different approach, but most of the time we don't

[andrueandersoncs]

(just wanted to say here up top I appreciate your input and the time you're taking to discuss this with me, please don't take my argumentation as a personal slight, I'm only interested in finding the truth of the matter and I mean no disrespect)

Considering that the code to actually implement "translate this relative to that" already exists, and you'd be removing the glue and associated properties from Node3D, adding only the glue back into a separate new Node without the need for the conditional logic and the Top Level property, this seems to me like you'd have both less code and less semantic burden overall. Granted, I don't have any idea what the code actually looks like, but if this isn't the case I think there are probably bigger architectural concerns to tackle than what we're discussing here.

Regarding the component-based structuring of nodes, I would have to disagree, nodes are not self-contained, nor are they meant to be used isolation. An obvious counter-example to this assertion is the Rigidbody3D node, which can't function without a CollisionShape3D. Even supposing this assertion were true, it only represents the current situation. Unless you're implying that the current situation is the best possible situation, it doesn't actually have any bearing on whether my suggestion is an improvement.

Having an exception does prove the need for the exception, otherwise it wouldn't exist. Granted, it doesn't prove the need for my proposal, but my claim is that this would be an improvement, not that it is adding functionality that doesn't already exist in some form. Indeed, I think most of my listed problems can be solved with hacks or workarounds. Hopefully I haven't misrepresented the situation by implying that these are the only problems caused by these semantic inconsistencies, they're just the ones I've personally encountered in my limited experience with the engine. I'm sure with a bit of time and consideration one could find many more examples.

[AThousandShips]

First of all thank you 🙂

Unless you're implying that the current situation is the best possible situation, it doesn't actually have any bearing on whether my suggestion is an improvement.

The answer is in the reasoning for why the engine doesn't rely on composition for almost all functionality

While there is no universally better approach, we found that using an inheritance-based approach resulted in better usability while still being fast enough for most use cases.

The fact is that functionality, especially fundamental functionality, defined by composition instead of the chosen type, is worse for performance, it involves much more lookups, that's why we've chosen to implement some of the exceptions like this

The case of collision shapes is a perfect counterexample of this, it's a vase where composition does help, by having all the functionality around positioning and hierarchy already there in a node type, otherwise we'd have to implement a separate system for handling the positioning of collision shapes, and expanding that to other contexts would be much more difficult

Take for example navigation, having collision shapes like this makes generating the navigation much easier, you just need to find those shapes, not extract data from the node using them

In my opinion there's very little need to break with the standard idea of the tree hierarchy, especially often enough and in varied enough ways that the systems in place isn't enough, with some room for improvement

I would suggest re-evaluating the approach you're taking, and see if you might solve this by changing the way you think about how to build your game, some times one can come into a situation with a fixed idea of how things should work, to the point where seeing it in a different light becomes harder, I have done that many times myself in various areas of programming and design

Those are just my thoughts on it, I think specifically controlling behaviour that is this fundamental with composition is not feasible, and it risks opening up many unexpected pitfalls, having more flexibility in controlling some parts of the way the scene tree works would be interesting, I just think the suggested approach isn't the way to go

[andrueandersoncs]

The fact is that functionality, especially fundamental functionality, defined by composition instead of the chosen type, is worse for performance, it involves much more lookups, that's why we've chosen to implement some of the exceptions like this

Interesting, I've often seen (and personally verified) claims that component-based approaches are actually much better for performance due to data locality. Perhaps the performance problems you're referring to were a consequence of implementation details rather than an architectural limitation? In my experience, a naive implementation of ECS would be something like what could be found in the popular book Game Programming Patterns - and I agree this approach can incur significant component lookup costs. The typical solution (found in, e.g. Unity) is called Archetypes. Since the component lookup is, broadly, a search, the way to speed it up is indexing. When a component is added or removed, the index relating components and entities is updated. This makes the adding and removing of components slower but during gameplay this happens much less frequently than the component lookups and is, in my experience, worth the tradeoff.

The case of collision shapes is a perfect counterexample of this, it's a vase where composition does help, by having all the functionality around positioning and hierarchy already there in a node type, otherwise we'd have to implement a separate system for handling the positioning of collision shapes, and expanding that to other contexts would be much more difficult

I don't think this is true in the hypothetical "Godot but with the potential for architectural changes" situation that we're discussing since positioning (translation, rotation) (of 2d and 3d vectors) is a general process. You wouldn't have to have a separate positioning system for collision shapes, they would be a node just like any other node and they could take advantage of the compositional approach: just add the appropriate node child and be opted-in to the positioning system.

Take for example navigation, having collision shapes like this makes generating the navigation much easier, you just need to find those shapes, not extract data from the node using them

I'm not sure I fully understand what you mean here by "having collision shapes like this" and how it makes navigation calculations easier. The change in question suggests a unified compositional approach and collision shapes are currently compositional. Without further context, it seems to me like it would be at least equally easy to do navigation calculation.

In my opinion there's very little need to break with the standard idea of the tree hierarchy, especially often enough and in varied enough ways that the systems in place isn't enough, with some room for improvement

To be completely clear, I'm not advocating against using a tree-based hierarchy. A compositional approach still forms a tree and has parent/child relationships. I'm advocating against using a single tree for multiple purposes. To put it another way, I'm advocating for multiple trees. Systems (rendering, physics, positioning, object management, etc) should not attempt to all infuse meaning into the same relationships because they do not all treat those relationships the same way. Some systems allow the child to implement behavior in the parent, some allow the parent to implement behavior in the child. At the very least, this is unnecessary complexity, given that, as I've shown, each of these behaviors can be implemented following a standardized "relationship direction" across all systems.

Those are just my thoughts on it, I think specifically controlling behaviour that is this fundamental with composition is not feasible, and it risks opening up many unexpected pitfalls, having more flexibility in controlling some parts of the way the scene tree works would be interesting, I just think the suggested approach isn't the way to go

Thank you again for your time and thoughts on this, it's really helpful to have someone to discuss it with to find those flaws in reasoning that a singular perspective creates. While I disagree, and I think there are many examples across other engines, that what we're referring to as fundamental behavior is not feasible, I respect and appreciate your opinion to the contrary. More flexibility is one of my goals with this proposal, though the benefit I'd be most excited about would be the increase in simplicity through the reduction of one-off edge case properties and behaviors. If every functionality was implemented as a compositional node, anything could be enabled/disabled/reconfigured without additional conditional logic in the code and without additional documentation and onboarding/learning burden to newcomers.

[AThousandShips]

You wouldn't have to have a separate positioning system for collision shapes
The change in question suggests a unified compositional approach and collision shapes are currently compositional

You are missing my point, this wasn't about your proposal, but about why we keep collision shapes as nodes and not integrated into the collision object, I only brought this up because you referenced it as an argument against keeping away from this way of composition, I'm pointing out the logic and reasoning for the places where we do use composition, instead of properties and attached resources 🙂

For this I'd like to see some reasons why the system with Top Level, Visibility Parent, and RemoteTransform are not sufficient to handle this break with the normal structuring and hierarchy, and what ways this would be better, especially having multiple trees, which would make for a very, very convoluted logic, and for significant pitfalls, for example, how would you store this hierarchy? How would you handle ownership of nodes? How would you handle instances in the scene tree? Etc.

And again the end question is: What are the reasons why you need to break with the hierarchy?

But for some arguments:

• Adding new nodes to control behavior instead of properties will, always, increase complexity, and code size, not reduce it, mind you the logic of all of the existing features are in one place, it's not handled in every individual type, it's implemented in the common node
• I don't see how this approach, as opposed to properties, is more flexible
• Adding new nodes to the tree invariably increases complexity and reduces performance, that's simply a reality with the way the tree works, each Node has a non-insignificant common logic and overhead that will be present no matter what
• This will not be needed for the majority of nodes in the majority of projects, and will in those cases be unnecessary complexity and clutter

Moving processing out of a node and into a child node (assuming this isn't simply "if the node is there, do X") will slow down things, unless I'm missing some solution here in how to handle the logic of looking up and processing this for free, because with a property there's no overhead for this

But finally Godot simply does not use ECS, and this in my opinion falls squarely under the general idea of ECS, namely have behavior and details of how a particular component works governed by another object not part of it, this has been based on long experience and the established systematics of Godot

Please see this article if you didn't already read it