A way to force many immediate physics object transformation updates

[wareya]

Describe the project you are working on

A first person shooter (https://github.com/wareya/Airwalker)

Describe the problem or limitation you are having in your project

I'm running into several problems with raycasts and physics tests because they use out-of-date positions from the end of the previous physics frame rather than the current physics frame, even if force_update_transform is called. In particular, this means that:

1. moving platforms are always going to use information from one frame in the past, causing stutter and/or visible rubber banding, no matter what
2. the raycasts used by infinitely small projectiles like rockets (yes, they must be infinitely small in my game) need to have a forwards collision margin and also do point tests on their past and future positions, because the edges of the player's collision hull can travel through them between physics updates, even at very low speeds.
3. I cannot implement lag compensation, because implementing it involves updating the position of every object in the world based on how hard the attacker is lagging and doing physics tests against the updated object positions immediately

Now, I could work around points 2 and 3 by implementing my own raycasting system that loops over every player in the game and does low-level raycasting math against their hull shape, but that's extremely silly to do inside of a fully-featured game engine, and something that I should not need to even consider doing.

(My project runs on godot 3, but godot 4 has the same exact problem.)

It could be that force_update_transform is just bugged right now (see research in first response), but even if it is, there's room for improvement here: a lot of the time, you want to force the transforms for lots of objects at once, and that can be done more efficiently if the broadphase is only updated once.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Add two new methods to PhysicsServer:

bool force_update_transforms(Array objects)

 Forcibly update the transforms of the given CollisionObjects, updating their child shapes,
 with their new positions being available to interactions from other physics objects
 immediately. The broadphase is also updated with respect to the given objects immediately.

bool force_set_transforms(Array object_rids, Array transforms)

 Does the same thing as `force_update_transforms`, but with RIDs and Transforms rather than
 with CollisionObjects.

If the PhysicsServer should not interact with or know about CollisionObjects, then the first method can be scrapped, leaving only the second method.

Equivalent methods would be added to Physics2DServer, as well.

The purpose of these methods is similar in spirit to the workaround to a similar problem posted here: godotengine/godot#30481 (comment). But this approach allows the game developer to queue up all necessary transformation changes to be executed at once, so that the broadphase update is minimally expensive.

These functions will allow game developers to perform physics tests between multiple moving objects at the same time, move objects in round-robin rather than lockstep, perform collision checks between multiple different time-traveled versions of their objects, do "evil"-but-sometimes-necessary things like send moving platforms to their new positions immediately rather than waiting until the next frame, etc, all without changing how normal code currently works or causing any backwards compatibility issues like a change to force_update_transform would.

Yes, forcibly updating a bunch of collision object transforms along with the broadphase, multiple times per frame, can be somewhat expensive, but you simply must be able to sometimes do inherently expensive things to make non-buggy games. Yes, it can cause bugs in the hands of an inexperienced developer, but the current situation of not being able to do it at all is also causing bugs.

TODO:

Should the broadphase be updated immediately? Should it be possible to, as a gdscript user, forcibly update transforms without updating the broadphase, and then update the broadphase in a second pass afterwards? Is there any benefit to being able to do so? Maybe if you could set an object's broadphase aabb to whatever you want, so you could trace through a bunch of different positions for that object without having to update its broadphase state but also without getting false negatives?

Would these functions be threadsafe or force a mutex stall or anything like that?

Should there be a way to get the "current" physics transforms for a list of objects so that they can be reset to their original transform with no side effects? Or is body_get_direct_state sufficient?

Should it be possible to immediately, forcibly update other parameters of PhysicsDirectBodyState (e.g. velocities) this way, or is supporting it just for the transform sufficient?

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Test project: PhysicsSyncTest.zip

This code:

            # does something, but not what I need!
            $"../Body".force_update_transform()
            
            force_raycast_update()

Would look like this instead:

            PhysicsServer.force_update_transforms([$"../Body"])
            force_raycast_update()

Or:

            PhysicsServer.force_set_transforms([$"../Body".get_rid()], [$"../Body".global_transform])
            force_raycast_update()

If this enhancement will not be used often, can it be worked around with a few lines of script?

Working around this for moving objects being hit by raycasts involves doing your own low-level raycasting code, for every possible shape type, including handling all the transformations yourself, and throwing away all of the benefits of the physics system, like the broadphase.

Working around this for all collision tests, rather than just raycasts, across all collision object types, is an untractably complex task, basically equivalent to writing your own physics server.

Is there a reason why this should be core and not an add-on in the asset library?

Can't be an add-on until the physics server is made fully pluggable, and even then it would need some amount of explicit engine support.

[wareya]

Research: Godot Physics and Bullet handle force_update_transform differently. In the z case of the example project (the project I uploaded here, not the FPS), calling force_update_transform on the physics body before updating the raycast makes no difference to the results with Godot Physics. With Bullet, it does make a difference, as it should, but the x case is still broken. If I had to guess, force_update_transform just doesn't do anything with Godot Physics, and with Bullet it doesn't cause the broadphase to be updated like it should.

This doesn't have an effect on this proposal, because this proposal provides a way of updating physics transforms that allows broadphase updates to be more efficient, but it's worth looking into.

[Calinou]

Related to #2821 (possible duplicate?).

[wareya]

I think that proposal overlaps with this one but is not redundant. That proposal is about controlling how physics ticks work (including rigidbody simulation?) to make it easier to do input prediction. Input prediction is a prerequisite for lag compensation, but is not lag compensation itself. This proposal is about controlling just specific state updates so that things like lag compensation are easier to implement. The state updates that lag compensation needs to do are not physics simulation steps and can be approached much more simply. You could emulate that proposal with this one, but I think it would be inefficient (and very ugly), and using that proposal to do this one would definitely be inefficient.

[MJacred]

related to #2332

[RavenmoreGIT]

I ran into the same problem:

• I was building a turn-based roguelike with movement on a square grid.
• I figured I can just use the engine to check if a position is free using colliders and raycasts.
• This does not work because after my entities move their colliders get updated on the next physics frame.
• All entities move within one frame, so their collider checks use info from the last frame, and there seems to be no way to force an update to their collider positions.

Solution would be to either introduce a sort of non-physics collider that gets updated on request, or add an option to force a collider update.

As it is as a stopgap measure I would have to keep an array of entities and use that to check position...but the engine already does that!