Critical Bug: AnticipatedNetworkVariable.Smooth() overtime in game, ends up making AnticipatedNetworkVaariable.Value out of sync, it does not match the AuthoritativeValue!

[Xeynt]

Description

I'm trying to implement Anticipation to my game which features Player Characters that moves using rb.AddForce(). As such, I'm not using neither AnticipatedNetworkTransform nor the default NetworkTransform (I'm using AnticipatedNetworkVariable for the rotation and movement of the player character).

I'm running the following setup in my Player script:
Player.cs

    public Vector3 position;
    public override void OnReanticipate(double lastRoundTripTime)
    {
        base.OnReanticipate(lastRoundTripTime);
        Debug.Log("Reanticipating");
        SmoothAnticipateNetworkVariable(rbRotation, 0.25f, Quaternion.Slerp);
        SmoothAnticipateNetworkVariable(rbPosition, 0.25f, Vector3.Slerp);
        position = rbPosition.AuthoritativeValue;
        //rb.rotation = rbRotation.Value;
        //Rb.position = rbPosition.Value;
    }

    private void SmoothAnticipateNetworkVariable<T>(AnticipatedNetworkVariable<T> anticipatedNetworkVariable, float durationSeconds, AnticipatedNetworkVariable<T>.SmoothDelegate how)
    {
        if (anticipatedNetworkVariable.ShouldReanticipate)
        {
            anticipatedNetworkVariable.Smooth(anticipatedNetworkVariable.PreviousAnticipatedValue, anticipatedNetworkVariable.AuthoritativeValue, durationSeconds, how);
        }
    }

    public override void FixedUpdate()
    {
        base.FixedUpdate();
        if (IsLocalPlayer)
        {
            ClientFixedUpdate();
        }
        if (IsServer)
        {
            ServerFixedUpdate();
        }
        rb.rotation = rbRotation.Value;
        Rb.position = rbPosition.Value;
    }

    private void ServerFixedUpdate()
    {
        if (moveDirection != Vector2.zero)
        {
            Quaternion targetRotation = Quaternion.LookRotation(new Vector3(moveDirection.x, 0, moveDirection.y));
            //rbRotation.Anticipate(Quaternion.Slerp(rb.rotation, targetRotation, Time.deltaTime * 10f));
            rbRotation.Anticipate(targetRotation);
            rb.rotation = targetRotation;
            //rb.rotation = rbRotation.Value;
        }
     }

    void ClientFixedUpdate()
    {
        if (clientInputDirectionRelativeToCamera != Vector3.zero)
        {
            // Rotate towards movement direction
            Quaternion targetRotation = Quaternion.LookRotation(clientInputDirectionRelativeToCamera);
        rbRotation.Anticipate(targetRotation);
        
        //rbRotation.Anticipate(Quaternion.Slerp(rb.rotation, targetRotation, Time.deltaTime * 10f));
        }
    }

PlayerMoveState (Runs on LocalClients and Server)

    public override void OnFixedUpdate()
    {
        if (playerFiniteStateMachine.IsLocalPlayer)
        {
            playerFiniteStateMachine.Rb.CharacterMoveRelativeToCameraDirection(playerFiniteStateMachine.ClientInputDirectionRelativeToCamera, playerFiniteStateMachine.MovementSpeed, playerFiniteStateMachine.IsGrounded, ref previousTargetVelocity);
            playerFiniteStateMachine.RPCAnimatorSetFloat(Player.AnimationParameters.horizontalVelocity.ToStringCached(), (playerFiniteStateMachine.ClientInputDirectionRelativeToCamera.magnitude > 0F) ? Math.Clamp(playerFiniteStateMachine.NormalizeMovementSpeed, 0.01F, 999F) : 0F);
        }
        else
        {
            playerFiniteStateMachine.Rb.CharacterMoveRelativeToCameraDirection(playerFiniteStateMachine.inputDirectionRelativeToCamera, playerFiniteStateMachine.MovementSpeed, playerFiniteStateMachine.IsGrounded, ref previousTargetVelocity);
            playerFiniteStateMachine.RPCAnimatorSetFloat(Player.AnimationParameters.horizontalVelocity.ToStringCached(), (playerFiniteStateMachine.inputDirectionRelativeToCamera.magnitude > 0F) ? Math.Clamp(playerFiniteStateMachine.NormalizeMovementSpeed, 0.01F, 999F) : 0F);
        }

        //if (playerFiniteStateMachine.IsServer || playerFiniteStateMachine.IsLocalPlayer)

        if (playerFiniteStateMachine.IsServer && !playerFiniteStateMachine.IsLocalPlayer)
            Debug.Log(playerFiniteStateMachine.Rb.position);

        playerFiniteStateMachine.rbPosition.Anticipate(playerFiniteStateMachine.Rb.position);
}

The issue is AnticipatedNetworkVariable.Smooth() in my example above, over time does not lerp towards the AuthoritativeValue. It happens on several scenarios, one of them when the player hits another player and tries to push the other player away.

As we can see from the screenshots below, the Server is sending the correct position data BUT when the client tries to Smooth() the data in OnReanticipation, the value is not the same. I've checked and AuthoratitativeValue is correct on the client side. So it's certainly the Smooth() on the client side that is not syncing and making AnticipatedNetworkVariable go towards the Authorative Value.

This seems very much like a breaking bug?

Reproduce Steps

1. Do not attach AnticipatedNetworkTransform OR NetworkTransform to your player gameobject. Move & Rotate your player object with Rigidbody.AddForce & Sync over the network the position and rotation with AnticipatedNetworkVariable instead. Have a level and at least 2 players in the game. All Clients must only be sending inputs to the Server and the Server must be the only one that move the players using rb.AddForce.

I'm using this level assets (Though any level/3d models should be fine): https://assetstore.unity.com/packages/essentials/tutorial-projects/starter-pack-synty-polygon-stylized-low-poly-3d-art-156819?srsltid=AfmBOorri35RnAhfSmXrG20RSqmKrzXEjNApOCqUq4hMd4T0SOHrFVAN

Actual Outcome

AnticipatedNetworkVariable.Smooth does not sync with AnticipatedNetworkVariable.AuthoritativeValue, overtime the position goes incorrect. At the start there's no issues, move around with the players and gradually they will go out of sync in position.

Expected Outcome

AnticipatedNetworkVariable.Smooth should sync constantly to AnticipatedNetworkVariable.AuthoritativeValue.

A clear and concise description of what you expected to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

• OS: Windows 11 64 bit
• Unity Version: Unity 6
• Netcode Version: 2.3.2
• Netcode Commit: Downloaded directly from PackageManager.

Additional Context

Add any other context about the problem here. Logs, code snippets would be useful here but please also consider attaching a minimal Unity project that reproduces the issue.

[Xeynt]

On further testing, when I remove the following:

        rb.rotation = rbRotation.Value;
        Rb.position = rbPosition.Value;

From FixedUpdate ^

And place it inside OnReanticipate() right after Smooth() is executed as seen here:

    private void SmoothAnticipateNetworkVariable<T>(AnticipatedNetworkVariable<T> anticipatedNetworkVariable, float durationSeconds, AnticipatedNetworkVariable<T>.SmoothDelegate how)
    {
        if (anticipatedNetworkVariable.ShouldReanticipate)
        {
            anticipatedNetworkVariable.Smooth(anticipatedNetworkVariable.PreviousAnticipatedValue, anticipatedNetworkVariable.AuthoritativeValue, durationSeconds, how);
                        rb.rotation = rbRotation.Value;
            Rb.position = rbPosition.Value;
        }
    }

The character moves only about half it's intended distance towards the authoritativeValue or rather, the movement speed of the character is halved, and it stop moving as soon as the player stops pressing the input so it goes out of sync. This proofs that either OnReanticipate or Smooth is causing the issue since .Value doesn't reach the AuthoritativeValue.

[NoelStephensUnity]

Are you using a client-server or distributed authority network topology?

[Xeynt]

Are you using a client-server or distributed authority network topology?

@NoelStephensUnity I'm using a client-server network topology. 1 player acts as the host while other players connect to the host. Player inputs are sent to the host/server and the server applies force to the rigidbodies. Position and rotation would be synced back to all clients.

I had this working with the default NetworkTransform previously, but upon discovering the new Anticipate system, I've removed the NetworkTransform and NetworkRigidbody, and am now using solely AnticipatedNetworkVariable and for position and rotation syncing respectively.

[Xeynt]

Are you using a client-server or distributed authority network topology?

@NoelStephensUnity I'm using a client-server network topology. 1 player acts as the host while other players connect to the host. Player inputs are sent to the host/server and the server applies force to the rigidbodies. Position and rotation would be synced back to all clients.

I had this working with the default NetworkTransform previously, but upon discovering the new Anticipate system, I've removed the NetworkTransform and NetworkRigidbody, and am now using solely AnticipatedNetworkVariable and for position and rotation syncing respectively.

Along with the info above, I've updated the reproduction steps in the main post to this:
Reproduce Steps
Do not attach AnticipatedNetworkTransform OR NetworkTransform to your player gameobject. Move & Rotate your player object with Rigidbody.AddForce & Sync over the network the position and rotation with AnticipatedNetworkVariable instead. Have a level and at least 2 players in the game. All Clients must only be sending inputs to the Server and the Server must be the only one that move the players using rb.AddForce.

[NoelStephensUnity]

Ahh I missed that... thank you for the additional information.

Unfortunately, the AnticipatedNetworkTransform was not written to account for synchronizing Rigidbodies and was really more of an example/starting point to having a form of predicted movement with rollback. With that said...

The server is going to be synchronizing deltas based on the transform and not the PhysX transform (Rigidbody) and the clients will be doing the same (which if you want to have proper kinematic to non-kinematic collisions the Rigidbody.MovePosition and Rigidbody.MoveRotation should be used) which means the clients are updating the GameObject transform during Update. It would not surprise me that you would get skewed results after a period of time using AnticipatedNetworkTransform and Rigidbodies.

This issue is most likely due to the AnticipatedNetworkTransform's implementation being primarily written to handle non-rigidbody motion without consideration of Rigidbodies (unfortunately).

I could turn this issue into a feature request if you would like?

However, if you want to use Rigidbody motion and have it properly synchronize... then I would use NetworkTransform with NetworkRigidbody and enable "User Rigidbody for Motion" on the NetworkRigidbody.