About Goal Guided Generation

[shallowdream1002]

Hi! Thank you for sharing your work!
When I want to test goal-guided trajectory generation, I encountered some problems. I set the task_mask to 1 at the last valid timestamp for one agent. But the generated trajectory is not reasonable, I tried to change the initial noise for 10 times, but not work. My setting is as following:

I run the pipeline.py directly, I use to log name 2021.05.12.22.00.38_veh-35_01008_01518. It is in nuplan mini dataset. And I use the nuplan.ckpt set in pipeline.py update_config function. In the infer_model function, I add one line bp_mask[:, 14, -1, :] = 1. I set the last timestamp of the 15th agent 0 noise. Then I run the pipeline.py. The pred trajectory of the 15th agent is like this,

In the last timestamp, the velocity of the agent is obviously unreasonable, the speed suddenly increases, the pred gif is:

I am confused, is it normal for this model? or what's wrong with my settings? Looking forward your reply. Thank you very much!

[zhouyunsong]

Hi, I’m curious how your goal point is defined — is it just a randomly selected point along the trajectory?

We evaluated the model's ability to follow the goal point, and the results show that trajectory generation tends to perform well when the goal is reasonably aligned with the history in terms of speed and direction.
However, when the goal point deviates significantly from the historical trajectory — especially when it falls outside the original data distribution (e.g., involving extreme acceleration or deceleration rarely seen during training) — the model tends to fail.

[shallowdream1002]

I directly set the last point of the trajectory ground truth value as the goal point. The goal point and the ground truth value are completely consistent. In the pipeline.py, I add one line between line 523 and line 524:

Is my setup correct? I run the pipeline.py with nuplan.ckpt and log name 2021.05.12.22.00.38_veh-35_01008_01518 in nuplan mini dataset, then I got the gif result above. I found that the model predicted results much slower than the ground truth.

[zhouyunsong]

In theory, it should be possible to add it this way, as long as it’s implemented correctly.
In my current setup within pipeline.py, I only implemented unconditional prediction — I haven’t tried the goal-conditioned setting, so I’m not entirely sure if it aligns with what’s done in visualization_nexus_callback.py.
In principle, you'd need to pay attention to whether task_mask, valid_mask, and the local_context(very important) are correctly set.

Another possible reason is that pipeline.py is designed as a simple demo, and it doesn’t include any constraint mechanisms — for example, SMA, which is often considered helpful for improving trajectory coherence, is not applied here.