Question:Is the model parameter update related to the reward？

[DouBi11]

I have a question: according to the code calculation method, this reward is unrelated to the model updates. During the training process of OPT-125M, the reward oscillates around a certain value and does not show a clear trend of converging in a specific direction.

[victor7246]

The reward function only uses the intrinsic measure of the weight matrices to be pruned. However, you can add a custom reward/penalty function as well. Optimizing the KS distance can be tricky for certain matrices without adequate hyperparameter tuning.

[DouBi11]

The reward function only uses the intrinsic measure of the weight matrices to be pruned. However, you can add a custom reward/penalty function as well. Optimizing the KS distance can be tricky for certain matrices without adequate hyperparameter tuning.
You mean the current reward function isn't directly linked to the model's updates? But shouldn't reinforcement learning ensure that the reward increases as the loss decreases?