[SimpleFSDP] Note on DCP checkpoint saving/loading (#1280)

Per this previous conversation:
#1273 (comment),
we found there will be additional all-gathers when saving/loading DCP in
simplefsdp.

I'm leaving a note here and will update after @fegin's checkpointing
upgrading.


![Screenshot 2025-06-10 at 10 53
14 AM](https://github.com/user-attachments/assets/748b3b68-4062-4840-b1e0-318a42d813ca)

Author: ruisizhang123

torchtitan/experiments/simple_fsdp/simple_fsdp.py

@@ -128,9 +128,11 @@ def _register_parametrization(
     module: nn.Module, param_names: List[str], parametrization: nn.Module
 ):
     """
-    it works with state_dict without incurring parametrization calls because
+    It works with state_dict without incurring parametrization calls because
     state_dict accesses parameters directly from self._parameters, not from getters
     https://github.com/pytorch/pytorch/blob/main/torch/nn/modules/module.py#L2141
+    TODO: In checkpoint saving/loading, avoid parametrization calls when calling
+    get_model_state_dict func in torchtitan's torchtitan/components/checkpoint.py.
     """
     param_name_to_property = {
         param_name: property(lambda self: parametrization(self._parameters[param_name]))