suggestion of adding torch.profile

[frankchieng]

i just check out the code of appendix-A/01_main-chapter-code /DDP-script.py,how about adding

from torch.profiler import profile
with profile() as prof:
    #the main function training code
if rank == 0:
    print("exporting trace")
    prof.export_chrome_trace("trace_ddp_simple.json")

than we can see the tracing profiling json file in google Chrome

[rasbt]

I like that idea. I actually planned to add bonus material on profiling once all the chapters are finished. It didn't quite make it into Appendix A because it was originally a main chapter, and the text was already borderline too long. So that's why I will do it as a standalone resource profiling codes from an existing chapter. I am not sure yet whether I will do it as a write-up or a video. Thanks for suggesting though!