Skip to content

[Contributions] Submitting performance benchmarks #1289

Answered by tianyu-l
asaiacai asked this question in Q&A
Discussion options

You must be logged in to vote

Great idea! This is more than welcome.

I would propose creating a dedicated folder benchmarks under root folder, where each file containing

  1. the person / entity / GH name
  2. hardware setup including GPUs, interconnections, etc.
  3. model / theme of benchmarking, e.g. llama3 / flux.1
  4. performance report, with configs used to reproduce, either via toml file or via this option https://github.com/pytorch/torchtitan/blob/main/torchtitan/config_manager.py#L47
  5. versions of torchtitan, torch, torchao, or any relevant dependency

Please give feedbacks, so that I can create the folder with a template / guidance to unblock your submission.

Replies: 1 comment 4 replies

Comment options

You must be logged in to vote
4 replies
@tianyu-l
Comment options

@asaiacai
Comment options

@tianyu-l
Comment options

@asaiacai
Comment options

Answer selected by asaiacai
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants