You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Training parameters can be configured via environment variables.
10
10
11
11
- Required:
12
-
-`TORCHTITAN_ROOT`
12
+
-`TORCHTITAN_ROOT`: Root directory of torchtitan in your local filesystem
13
13
- Optional:
14
-
-`RECIPE`: rowwise|tensorwise. defaults to tensorwise.
15
-
-`BATCH_SIZE`: defaults to 1.
16
-
-`STEPS`: defaults to 100.
14
+
-`FLOAT8_RECIPE_WITH_BEST_SETTINGS`: "rowwise" or "tensorwise". Applies float8 training with the specified scaling recipe, as well as additional training configs which are optimal for that scaling recipe. See `float8_training_benchmark.sh` for more details.
15
+
-`BATCH_SIZE`: Defaults to 1.
16
+
-`STEPS`: Defaults to 100.
17
17
18
18
**NOTE**: `torch.compile` and FSDP2 are always used. Other forms of parallelism supported in torchtitan are not yet supported in this script.
0 commit comments