Skip to content

Commit 8ad1c44

Browse files
authored
Update README.md (#521)
Added Known Issues section to document meta 405B FP 8 model mem fault and work around.
1 parent e26141f commit 8ad1c44

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

docs/dev-docker/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,9 @@ Pull the most recent validated docker image with `docker pull rocm/vllm-dev:main
2626
- Detokenizer disablement
2727
- Torch.compile support
2828

29+
## Known Issues and Workarounds
30+
- Mem fault encountered when running the model meta 405 fp8. To workaround this issue, set PYTORCH_TUNABLEOP_ENABLED=0
31+
2932
## Performance Results
3033

3134
The data in the following tables is a reference point to help users validate observed performance. It should not be considered as the peak performance that can be delivered by AMD Instinct™ MI300X accelerator with vLLM. See the MLPerf section in this document for information about MLPerf 4.1 inference results. The performance numbers above were collected using the steps below.

0 commit comments

Comments
 (0)