Skip to content

feat!: Update to new runners, update PyTorch, add ARM builds, add Blackwell support, add sglang image #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 95 commits into from
Feb 22, 2025
Merged

feat!: Update to new runners, update PyTorch, add ARM builds, add Blackwell support, add sglang image #87

merged 95 commits into from
Feb 22, 2025

Conversation

Eta0
Copy link
Collaborator

@Eta0 Eta0 commented Feb 14, 2025

Many updates (CI, building for more architectures, PyTorch libraries, etc.)

This PR is to merge an overly-long-running branch back to main comprising most updates to PyTorch and the CI system from the last few months. Some highlights include:

  • Switched to use newer GHA CI runners
  • Added support for multi-arch amd64+arm64 image builds (now the default)
    • The ml-containers/torch and ml-containers/torch-extras build processes were adapted to support arm64, to match
  • Dropped Ubuntu 20.04 builds
    • Python 3.8, included in Ubuntu 20.04, is now EOL and unsupported by PyTorch
  • Added CUDA 12.8 builds for PyTorch
  • Added PyTorch builds with compute capabilities 9.0a, 10.0, and 10.0a enabled (on applicable CUDA versions)
  • Added PyTorch builds with the CXX11 ABI disabled, in addition to the ones with it enabled (previous default)
    • This change will likely be reverted later, as PyTorch is planning to migrate to enabling the CXX11 ABI for their own builds, which will mean that keeping it enabled will lead to the best compatibility
  • Updated PyTorch (and friends) to v2.6.0
  • Updated many ml-containers/torch and ml-containers/torch-extras bundled libraries
  • Added Flash Attention v3 to the ml-containers/torch image
  • Added an sglang image

There were also various bugfixes, improvements, and customizability enhancements added along the way noted in commit messages.

Eta0 added 30 commits November 4, 2024 14:48
@Eta0
feat(torch): Update PyTorch to v2.5.1 & update CUDA 12.6
9a5990d
@Eta0
feat(torch): Update torch:nccl base images
eec6daa
@Eta0
ci(torch): Change torch:nccl matrix build layout
e411a3d
@Eta0
ci: Update action versions
8612038
@Eta0
ci: Use remote BuildKit worker & new runners
f58c927
@Eta0
ci(torch-nightly): Only filter specific fields from configs' include
a3f1d78
@Eta0
ci(torch-nightly): Change del() syntax in yq filter
e6bb688
@Eta0
ci(torch-nightly): Treat include as an array in yq filter
4fe66e2
@Eta0
ci(torch-nightly): Exclude ubuntu20.04 from torch-nightly builds
1c31940
@Eta0
ci(torch-nightly): Filter out entire include key for torch:base
092a837
@Eta0
ci: Build for multiple architectures
3448307 
[skip ci]
@Eta0
ci: Build only for linux/amd64
3b6b17d 
[skip ci]
@Eta0
fix(torch): Include a post-v2.5.1 bugfix patch when building PyTorch
35c959b
@Eta0
Merge branch 'es/torch-updates' into es/actions
0c5c0f0
@Eta0
feat(torch): Parameterize compiler_wrapper.f95
cdafd6b 
[skip ci]
@Eta0
fix(torch): Enable preprocessor when compiling compiler_wrapper.f95
e0bd93a 
[skip ci]
@Eta0
build(torch): Make the build process less architecture-dependent
074edd5 
[skip ci]
@Eta0
fix(torch): Use cmake after installing it instead of before
dc97cd6 
[skip ci]
@Eta0
build(torch): Allow customizing -march with --build-args
85f3b0e 
[skip ci]
@Eta0
build(torch): Allow customizing MAX_JOBS as a build arg
68121aa 
[skip ci]
@Eta0
build(torch): Don't apply custom MAX_JOBS to flash-attn build
d35022b 
[skip ci]
@Eta0
build(torch): Line-buffer grep output when building PyTorch
8305f7f 
[skip ci]
@Eta0
build(torch): Filter more output when building PyTorch
daa5a4b 
[skip ci]
@Eta0
build(torch): Allow customizing TransformerEngine build arches
c940ad4
@Eta0
feat(torch): Add flash-attn 3 beta
863fca5 
[skip ci]
@Eta0
build(torch): Filter even more output when building PyTorch
b4ce2da
@Eta0
build(torch-extras): Configure compiler_wrapper.f95 parameters
87c22ff
@Eta0
build(torch-extras): Allow overriding MAX_JOBS and NVCC_APPEND_FLAGS
9cb44f4 
[skip ci]
@Eta0
fix(torch-extras): Add missing + in parameter expansion
4a6f2a7 
[skip ci]
@Eta0
feat(torch-extras): Build DeepSpeed-Kernels
1dfcbc1 
[skip ci]
Eta0 added 6 commits February 7, 2025 01:14
@Eta0
build(sglang): Use USE_CUDNN and USE_CUSPARSELT flags in vLLM build
855d2f3
@Eta0
fix(sglang): Remove extraneous rmdir build step
dc70ca8
@Eta0
fix(sglang): Skip apt prompts
f113d38
@Eta0
ci(torch-nightly): Update runner image version
f63ddbe
@Eta0
ci: Parameterize build platforms
868a611 
[skip ci]
@Eta0
fix(torch): Filter 10.0 arch builds on unsupported CUDA versions again
0e44116 
The previous method didn't work when 10.0 was included in the
BUILD_TORCH_CUDA_ARCH_LIST build argument, so this uses shell
parameter expansion hackery to get around that.

This also keeps the previous logic, but switches it
to force sm_100a builds on supported CUDA versions.
@Eta0 Eta0 added bug Something isn't working enhancement New feature or request labels Feb 14, 2025
@Eta0 Eta0 requested a review from wbrown February 14, 2025 21:29
@Eta0 Eta0 self-assigned this Feb 14, 2025
wbrown
wbrown reviewed Feb 14, 2025
View reviewed changes
Copy link
Collaborator

@wbrown wbrown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heckin' big. One question, otherwsie looks great to me.

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

5 similar comments
@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351603
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

1 similar comment
@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351603
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

3 similar comments
@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@github-actions GitHub Actions
Copy link

@Eta0 Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/13338351608
Image: ``

@wbrown wbrown merged commit d6cea4b into main Feb 22, 2025
40 of 61 checks passed
@wbrown wbrown deleted the es/actions branch February 22, 2025 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wbrown wbrown wbrown approved these changes

Assignees

@Eta0 Eta0

Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Eta0 @wbrown