feat(runtimes): Add LoRA/QLoRA/DoRA support in LLM Trainer V2#2832

Merged

google-oss-prow[bot] merged 28 commits intokubeflow:masterfrom

Electronic-Waste:feat/lora-support

Oct 15, 2025

Commits on Oct 14, 2025

feat(runtimes): Add lora recipe and config in torch.plugin.
Electronic-Waste
committed
chore: add --trainer-use-lora flag and related logics.
Electronic-Waste
committed
feat(lora): Add support for QLoRA.
Electronic-Waste
committed
fix(lora): fix extra rdzv backend in single device mode.
Electronic-Waste
committed
test(lora): Add UTs for lora/qlora in trainingruntime.
Electronic-Waste
committed
fix(lora): remove extra quote symbol in lora attn module.
Electronic-Waste
committed
fix(lint): fix lint error.
Electronic-Waste
committed
fix(runtime): check default GPU resources allocated in CTRs.
Electronic-Waste
committed
chore(runtime): add getNumProcPerNode.
Electronic-Waste
committed
fix(runtime): remove specific vendor for GPU.
Electronic-Waste
committed
test(runtime): add UTs for lora validation.
Electronic-Waste
committed
fix(runtimes): remove extra flags for torchtune.
Electronic-Waste
committed
Update pkg/runtime/framework/plugins/torch/torch.go

Electronic-Waste
and
andreyvelich
committed
chore(runtime): add extractGPUCountFromRuntime.
Electronic-Waste
committed
fix(runtimes): remove filtering args logics.
Electronic-Waste
committed
fix(runtimes): use fallbackNumProcPerNode if GPU is set in runtimes.
Electronic-Waste
committed
refactor(runtimes): refactor getNumProcPerNode logics.
Electronic-Waste
committed
fix(lint): fix lint error.
Electronic-Waste
committed
fix(runtimes): call getNumProcPerNode only once.
Electronic-Waste
committed
test(runtime): fix UTs for torch.
Electronic-Waste
committed
fix(runtimes): add limits to calculation.
Electronic-Waste
committed
refactor(runtimes): reorg nppNode related logics.
Electronic-Waste
committed
fix(runtimes): fix lints.
Electronic-Waste
committed
fix(runtimes): update condition that use nppNode in Limits.
Electronic-Waste
committed
refactor(runtimes): add torchtune.go
Electronic-Waste
committed
fix(lint): fix lint error.
Electronic-Waste
committed
refactor(torch): only reset nppNode when using CPU.
Electronic-Waste
committed
chore(runtimes): remove getNumProcPerNodeFromCPU.
Electronic-Waste
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(runtimes): Add LoRA/QLoRA/DoRA support in LLM Trainer V2#2832

feat(runtimes): Add LoRA/QLoRA/DoRA support in LLM Trainer V2#2832
google-oss-prow[bot] merged 28 commits intokubeflow:masterfrom
Electronic-Waste:feat/lora-support

Commits on Oct 14, 2025

feat(runtimes): Add lora recipe and config in torch.plugin.

chore: add --trainer-use-lora flag and related logics.

feat(lora): Add support for QLoRA.

fix(lora): fix extra rdzv backend in single device mode.

test(lora): Add UTs for lora/qlora in trainingruntime.

fix(lora): remove extra quote symbol in lora attn module.

fix(lint): fix lint error.

fix(runtime): check default GPU resources allocated in CTRs.

chore(runtime): add getNumProcPerNode.

fix(runtime): remove specific vendor for GPU.

test(runtime): add UTs for lora validation.

fix(runtimes): remove extra flags for torchtune.

Update pkg/runtime/framework/plugins/torch/torch.go

chore(runtime): add extractGPUCountFromRuntime.

fix(runtimes): remove filtering args logics.

fix(runtimes): use fallbackNumProcPerNode if GPU is set in runtimes.

refactor(runtimes): refactor getNumProcPerNode logics.

fix(lint): fix lint error.

fix(runtimes): call getNumProcPerNode only once.

test(runtime): fix UTs for torch.

fix(runtimes): add limits to calculation.

refactor(runtimes): reorg nppNode related logics.

fix(runtimes): fix lints.

fix(runtimes): update condition that use nppNode in Limits.

refactor(runtimes): add torchtune.go

fix(lint): fix lint error.

refactor(torch): only reset nppNode when using CPU.

chore(runtimes): remove getNumProcPerNodeFromCPU.

feat(runtimes): Add LoRA/QLoRA/DoRA support in LLM Trainer V2#2832google-oss-prow[bot] merged 28 commits intokubeflow:masterkubeflow/trainer:masterfrom Electronic-Waste:feat/lora-supportElectronic-Waste/training-operator:feat/lora-supportCopy head branch name to clipboard

Commits

Commits on Oct 14, 2025

feat(runtimes): Add LoRA/QLoRA/DoRA support in LLM Trainer V2#2832
google-oss-prow[bot] merged 28 commits intokubeflow:masterfrom
Electronic-Waste:feat/lora-support