Skip to content

Add Qwen3 Moe #2260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Add Qwen3 Moe #2260

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e528378
qwen3 moe init
kanpuriyanawab May 19, 2025
84043a3
bug fixes
kanpuriyanawab May 20, 2025
750412c
update
kanpuriyanawab May 20, 2025
9b3d779
Merge branch 'keras-team:master' into qwen3_moe
kanpuriyanawab May 24, 2025
6b74171
address comments
kanpuriyanawab May 24, 2025
730a9c4
address comments
kanpuriyanawab May 27, 2025
5f90d10
update output matching script
kanpuriyanawab May 27, 2025
cda9cfc
fix test
kanpuriyanawab May 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions keras_hub/api/models/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,12 @@
from keras_hub.src.models.qwen.qwen_tokenizer import (
QwenTokenizer as QwenTokenizer,
)
from keras_hub.src.models.qwen3_moe.qwen3_causal_lm_preprocessor import (
Qwen3MoeCausalLMPreprocessor as Qwen3MoeCausalLMPreprocessor,
)
from keras_hub.src.models.qwen3_moe.qwen3_moe_backbone import (
Qwen3MoeBackbone as Qwen3MoeBackbone,
)
from keras_hub.src.models.qwen_moe.qwen_moe_backbone import (
QwenMoeBackbone as QwenMoeBackbone,
)
Expand Down
3 changes: 3 additions & 0 deletions keras_hub/api/tokenizers/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@
from keras_hub.src.models.qwen.qwen_tokenizer import (
QwenTokenizer as QwenTokenizer,
)
from keras_hub.src.models.qwen3_moe.qwen3_moe_tokenizer import (
Qwen3MoeTokenizer as Qwen3MoeTokenizer,
)
from keras_hub.src.models.qwen_moe.qwen_moe_tokenizer import (
QwenMoeTokenizer as QwenMoeTokenizer,
)
Expand Down
17 changes: 17 additions & 0 deletions keras_hub/src/models/qwen3_moe/qwen3_causal_lm_preprocessor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
from keras_hub.src.api_export import keras_hub_export
from keras_hub.src.models.causal_lm_preprocessor import CausalLMPreprocessor
from keras_hub.src.models.qwen3_moe.qwen3_moe_backbone import Qwen3MoeBackbone
from keras_hub.src.models.qwen3_moe.qwen3_moe_tokenizer import Qwen3MoeTokenizer


@keras_hub_export(
[
"keras_hub.models.Qwen3MoeCausalLMPreprocessor",
]
)
class Qwen3MoeCausalLMPreprocessor(CausalLMPreprocessor):
backbone_cls = Qwen3MoeBackbone
tokenizer_cls = Qwen3MoeTokenizer

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
Loading
Loading