V0.3版本,运行时出错,如下: #1231
iseebrowser
started this conversation in
General
Replies: 1 comment
-
解决了,是GIT_LFS_SKIP_SMUDGE=1 git clone https://hf-mirror.com/Qwen/Qwen3-30B-A3B下载的tokenizer.json文件错误。在windows下载后上传到Ubuntu解决。错误的文件内容是133字节。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
python ktransformers/server/main.py
--architectures Qwen3MoeForCausalLM
--host 0.0.0.0 --port 6006 --api_key htxt-ai
--model_name Qwen3-30B-A3B
--model_path /data/models/Qwen3-30B-A3B
--gguf_path /data/models/Qwen3-30B-A3B-GGUF/BF16
--optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve-amx.yaml
--temp 0.6
--backend_type balance_serve
2025-04-29 19:35:00,800 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
flash_attn not found, flashinfer unit test needed it. If you are using balance serve, ignore this.
set start method
Traceback (most recent call last):
File "/home/blackfog/ktransformers/ktransformers/server/main.py", line 122, in
main()
File "/home/blackfog/ktransformers/ktransformers/server/main.py", line 109, in main
create_interface(config=cfg, default_args=cfg)
File "/home/blackfog/miniconda3/envs/ktransformers/lib/python3.12/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface
GlobalInterface.interface = BackendInterface(default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/blackfog/miniconda3/envs/ktransformers/lib/python3.12/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 309, in init
self.tokenizer = AutoTokenizer.from_pretrained(args.model_dir, trust_remote_code=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/blackfog/miniconda3/envs/ktransformers/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1009, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/blackfog/miniconda3/envs/ktransformers/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2062, in from_pretrained
return cls._from_pretrained(
^^^^^^^^^^^^^^^^^^^^^
File "/home/blackfog/miniconda3/envs/ktransformers/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2302, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/blackfog/miniconda3/envs/ktransformers/lib/python3.12/site-packages/transformers/models/qwen2/tokenization_qwen2_fast.py", line 120, in init
super().init(
File "/home/blackfog/miniconda3/envs/ktransformers/lib/python3.12/site-packages/transformers/tokenization_utils_fast.py", line 117, in init
fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception: expected value at line 1 column 1
Beta Was this translation helpful? Give feedback.
All reactions