[Bug]: 基于 XPU 平台使用 PaddleNLP 微调后的 checkpoint 无法通过 from_pretrained 加载

[YoctoHan]

软件环境

- paddle2onnx                   2.0.1
- paddlefsl                     1.1.0
- paddlenlp                     3.0.0b4
- paddlepaddle                  3.2.0
- paddlepaddle-xpu              3.2.0
- paddleslim                    2.6.0

重复问题

• I have searched the existing issues

错误描述

在昆仑芯 P800 平台上基于 PaddlePaddle 和 PaddleNLP 微调后的 checkpoint 无法通过 from_pretrained 正常加载。

稳定复现步骤 & 代码

复现步骤

启动微调

#!/bin/bash

# 定义日志目录
LOG_DIR="/workspace/sft/logs"
LOG_FILE="${LOG_DIR}/finetune_$(date +%Y%m%d_%H%M%S).log"

# 创建日志目录（如果不存在）
mkdir -p "${LOG_DIR}"

# 切换到工作目录
cd /workspace/PaddleNLP/llm || exit 1

# 执行脚本，后台运行并将日志存储到文件
nohup python -u -m paddle.distributed.launch \
    --gpus "0,1,2,3,4,5,6,7" \
    run_finetune.py /workspace/PaddleNLP/llm/config/aiXcoder/sft_argument.json \
    > "${LOG_FILE}" 2>&1 &

# 提示用户日志位置
echo "Training started. Logs are being written to: ${LOG_FILE}"

# 实时查看日志
tail -f "${LOG_FILE}"

微调结束后使用 from_pretrained 加载权重

from paddlenlp.transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("/workspace/sft/sft_ckpts/", dtype="float32")

输出如下：

[2025-09-18 11:33:46,454] [    INFO] - We are using <class 'paddlenlp.transformers.llama.modeling.LlamaForCausalLM'> to load '/workspace/sft/sft_ckpts/'.
[2025-09-18 11:33:46,454] [    INFO] - Loading configuration file /workspace/sft/sft_ckpts/config.json
[2025-09-18 11:33:46,455] [    INFO] - Loading weights file /workspace/sft/sft_ckpts/model_state.pdparams
[2025-09-18 11:34:00,894] [    INFO] - Loaded weights file from disk, setting weights to model.
W0918 11:34:00.904455 727857 xpu_context.cc:187] Please NOTE: xpu device: 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/dist-packages/paddlenlp/transformers/auto/modeling.py", line 798, in from_pretrained
    return cls._from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddlenlp/transformers/auto/modeling.py", line 346, in _from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddlenlp/transformers/model_utils.py", line 2567, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys = cls._load_pretrained_model(
  File "/usr/local/lib/python3.10/dist-packages/paddlenlp/transformers/model_utils.py", line 2254, in _load_pretrained_model
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM:
        Skip loading for lm_head.weight. lm_head.weight receives a shape [49152, 4096], but the expected shape is [4096, 49152].
        You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.

稳定复现，看报错应该是一个权重张量的形状发生了转置。

[YoctoHan]

使用以下转换脚本调整了 lm_head.weight 的shape 之后可以正常启动了，转换脚本如下：

import paddle

# 参数文件路径
state_dict_path = "/workspace/sftbak/sft_ckpts/model_state.pdparams"

# 1. 加载参数
state_dict = paddle.load(state_dict_path)

# 检查有没有目标参数
if "lm_head.weight" not in state_dict:
    raise KeyError("state_dict 中没有找到 lm_head.weight")

print("原始形状:", state_dict["lm_head.weight"].shape)

# 2. 转置参数
state_dict["lm_head.weight"] = state_dict["lm_head.weight"].T

print("转置后形状:", state_dict["lm_head.weight"].shape)

# 3. 保存覆盖原文件
paddle.save(state_dict, state_dict_path)

print("已完成转置并覆盖保存到", state_dict_path)

但是转换完成后遇到了新的问题，就是模型仅能在非常短的输入下进行推理，输入稍微长一些就会报错，因为报错在 XDNN，排查起来可能比较麻烦：

XCCL /usr/local/lib/python3.10/dist-packages/paddle/base/../libs/libbkcl.so loaded
[16:25:27][tjdm-p800-vd-xhl-node12.tjdm][820708:820708][WARN][BKCL][globals.cpp:268] xccl version: 6ab4ffb [rdma] [XPUQA-232: update xtdk/xre version] build data: Jul 24 2025 10:02:10
/usr/local/lib/python3.10/dist-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
正在加载模型: /workspace/sftbak/sft_ckpts
[2025-09-18 16:25:29,456] [    INFO] - We are using <class 'paddlenlp.transformers.llama.modeling.LlamaForCausalLM'> to load '/workspace/sftbak/sft_ckpts'.
[2025-09-18 16:25:29,456] [    INFO] - Loading configuration file /workspace/sftbak/sft_ckpts/config.json
[2025-09-18 16:25:29,457] [    INFO] - Loading weights file /workspace/sftbak/sft_ckpts/model_state.pdparams
file: /workspace/sftbak/sft_ckpts/model_state.pdparams is paddle weight.
[2025-09-18 16:25:42,950] [    INFO] - Loaded weights file from disk, setting weights to model.
W0918 16:25:42.966739 820708 xpu_context.cc:187] Please NOTE: xpu device: 0
[2025-09-18 16:26:48,891] [    INFO] - All model checkpoint weights were used when initializing LlamaForCausalLM.

[2025-09-18 16:26:48,892] [    INFO] - All the weights of LlamaForCausalLM were initialized from the model checkpoint at /workspace/sftbak/sft_ckpts.
If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.
[2025-09-18 16:26:48,894] [    INFO] - Loading configuration file /workspace/sftbak/sft_ckpts/generation_config.json
✅ 模型加载成功
==================================================
✅ aiXcoder 聊天机器人已启动
  输入 'q', 'quit' 或 'exit' 可退出
  输入 'clear' 清除对话历史
  输入 'help' 查看帮助信息
==================================================

用户：quick sort
[2025-09-18 16:27:00,434] [ WARNING] - model.generation_config is in conflict with model.config, model.config is used.

aiXcoder：Here is an implementation of the quick sort algorithm in Python:

def quick_sort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[0]
    lesser = [x for x in arr[1:] if x <= pivot]
    greater = [x for x in arr[1:] if x > pivot]
    return quick_sort(lesser) + [pivot] + quick_sort(greater)

# Example usage
arr = [7, 2, 4, 1, 5, 7, 3, 9, 2, 4, 6]
sorted_arr = quick_sort(arr)
print(sorted_arr)


Output:
[1, 2, 2, 3, 4, 4, 5, 6, 7, 7, 8, 9]


In this implementation, the `quick_sort` function takes an array `arr` as input and recursively sorts it using the quick sort algorithm. The base case is when the length of the array is less than or equal to 1, in which case it is already considered sorted and returned as is.

In the recursive case, the function selects the first element of the array as the pivot. It then creates two new arrays, `lesser` and `greater`, which contain all elements less than or equal to the pivot and all elements greater than the pivot, respectively. The function then recursively calls `quick_sort` on the `lesser` and `greater` arrays and concatenates the sorted `lesser` array, the pivot, and the sorted `greater` array to obtain the final sorted array.

Note that the time complexity of quick sort is O(n log n) in the average case and O(n^2) in the worst case, where n is the number of elements in the array.

用户：Please write a C++ implementation of quicksort.

❌ 发生错误：(External) strided_copy XDNN Error <1>, XDNN_INVALID_PARAM  (at /paddle/paddle/phi/kernels/xpu/strided_copy_kernel.cc:84)

推理脚本：

"""
aiXcoder 聊天机器人
基于 PaddleNLP 的对话生成系统
"""

import os
import sys
from typing import List, Tuple, Optional, Dict, Any
from dataclasses import dataclass
import paddle
from paddlenlp.transformers import AutoTokenizer, AutoModelForCausalLM


@dataclass
class GenerationConfig:
    """生成配置类"""
    max_new_tokens: int = 1024
    min_new_tokens: int = 1
    top_p: float = 0.7
    temperature: float = 0.95
    repetition_penalty: float = 1.0
    num_beams: int = 1
    early_stopping: bool = True
    max_length: int = 2048


class AIXcoderChat:
    """aiXcoder 聊天机器人类"""
    
    def __init__(
        self,
        model_path: str,
        device: str = 'gpu',
        dtype: str = 'float32',
        generation_config: Optional[GenerationConfig] = None
    ):
        """
        初始化聊天机器人
        
        Args:
            model_path: 模型路径
            device: 设备类型 ('gpu' 或 'cpu')
            dtype: 数据类型
            generation_config: 生成配置
        """
        self.model_path = model_path
        self.device = device
        self.generation_config = generation_config or GenerationConfig()
        
        # 设置 Paddle 配置
        paddle.set_device(device)
        paddle.set_default_dtype(dtype)
        
        # 加载模型和分词器
        self._load_model()
        
    def _load_model(self):
        """加载模型和分词器"""
        try:
            print(f"正在加载模型: {self.model_path}")
            self.tokenizer = AutoTokenizer.from_pretrained(self.model_path)
            # 设置 pad_token
            if self.tokenizer.pad_token is None:
                self.tokenizer.pad_token = self.tokenizer.eos_token
            self.model = AutoModelForCausalLM.from_pretrained(self.model_path, convert_from_torch=True, dtype="float32")
            self.model.eval()
            print("✅ 模型加载成功")
        except Exception as e:
            print(f"❌ 模型加载失败: {str(e)}")
            raise
    
    def _prepare_input(self, query: str) -> Dict[str, paddle.Tensor]:
        """
        准备模型输入
        
        Args:
            query: 用户输入的问题
            
        Returns:
            包含 input_ids, attention_mask, position_ids 的字典
        """
        # 编码
        encoded_input = self.tokenizer(
            query,
            return_tensors="pd",
            padding=True,
            truncation=True,
            max_length=self.generation_config.max_length
        )
        
        # 获取 input_ids
        input_ids = encoded_input['input_ids']
        batch_size, seq_len = input_ids.shape
        
        # 创建 attention_mask
        if 'attention_mask' in encoded_input:
            attention_mask = encoded_input['attention_mask']
        else:
            pad_token_id = getattr(self.tokenizer, 'pad_token_id', 0)
            attention_mask = (input_ids != pad_token_id).astype('int64')
        
        # 创建 position_ids
        position_ids = paddle.arange(seq_len, dtype='int64').unsqueeze(0).expand([batch_size, seq_len])
        
        return {
            'attention_mask': attention_mask,
            'input_ids': input_ids,
            'position_ids': position_ids
        }
    
    def _decode_output(self, outputs: paddle.Tensor) -> str:
        """
        解码模型输出
        
        Args:
            outputs: 模型输出的 tensor
            
        Returns:
            解码后的文本
        """
        if isinstance(outputs, paddle.Tensor):
            output_ids = outputs.numpy()
        else:
            output_ids = outputs[0].numpy()
            
        if len(output_ids.shape) > 1:
            output_ids = output_ids[0]
            
        output_ids = [int(x) for x in output_ids.tolist()]
        response_text = self.tokenizer.decode(output_ids, skip_special_tokens=True)
        
        # 处理特殊的结束标记
        if "_<AIX" in response_text:
            response_text = response_text.split("_<AIX")[0].strip()
        
        return response_text
    
    def chat(self, query: str, history: Optional[List[Tuple[str, str]]] = None) -> Tuple[str, List[Tuple[str, str]]]:
        """
        处理单轮对话
        
        Args:
            query: 用户输入的问题
            history: 对话历史，列表，结构为[(问, 答), ...]
            
        Returns:
            response: 模型的回复
            new_history: 更新后的对话历史
        """
        # 准备输入
        encoded_input = self._prepare_input(query)
        
        # 推理
        outputs = self.model.generate(
            input_ids=encoded_input["input_ids"],
            attention_mask=encoded_input.get("attention_mask", None),
            max_new_tokens=self.generation_config.max_new_tokens,
            min_new_tokens=self.generation_config.min_new_tokens,
            top_p=self.generation_config.top_p,
            temperature=self.generation_config.temperature,
            repetition_penalty=self.generation_config.repetition_penalty,
            num_beams=self.generation_config.num_beams,
            early_stopping=self.generation_config.early_stopping
        )
        
        # 解码输出
        response = self._decode_output(outputs)
        
        # 更新历史
        if history is None:
            history = []
        history.append((query, response))
        
        return response, history


class InteractiveChat:
    """交互式聊天界面"""
    
    def __init__(self, chatbot: AIXcoderChat):
        """
        初始化交互式聊天
        
        Args:
            chatbot: AIXcoderChat 实例
        """
        self.chatbot = chatbot
        self.history = []
        
    def print_welcome(self):
        """打印欢迎信息"""
        print("=" * 50)
        print("✅ aiXcoder 聊天机器人已启动")
        print("  输入 'q', 'quit' 或 'exit' 可退出")
        print("  输入 'clear' 清除对话历史")
        print("  输入 'help' 查看帮助信息")
        print("=" * 50)
        
    def print_help(self):
        """打印帮助信息"""
        print("\n帮助信息:")
        print("- 直接输入问题进行对话")
        print("- 'q', 'quit', 'exit': 退出程序")
        print("- 'clear': 清除对话历史")
        print("- 'help': 显示此帮助信息")
        
    def run(self):
        """运行交互式聊天"""
        self.print_welcome()
        
        while True:
            try:
                # 获取用户输入
                query = input("\n用户：").strip()
                
                # 处理特殊命令
                if query.lower() in ['q', 'quit', 'exit']:
                    print("  再见！")
                    break
                elif query.lower() == 'clear':
                    self.history = []
                    print(" ️ 对话历史已清除")
                    continue
                elif query.lower() == 'help':
                    self.print_help()
                    continue
                elif not query:
                    print("⚠️ 请输入内容")
                    continue
                
                # 进行对话
                response, self.history = self.chatbot.chat(query, self.history)
                print(f"\naiXcoder：{response}")
                
            except KeyboardInterrupt:
                print("\n\n  检测到 Ctrl+C，退出程序。")
                break
            except Exception as e:
                print(f"\n❌ 发生错误：{str(e)}")
                continue


def main():
    """主函数"""
    # 模型配置
    model_path = "/workspace/sftbak/sft_ckpts"  # 模型路径
    device = 'xpu'  # 使用 GPU
    
    # 创建生成配置
    gen_config = GenerationConfig(
        max_new_tokens=1024,
        min_new_tokens=1,
        top_p=0.7,
        temperature=0.95,
        repetition_penalty=1.0,
        num_beams=1,
        early_stopping=True
    )
    
    try:
        # 创建聊天机器人
        chatbot = AIXcoderChat(
            model_path=model_path,
            device=device,
            generation_config=gen_config
        )
        
        # 创建交互式聊天界面并运行
        interactive_chat = InteractiveChat(chatbot)
        interactive_chat.run()
        
    except Exception as e:
        print(f"❌ 程序启动失败：{str(e)}")
        sys.exit(1)


if __name__ == '__main__':
    main()