Skip to content

Conversation

drcege
Copy link
Collaborator

@drcege drcege commented Oct 21, 2024

Summary

  • Unified SFT data format
    • add query_key, response_key, history_key in base_op
  • New OPs & unit-tests
    • generate_qa_from_examples_mapper
    • generate_qa_from_text_mapper
    • optimize_qa_mapper
    • optimize_query_mapper
    • optimize_response_mapper
  • Fixed model inference (use chat_template)
    • use chat interface for vllm
    • use pipeline for HF
  • Updated docs
  • Optimized model loading
    • load model directly to GPU instead of moving it to CPU first
    • properly configure vllm to load only in tensor parallel mode
    • preload only models without file locks
    • sort prepare_*_model functions

@drcege drcege marked this pull request as ready for review October 22, 2024 07:40
@drcege drcege added the enhancement New feature or request label Oct 23, 2024
@drcege
Copy link
Collaborator Author

drcege commented Oct 23, 2024

Note: extract_qa_mapper 改为 generate_qa_from_text_mapper 后输出格式变化,不再产出 chatml 的 json string,而是分解为具体的 sft 样本,注意后续对齐。

@drcege drcege requested a review from BeachWang October 23, 2024 08:48
@drcege drcege marked this pull request as ready for review October 29, 2024 03:04
@drcege drcege changed the title align sft formats & new ops [Ready] align sft formats & new ops Oct 31, 2024
Copy link
Collaborator

@BeachWang BeachWang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@drcege drcege merged commit 65d7c91 into main Nov 5, 2024
3 checks passed
@yxdyc yxdyc mentioned this pull request Dec 10, 2024
3 tasks
@HYLcool HYLcool deleted the sft/align_ops branch February 24, 2025 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants