Speculative decoding for DeepSeek-v3? #12895

segmond started this conversation in Ideas

segmond
Apr 11, 2025

https://huggingface.co/lmsys/DeepSeek-V3-0324-NextN They are using this to perform speculative decoding for v3 with sglang, is such a thing possible for llama.cpp?

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment