Seeking Advice: Implementing or Ideas for Streaming Rendering in Milkdown (AI-Assisted Writing Scenario)

[xz-lovewj]

Hi community!

Has anyone successfully implemented streaming rendering for the Milkdown editor? Or does anyone have good ideas or approaches on how to achieve this?

My specific use case is: I'm trying to integrate an AI assistant (e.g., an LLM) with Milkdown to help users auto-complete or continue writing documents. For this to work well, I need to implement:

Streaming Output: The text generated by the AI assistant is returned incrementally, word-by-word or chunk-by-chunk (not as one complete block).

Streaming Rendering: The Milkdown editor needs to render (insert) these incremental chunks into the document at the current cursor position in real-time and smoothly, providing a good user experience (e.g., cursor tracking, scroll position maintenance).

I'm currently exploring how to synchronize this streaming output with the editor's rendering. I'd love to hear your thoughts on:

Are there any existing Milkdown plugins or extensions that implement similar functionality?

If not, how would you approach architecting or modifying Milkdown to achieve this? For instance:

What's an efficient and safe way to insert the arriving text fragments into the editor's document model?

How to ensure the editor's state (cursor, selection, history) remains correct and usable during continuous insertion?

How to handle potential network latency or interruptions to maintain rendering continuity?

Any suggestions for performance optimization or debouncing/throttling?

Any relevant ideas or experiences are highly welcome!

Thanks in advance for your insights and suggestions!

大家好！

请问社区里是否有朋友成功实现过 Milkdown 编辑器的流式渲染 (streaming rendering)？或者对于如何实现这一点有什么好的思路或想法吗？

我的具体场景是：我正在尝试集成一个智能助手（例如 LLM）到 Milkdown 中，用于辅助用户续写文档。在这个过程中，我需要实现：

流式输出 (Streaming Output): 智能助手生成的文本内容是逐字、分批返回的（不是一次性返回整段）。

流式渲染 (Streaming Rendering): Milkdown 编辑器需要能够实时、流畅地将这些分批返回的文本内容渲染（插入）到当前文档的光标位置，并保持良好的用户体验（如光标跟随、滚动条定位）。

我目前正在思考如何实现这种输出与渲染的“流式”同步。想请教大家：

是否有现成的 Milkdown 插件或扩展实现了类似功能？

如果没有现成的，大家觉得应该如何架构或改造 Milkdown 来实现这种需求？比如：

如何高效、安全地将分批到达的文本片段插入到编辑器的文档模型中？

如何确保在连续插入过程中编辑器的状态（光标、选区、历史记录）保持正确和可用？

如何处理可能出现的网络延迟或中断，保证渲染的连贯性？

是否有性能优化或防抖/节流方面的建议？

非常期待听到大家的经验分享、建议或任何相关的想法！感谢！

[gweesin]

As you know, lots of editors currently display auto-completions as ghost text, similar to GitHub Copilot or Cursor. However, I’m not sure if this approach provides the best user experience.

[xz-lovewj]

这种流式渲染功能，不应类似于需要询问用户是否接受建议的编码智能助手，而应更像一个写手——当用户通过唤起命令（如续写、优化等需求）触发它时，它会基于当前进度，以类似打印机输出的模式自动完成剩余部分。完成后，用户可通过快捷键或其他操作回退到修改前的版本。当需要微调时，用户也可直接在已生成的内容上进行修改。

My idea is that this streaming rendering functionality should not resemble a coding assistant that constantly asks for acceptance of its suggestions. Instead, it should function more like a writer – when activated by a user command (such as "continue", "optimize", or other requests), it automatically completes the remaining content based on the current progress in a printer-like mode. Once finished, users should be able to revert to a pre-modified version via shortcuts or other actions. When fine-tuning is needed, users can make edits directly on the generated content.

[xz-lovewj]

对于这些流式渲染，我们可以定义一些自定义的 markdown 语法，甚至可以将这些语法作为提示词，提前告知 AI 助手哪些格式是可以识别的。这样一来，在 AI 继续生成内容时，我们就无需再次整理格式，当结果令人满意时也更为方便。当然，可能还有其他更好的想法。
我对前端技术了解有限，对 remark 和 ProseMirror 生态也不够熟悉，而且英语水平有限，因此没有太多思路。希望这里的朋友们能提供一些好的建议或已有的相关案例供参考。

For these stream-based renderings, we can define custom Markdown syntax, and even use these syntaxes as prompts to inform the AI assistant in advance about which formats it can recognize. This way, when the AI continues generating content and the result meets our expectations, we won't need to spend extra effort on formatting. Of course, there might be many better ideas.
I am not very proficient in front-end technologies, nor do I have deep knowledge of the remark and ProseMirror ecosystems. Additionally, my English skills are limited, so I don’t have many ideas myself. I’d like to see if others here have better suggestions or existing similar cases for reference.