Suggestion: Integrate MoR (Mixture of Reasoning) into Chapter 3 or as a bonus

[gzqccnu]

Hello! First and foremost, thank you deeply for your incredible work on this project—especially Chapter 3, which has been an invaluable resource for understanding transformer mechanisms clearly and comprehensively.

I’m writing to share an exciting recent development: DeepMind has published new research on MoR (Mixture of Reasoning), a framework that shows significant advantage in enhancing model reasoning capabilities. From what I’ve gathered, MoR’s key strengths include:

• Parameter Sharing: Significantly reduce the number of parameters through weight binding.
• Computational Routing: Avoid redundant computations (and reduce ineffective FLOPs) via token routing.
• Recursion-Level Caching: Cache key-value pairs on demand to reduce memory access overhead and improve throughput efficiency.

Given these advantages, I wonder if you would consider integrating a section on MoR into Chapter 3 or as a bonus. It could complement the existing content by connecting foundational concepts to cutting-edge reasoning techniques, enriching readers’ understanding of advanced reasoning frameworks.

Here's link of MoR: Paper, Code

[rasbt]

Hi there,
I am glad to hear that you liked the book! Also, congrats on the paper, and thanks for suggesting! However, this would not be a good fit here for now as I want to keep this project a bit more foundational (versus covering the latest research trends). I am not saying that I am not interested in the latest research trends, but I prefer to cover them in my blog (https://magazine.sebastianraschka.com/)

[gzqccnu]

Got it. Looking forward to your blog about it!