分享自己用 MiniMind 编写的精读笔记

Thank you to the author. The source code and training process of MiniMind are very complete, and I have basically followed it to learn the entire process of LLM. When reading, I took notes on minimind-deep-dive (https://GitHub.com/Enping-Hu/minimind-deep-dive), explaining each section of the source code, and trying to explain why each implementation is written in this way. However, I did not just stop at "how to write MiniMind": after reading each component, I extended it to the underlying technical context, such as from RMSNorm to the evolution of normalization, from GRPO to its variant family. This part has become more and more extensive, and it has been collected into an appendix separately. It covers topics that are included in MiniMind but not expanded (such as the extension of RoPE length, the compression of KV cache from MHA to MLA, the evolution of MoE), and also adds topics that are not covered by MiniMind but are indispensable for further development, such as quantization, speculative decoding, PagedAttention, RLHF overview, and the history of open-source models. The idea is to use MiniMind as an entry point rather than an endpoint: it is enough to read the source code and follow the main line, and if you want to go deeper, you can follow the links in the appendix. I also did two things: I made a source code diff (QK-Norm, removing the shared expert, changing the PPO five-model to four-model, defaulting to CISPO in GRPO) for MiniMind2 and MiniMind-3, and compared the training process using a chain of logits→loss→backward. I also ran each stage on the server, and the comparison between curves and fixed prompts is recorded in Chapter 10. The notes are now quite complete, and I will continue to update them with the things I learn. I saw that some tutorials and disassembly-type sharing were included in the acknowledgments. If it is appropriate, can we also put one here? If it is not appropriate, it is okay to put it here for reference.