Would appreciate tips on understanding the project structure and prerequisites for reading the source code!

[rikoras]

I am very excited about this amazing project, which allows me to try many innovations, and I want to thank Georgi and all contributors.

I have already deployed llama.cpp on some devices and have run some examples. It works well, but I am not satisfied with only running some demos. I want to study the project comprehensively. Furthermore, I want to try some optimizations for specific applications (if there is an idea).

Here are the skills I have got:

• C/C++ programming. I can do some embedded programming with C/C++, and I have taken a few courses on computer system and architecture.

• Basic understanding of Llama. I have read some related papers including Attention is all you need and Llama.int8, I know how the Llama model works, and I can finish a model implementation with Pytorch.

• Ability to read manuals. English is not my native language (sorry if you feel frustrated while reading this 😔), but I can try my best.

And here are my limitations:

• I have trouble getting started on a relatively large project. I have worked on some student projects, but when it comes to a real-world project, I feel lost amid plenty of details (makefiles, utils, and so on). I want to have an overview of llama.cpp and I may need some help.

• I am not familiar with parallel programming libraries such as MPI and OpenMP.

• I don't know much about the ggml repo.

I would really appreciate some advice, thanks!

[KerfuffleV2]

I guess it depends in part what sort of stuff you want to do. For example, do you want to write or modify applications that depend on llama.cpp as a library, or do you want to add features to the library that applications can consume?

The way I got started was just making some pretty simple, self-contained contributions. If you can find some small thing you want to fix or improve, that's one way to start getting familiar.

You could also review examples/simple — that's basically the simplest example of using llama.cpp as a library from C/C++.

Just curious, is your native language Mandarin?

[rikoras]

My main interest is about quantization. Anyway, I am trying to review example/simple , and it does seem like a good starting point!

You're right, my native language is Mandarin. I hope there won't be too many difficulties in communication : ).

[KerfuffleV2]

My main interest is about quantization.

Like changing existing quantizations? Or adding a new one?

You're right, my native language is Mandarin. I hope there won't be too many difficulties in communication : ).

Not for me! I just wondered since I saw some Chinese characters in your GitHub profile and I'm actually working on learning the language. Your English is pretty advanced and unfortunately my Mandarin isn't so if you had trouble understanding the English version of something you'd probably get better results just using machine translation rather than trying asking me in Mandarin. Otherwise I'd offer to help that way.

[rikoras]

Like changing existing quantizations? Or adding a new one?

I would like to contribute to existing quantizations. Recently, I've been following related research, and I get the sense that there is still much work to be done to get better performance on edge devices. Perhaps I can make some subtle optimizations.
I am also intrigued by some cool concepts like 1-bit quantization. However, as a beginner, I believe I need more practice before becoming a pioneer.

I'm actually working on learning the language.

That would be great! Wish you could make a good progress🙂.