GGML Shape & Strides question #1111
akapoor3518
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi,
I was looking GGML code it has ne & nb array under tensor structure. ne is used for shape & nb is used for stride. I have following questions:
1 For Llama.cpp application how we are tuning shape & stride for tensor for particular GGML operation. The reason i asked Base on this information we will general Kernel code for our custom hardware which will depend on ne & nb value.
2 Do we have any recommendation for shape & stride for Matrix Multiplication , does it defer for every model?
3 Can backend can control ne & nb value for tensor or it can be only controlled o trough llama.cpp application?
4 Our Customer hardware kernel are fixed size of ne & nb, once we generate the custom kernel, ne & nb cant change hence we would like to know what is recommendation for ne & nb to model run using llama.cpp or ollama application
Thanks,
Anoop
Beta Was this translation helpful? Give feedback.
All reactions