How to rewrite the compute graph in the backend

[marty1885]

Hi,

I've asked this question in the GGML discussion board with no reply so decided to try the llama.cpp one

I'm working on a NPU backend and the NPU support 1D and 2D convolutions as a native functionality. AFAIK GGML implements convolution as a combination of im2col and matmul. I looked into the backend data structure and found graph_plan_create but it is documented to be not used right now.

Is there a place I can walk and rewrite the compute graph? So I can transform im2col + matmul into corresponding Conv1D and Conv2D operations.

Thanks

[ggerganov]

There is a discussion about how to add support for fusing operations: #5413

However, I don't think we have a good answer yet for the specific case of ggml_conv operators - see #5413 (comment). So at the moment, I can't recommend anything, but hopefully we will figure out how to do it

[marty1885]

Thanks. I'll read the discussion thread and see if I can contribute anything.