Updated example to generate .dot file of ggml_cgraph

[taronaeo]

Last example I saw was #589 (comment) and it appears the codebase has changed so much that it appears it no longer resides in src/llama.cpp. Closest hit for that patch is in src/llama-context.cpp where

llama.cpp/src/llama-context.cpp

Lines 1045 to 1048 in 982e347

	// plot the computation graph in dot format (for debugging purposes)
	//if (n_past%100 == 0) {
	// ggml_graph_dump_dot(gf, NULL, "llama.dot");
	//}

still exists.

Uncommenting that code throws the following error as well

/opt/llama.cpp/src/llama-context.cpp: In member function ‘int llama_context::decode(const llama_batch&)’:
/opt/llama.cpp/src/llama-context.cpp:1046:13: error: ‘n_past’ was not declared in this scope
 1046 |         if (n_past%100 == 0) {
      |             ^~~~~~
/opt/llama.cpp/src/llama-context.cpp:1047:33: error: ‘gf’ was not declared in this scope
 1047 |             ggml_graph_dump_dot(gf, NULL, "llama.dot");
      |

Do we have an updated example for generating the dot file? I'm trying to visualise the cgraph

[CISC]

Ouch, this is the risk of keeping commented out code. :)

This returns the graph result, so should be res instead of gf I don't think you can retrieve the graph in this function any more:

llama.cpp/src/llama-context.cpp

Line 1012 in 982e347

const auto res = process_ubatch(ubatch, LLM_GRAPH_TYPE_DECODER, mctx.get(), status);

Unsure what n_past is supposed to be...

[taronaeo]

Unfortunate. I poked around further and did this patch since I'm testing our IBM Granite model:

diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index a322fc39..3212d896 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -14008,7 +14008,7 @@ struct llm_build_granite : public llm_graph_context {
 
         ggml_tensor * inp_out_ids = build_inp_out_ids();
 
-        for (int il = 0; il < n_layer; ++il) {
+        for (int il = 0; il < 1; ++il) {
             ggml_tensor * inpSA = inpL;
 
             // norm
@@ -14052,6 +14052,8 @@ struct llm_build_granite : public llm_graph_context {
         res->t_logits = cur;
 
         ggml_build_forward_expand(gf, cur);
+        ggml_graph_print(gf);
+        ggml_graph_dump_dot(gf, NULL, "granite.dot");
     }
 
     ggml_tensor * build_attention_layer(

Received a Segmentation fault (core dumped) but it generated a dot file. Extracting it shows this

Not sure if this patch is correct or wrong since it looks less inline as compared to the example @/ggerganov provided.

[CISC]

Apart from being rendered vertically instead of horizontally it looks right.

[ggerganov]

To avoid the segfault, use this:

for (int il = n_layer-1; il < n_layer; ++il) {