how to add an extra fixed tensor to the token embedding in gpt2 arch

[Francis235]

Hi, I want to know how to add an extra fixed tensor to the token embedding. My model is based on gpt2, and my input format is '[mel][bos][token][eos]', I need to add my mel_embed(a fixed vector, such as 1x1600) to the embedding of '[mel][bos][token][eos]', the python code is:

hidden_states = inputs_embeds + position_embeds
hidden_states = torch.cat([mel_embeds, hidden_states], dim=1)

I try to create a tensor in build_gpt2 as the following:

std::string inp_mel_path = "mel-token.txt";
        std::vector<float> inp_mel_data;
        read_txt_to_vec(inp_mel_path, inp_mel_data);
        std::cout << "inp_mel_data size: " << inp_mel_data.size() << std::endl;
        std::cout << "inp_mel_data[0]: " << inp_mel_data[0] << std::endl;
        std::cout << "inp_mel_data[1599]: " << inp_mel_data[1599] << std::endl;
        // create tensor from vector<float>
        int64_t emb_size = 1600; 
        struct ggml_tensor * inp_mel = ggml_new_tensor_1d(ctx0, GGML_TYPE_F32, emb_size);
        inp_mel->data = new float[emb_size];
        for (int i0 = 0; i0 < emb_size; i0++) {
            ((float *)inp_mel->data)[i0] = inp_mel_data[i0];
        }
        inp_mel = ggml_cont(ctx0, ggml_reshape_4d(ctx0, inp_mel, emb_size, 1, 1, 1));
inpL = ggml_concat(ctx0, inp_mel, inpL, 1);

but I get the following error:
llama-cli: /workspace/llama.cpp/ggml/src/ggml-backend.c:1574: ggml_backend_sched_split_graph: Assertion `src_backend_id != -1' failed.

I think I should create the inp_me tensor in advance just like the model weight and bias created in llm_load_tensors(), but I don't know how to do that. Any suggestion? Thanks in advance.