[examples/llama.android]How to use model+lora adapter #8478

mountgs · 2024-07-14T10:55:33Z

mountgs
Jul 14, 2024

My usage method:

llama_model_params model_params = llama_model_default_params();

//Model in gguf format
auto path_to_model = env->GetStringUTFChars(filename, 0);
//The LoRA in bin format generated using convert-lora-to-ggml.
auto path_to_lora = env->GetStringUTFChars(loraname, 0);

auto model = llama_load_model_from_file(path_to_model, model_params);
//Here path_base_model is passed in path_to_model
auto success = llama_model_apply_lora_from_file(model, path_to_lora, 1, path_to_model, 1);

Now,there will be an error when loading Lora

08-02 11:17:44.316 17216 17341 D LLamaAndroid: AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 1 | SVE = 0 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | 
08-02 11:17:44.317 17216 17341 I llama-android.cpp: Loading model from /storage/emulated/0/Android/data/com.example.llama/files/llama-13b.Q2_K.gguf
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: loaded meta data with 19 key-value pairs and 363 tensors from /storage/emulated/0/Android/data/com.example.llama/files/llama-13b.Q2_K.gguf (version GGUF V2)
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   0:                       general.architecture str              = llama
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   1:                               general.name str              = meta-llama-13b
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   2:                       llama.context_length u32              = 2048
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   3:                     llama.embedding_length u32              = 5120
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   4:                          llama.block_count u32              = 40
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 13824
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 40
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 40
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000001
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv  10:                          general.file_type u32              = 10
08-02 11:17:44.321 17216 17341 I llama-android.cpp: llama_model_loader: - kv  11:                       tokenizer.ggml.model str              = llama
08-02 11:17:44.326 17216 17341 I llama-android.cpp: llama_model_loader: - kv  12:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
08-02 11:17:44.335 17216 17341 I llama-android.cpp: llama_model_loader: - kv  13:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - kv  14:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - kv  15:                tokenizer.ggml.bos_token_id u32              = 1
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - kv  16:                tokenizer.ggml.eos_token_id u32              = 2
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - kv  17:            tokenizer.ggml.unknown_token_id u32              = 0
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - kv  18:               general.quantization_version u32              = 2
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - type  f32:   81 tensors
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - type q2_K:   81 tensors
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - type q3_K:  200 tensors
08-02 11:17:44.336 17216 17341 I llama-android.cpp: llama_model_loader: - type q6_K:    1 tensors
08-02 11:17:44.343 17216 17341 I llama-android.cpp: llm_load_vocab: special tokens cache size = 259
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_vocab: token to piece cache size = 0.1684 MB
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: format           = GGUF V2
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: arch             = llama
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: vocab type       = SPM
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_vocab          = 32000
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_merges         = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: vocab_only       = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_ctx_train      = 2048
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_embd           = 5120
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_layer          = 40
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_head           = 40
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_head_kv        = 40
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_rot            = 128
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_swa            = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_embd_head_k    = 128
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_embd_head_v    = 128
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_gqa            = 1
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_embd_k_gqa     = 5120
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_embd_v_gqa     = 5120
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: f_norm_eps       = 0.0e+00
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: f_norm_rms_eps   = 1.0e-06
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: f_clamp_kqv      = 0.0e+00
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: f_logit_scale    = 0.0e+00
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_ff             = 13824
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_expert         = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_expert_used    = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: causal attn      = 1
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: pooling type     = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: rope type        = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: rope scaling     = linear
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: freq_base_train  = 10000.0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: freq_scale_train = 1
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: n_ctx_orig_yarn  = 2048
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: rope_finetuned   = unknown
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: ssm_d_conv       = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: ssm_d_inner      = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: ssm_d_state      = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: ssm_dt_rank      = 0
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: model type       = 13B
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: model ftype      = Q2_K - Medium
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: model params     = 13.02 B
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: model size       = 5.06 GiB (3.34 BPW) 
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: general.name     = meta-llama-13b
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: BOS token        = 1 '<s>'
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: EOS token        = 2 '</s>'
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: UNK token        = 0 '<unk>'
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: LF token         = 13 '<0x0A>'
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_print_meta: max token length = 48
08-02 11:17:44.346 17216 17341 I llama-android.cpp: llm_load_tensors: ggml ctx size =    0.17 MiB
08-02 11:17:44.604  1633  4300 I SDM     : CWB config passed by cwb_client : tappoint 1  CWB_ROI : (996.000000 131.000000 1062.000000 208.000000)
08-02 11:17:44.644  1633 17342 I SDM     : HWCSession::NotifyCWBStatus: Notify the client about buffer status 0.
08-02 11:17:44.644  1633 17342 I SDM     : HWCSession::ProcessCWBStatus: CWB queue is empty. Display: 0
08-02 11:17:44.700  1910  2507 D VRR [OTI]: handleIdle from 0 to 1
08-02 11:17:44.700  1910  2507 D VRR [OTI-HW]: isStateReady 0
08-02 11:17:44.700  1910  2507 D VRR [OTI-HW]: rate range: min {id=0, hwcId=0, resolution=1440x3168, refreshRate=60.00 Hz, dpi=515.15x509.29, group=0} cur {id=0, hwcId=0, resolution=1440x3168, refreshRate=60.00 Hz, dpi=515.15x509.29, group=0} max {id=0, hwcId=0, resolution=1440x3168, refreshRate=60.00 Hz, dpi=515.15x509.29, group=0}
08-02 11:17:44.700  1910  2507 D VRR [OTI-HW]: keep rate.
08-02 11:17:44.700  1910  2507 D VRR [OTI-ADFR]: getBestCvtRate 0, modeTypeIsReady 1, currentFps 60
08-02 11:17:44.640  1113  1113 W [253241.994249]: [LCD][0][WARN][oplus_panel_cmdq_pack_status_reset:972] not in dsi mode, should not reset cmdq pack status
08-02 11:17:44.750 17216 17341 I llama-android.cpp: llm_load_tensors:        CPU buffer size =  5177.12 MiB
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.750 17216 17341 I llama-android.cpp: .
08-02 11:17:44.751 17216 17341 I llama-android.cpp: .
08-02 11:17:44.751 17216 17341 I llama-android.cpp: 
08-02 11:17:44.751 17216 17341 I llama-android.cpp: llama_apply_lora_from_file_internal: applying lora adapter from '/storage/emulated/0/Android/data/com.example.llama/files/ggml-adapter-model.bin' - please wait ...
08-02 11:17:44.752 17216 17341 I llama-android.cpp: llama_apply_lora_from_file_internal: r = 8, alpha = 16, scaling = 2.00
08-02 11:17:44.752 17216 17341 I llama-android.cpp: llama_apply_lora_from_file_internal: loading base model from '/storage/emulated/0/Android/data/com.example.llama/files/llama-13b.Q2_K.gguf'
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: loaded meta data with 19 key-value pairs and 363 tensors from /storage/emulated/0/Android/data/com.example.llama/files/llama-13b.Q2_K.gguf (version GGUF V2)
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   0:                       general.architecture str              = llama
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   1:                               general.name str              = meta-llama-13b
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   2:                       llama.context_length u32              = 2048
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   3:                     llama.embedding_length u32              = 5120
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   4:                          llama.block_count u32              = 40
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 13824
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 40
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 40
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000001
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv  10:                          general.file_type u32              = 10
08-02 11:17:44.756 17216 17341 I llama-android.cpp: llama_model_loader: - kv  11:                       tokenizer.ggml.model str              = llama
08-02 11:17:44.759 17216 17216 I Quality : Skipped: false 22 cost 370.3878 refreshRate 16543239 bit true processName com.example.llama
08-02 11:17:44.760 17216 17341 I llama-android.cpp: llama_model_loader: - kv  12:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
08-02 11:17:44.769 17216 17341 I llama-android.cpp: llama_model_loader: - kv  13:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - kv  14:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - kv  15:                tokenizer.ggml.bos_token_id u32              = 1
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - kv  16:                tokenizer.ggml.eos_token_id u32              = 2
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - kv  17:            tokenizer.ggml.unknown_token_id u32              = 0
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - kv  18:               general.quantization_version u32              = 2
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - type  f32:   81 tensors
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - type q2_K:   81 tensors
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - type q3_K:  200 tensors
08-02 11:17:44.771 17216 17341 I llama-android.cpp: llama_model_loader: - type q6_K:    1 tensors
08-02 11:17:44.771  1633  1868 I SDM     : HWDeviceDRM::SetupConcurrentWriteback: Tear down the Concurrent Writeback topology
08-02 11:17:44.775 17216 17341 W llama-android.cpp: llama_apply_lora_from_file_internal: warning: using a lora adapter with a quantized model may result in poor quality, use a f16 or f32 base model with --lora-base

08-02 11:17:46.343 17216 17341 F libc    : Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x799b5f9fa0 in tid 17341 (Llm-RunLoop), pid 17216 (m.example.llama)
08-02 11:17:46.448 17358 17358 I crash_dump64: obtaining output fd from tombstoned, type: kDebuggerdTombstoneProto
08-02 11:17:46.449  1290  1290 I tombstoned: received crash request for pid 17341
08-02 11:17:46.450 17358 17358 I crash_dump64: performing dump of process 17216 (target tid = 17341)
08-02 11:17:46.530  2936  3117 E OplusTouchDaemon: [OplusTouchDaemon][touchMonitorRoutine][79]: touchMonitor stop, isEnable:1, currentState:1
08-02 11:17:46.613 17358 17358 F DEBUG   : Process name is com.example.llama, not key_process
08-02 11:17:46.613 17358 17358 F DEBUG   : keyProcess: 0
08-02 11:17:46.613 17358 17358 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
08-02 11:17:46.613 17358 17358 F DEBUG   : Build fingerprint: 'OPPO/PHY110/OP565FL1:14/UP1A.231005.007/U.168b5cd_1d0-1:user/release-keys'
08-02 11:17:46.613 17358 17358 F DEBUG   : Revision: '0'
08-02 11:17:46.613 17358 17358 F DEBUG   : ABI: 'arm64'
08-02 11:17:46.613 17358 17358 F DEBUG   : Timestamp: 2023-08-02 11:17:46.454567027+0800
08-02 11:17:46.613 17358 17358 F DEBUG   : Process uptime: 12s
08-02 11:17:46.613 17358 17358 F DEBUG   : Cmdline: com.example.llama
08-02 11:17:46.613 17358 17358 F DEBUG   : pid: 17216, tid: 17341, name: Llm-RunLoop  >>> com.example.llama <<<
08-02 11:17:46.613 17358 17358 F DEBUG   : uid: 10340
08-02 11:17:46.613 17358 17358 F DEBUG   : tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
08-02 11:17:46.613 17358 17358 F DEBUG   : pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
08-02 11:17:46.613 17358 17358 F DEBUG   : signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x000000799b5f9fa0
08-02 11:17:46.613 17358 17358 F DEBUG   :     x0  000000799b5f9fa0  x1  b400007b46850000  x2  0000000000834000  x3  000000799b5f9fa0
08-02 11:17:46.613 17358 17358 F DEBUG   :     x4  b400007b47084000  x5  000000799be2dfa0  x6  b400007b47083ffc  x7  b400007b4d164fc0
08-02 11:17:46.613 17358 17358 F DEBUG   :     x8  000000799b5f9fa0  x9  0000000000834000  x10 0000000000000000  x11 0000000000000000
08-02 11:17:46.613 17358 17358 F DEBUG   :     x12 0000000000834000  x13 0000000000000001  x14 0000000000000000  x15 0000000000069000
08-02 11:17:46.613 17358 17358 F DEBUG   :     x16 0000007b4770d200  x17 0000007bef555a40  x18 0000007b48b42000  x19 b400007b4d350190
08-02 11:17:46.613 17358 17358 F DEBUG   :     x20 b400007ae8b9c080  x21 b400007b46850000  x22 b400007ae7488890  x23 0000007b4774e658
08-02 11:17:46.613 17358 17358 F DEBUG   :     x24 b400007ae73f1900  x25 b400007ae7488820  x26 b400007b4d3607d0  x27 b400007ae8b9ed60
08-02 11:17:46.613 17358 17358 F DEBUG   :     x28 0000000000000000  x29 0000007b4d6d2cf0
08-02 11:17:46.613 17358 17358 F DEBUG   :     lr  0000007ad7469aa8  sp  0000007b4d6d2b80  pc  0000007bef555b38  pst 0000000020001000
08-02 11:17:46.613 17358 17358 F DEBUG   : 56 total frames
08-02 11:17:46.613 17358 17358 F DEBUG   : backtrace:
08-02 11:17:46.613 17358 17358 F DEBUG   :       #00 pc 0000000000095b38  /apex/com.android.runtime/lib64/bionic/libc.so (__memcpy_aarch64_simd+248) (BuildId: f5f7a5dceb6ab17a1a448521eb52ce18)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #01 pc 00000000000d9aa4  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk!libllama.so (offset 0xee8000) (llama_model_apply_lora_from_file+3328) (BuildId: 1e5b05464f1384b642653407363d4b3e148d42ca)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #02 pc 0000000000050580  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk!libllama-android.so (offset 0x454000) (Java_android_llama_cpp_LLamaAndroid_load_1model_1apply_1lora+196) (BuildId: 063f3fa9c8c75e3f3459541bd5c0b24a46614fad)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #03 pc 00000000002b4e30  /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+144) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #04 pc 000000000029e3a4  /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+612) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #05 pc 000000000029b044  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+1048) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #06 pc 00000000003f1524  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+2068) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #07 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #08 pc 000000000000308c  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk (android.llama.cpp.LLamaAndroid.access$load_model_apply_lora+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #09 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #10 pc 000000000029b620  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2548) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #11 pc 00000000003f1334  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+1572) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #12 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #13 pc 0000000000002740  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk (android.llama.cpp.LLamaAndroid$load$2.invokeSuspend+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #14 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #15 pc 000000000029b620  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2548) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #16 pc 00000000003f12b0  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+1440) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #17 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #18 pc 0000000000289490  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk (kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #19 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #20 pc 00000000002b8a54  /apex/com.android.art/lib64/libart.so (artQuickToInterpreterBridge+1076) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #21 pc 00000000002b4f68  /apex/com.android.art/lib64/libart.so (art_quick_to_interpreter_bridge+88) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #22 pc 0000000002008808  /memfd:jit-cache (deleted) (offset 0x2000000) (kotlinx.coroutines.DispatchedTask.run+2024)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #23 pc 000000000029e3a4  /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+612) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #24 pc 000000000029b044  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+1048) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #25 pc 00000000003f12b0  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+1440) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #26 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #27 pc 0000000000248bbc  /apex/com.android.art/javalib/core-oj.jar (java.util.concurrent.ThreadPoolExecutor.runWorker+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #28 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #29 pc 000000000029b620  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2548) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #30 pc 00000000003f12b0  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+1440) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #31 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #32 pc 0000000000247a58  /apex/com.android.art/javalib/core-oj.jar (java.util.concurrent.ThreadPoolExecutor$Worker.run+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #33 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #34 pc 000000000029b620  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2548) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #35 pc 00000000003f12b0  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+1440) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #36 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #37 pc 0000000000002970  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk (android.llama.cpp.LLamaAndroid$runLoop$1$1.invoke+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #38 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #39 pc 000000000029b620  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2548) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #40 pc 00000000003f12b0  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+1440) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #41 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #42 pc 0000000000002930  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk (android.llama.cpp.LLamaAndroid$runLoop$1$1.invoke+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #43 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #44 pc 000000000029b620  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2548) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #45 pc 00000000003f12b0  /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp<false>(art::interpreter::SwitchImplContext*)+1440) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #46 pc 00000000002b75d8  /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+8) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #47 pc 00000000002870a4  /data/app/~~CHQN8FPYF5ezSssZ0JiTRw==/com.example.llama-3MJPVjBz6lDiIm5sKYqEbg==/base.apk (kotlin.concurrent.ThreadsKt$thread$thread$1.run+0)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #48 pc 00000000002b9118  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238.llvm.207842230480696810)+240) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #49 pc 00000000002b8a54  /apex/com.android.art/lib64/libart.so (artQuickToInterpreterBridge+1076) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #50 pc 00000000002b4f68  /apex/com.android.art/lib64/libart.so (art_quick_to_interpreter_bridge+88) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #51 pc 000000000029e3a4  /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+612) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #52 pc 0000000000361c04  /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+148) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #53 pc 00000000004b9314  /apex/com.android.art/lib64/libart.so (art::Thread::CreateCallback(void*)+940) (BuildId: f6680171f023987ef88876e0991b7ed5)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #54 pc 0000000000104fe4  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+208) (BuildId: f5f7a5dceb6ab17a1a448521eb52ce18)
08-02 11:17:46.613 17358 17358 F DEBUG   :       #55 pc 000000000009e77c  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: f5f7a5dceb6ab17a1a448521eb52ce18)
08-02 11:17:46.629  1290  1290 E tombstoned: Tombstone written to: tombstone_03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[examples/llama.android]How to use model+lora adapter #8478

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

[examples/llama.android]How to use model+lora adapter #8478

Uh oh!

mountgs Jul 14, 2024

Replies: 0 comments

mountgs
Jul 14, 2024