Failed to execute script 'koboldcpp' due to unhandled exception!

[Vl-Kawai]

Hello!
I am tryed to run koboldcpp.exe with Alpaca ggml-model-q4_1.bin but it "Failed to execute script 'koboldcpp' due to unhandled exception!"
What can I do to solve this?

I have 16 Gb RAM and core i7 3770k if it important.

[henk717]

Your CPU does not have AVX2 listed as a supported instruction, have you tried the noavx2 version?

[prostoman]

The same. It was not possible to launch any of 10 models. I also tried v1.3_noavx2. The result is one:
OSError: [WinError -1073741795] Windows Error 0xc000001d
[5388] Failed to execute script 'koboldcpp' due to unhandled exception!

if necessary, I can attach a screenshot for all models and versions

16 Gb RAM and Xeon E5-1620

[greenvision4]

I agree I have the same situation processor FX-8300 but
my friend got it working the first time he has a Ryzen 5 3600x processor

[Kotofay]

Welcome to KoboldCpp - Version 1.4
Attempting to use OpenBLAS library for faster prompt ingestion. A compatible libopenblas.dll will be required.
Initializing dynamic library: koboldcpp_blas.dll
Loading model: F:\bin\ChatGPT\koala-13B-4bit-128g.GGML.bin
[Parts: 1, Threads: 7]

---

Identified as LLAMA model: (ver 3)
Attempting to Load...

System Info: AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |
llama.cpp: loading model from F:\bin\ChatGPT\koala-13B-4bit-128g.GGML.bin
llama_model_load_internal: format = ggjt v1 (latest)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 2048
llama_model_load_internal: n_embd = 5120
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 40
llama_model_load_internal: n_layer = 40
llama_model_load_internal: n_rot = 128
llama_model_load_internal: f16 = 4
llama_model_load_internal: n_ff = 13824
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 13B
llama_model_load_internal: ggml ctx size = 73.73 KB
llama_model_load_internal: mem required = 11749.65 MB (+ 1608.00 MB per state)
Traceback (most recent call last):
File "koboldcpp.py", line 401, in
File "koboldcpp.py", line 352, in main
File "koboldcpp.py", line 66, in load_model
OSError: [WinError -1073741795] Windows Error 0xc000001d
[2652] Failed to execute script 'koboldcpp' due to unhandled exception!

Tried running this model to llama, works fine.

[LostRuins]

There is a known regression for certain llama models on the parent repo: ggml-org#903

Can anyone compare with the most recent CI/CD .exe build of llama.cpp https://github.com/ggerganov/llama.cpp/releases/tag/master-8b67998 and see if you encounter the same error? I want to make sure it's not an upstream bug first

[LostRuins]

also since it seems quite a few people are encountering this issue, might be helpful to list which models you are trying to load, and what format (as displayed in console) they are shown to be in

[ManOfFlash]

koboldcpp.exe --noblas D:\Alpaca-33B-ggml-4bit-LoRA-merged_2023-03-31\alpaca-33b-ggml-q4_0-lora-merged\ggml-model-q4_0.bin
Welcome to KoboldCpp - Version 1.4
Initializing dynamic library: koboldcpp.dll
Loading model: D:\Alpaca-33B-ggml-4bit-LoRA-merged_2023-03-31\alpaca-33b-ggml-q4_0-lora-merged\ggml-model-q4_0.bin
[Parts: 1, Threads: 3]

---

Identified as LLAMA model: (ver 3)
Attempting to Load...

System Info: AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
llama.cpp: loading model from D:\Alpaca-33B-ggml-4bit-LoRA-merged_2023-03-31\alpaca-33b-ggml-q4_0-lora-merged\ggml-model-q4_0.bin
llama_model_load_internal: format = ggjt v1 (latest)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 2048
llama_model_load_internal: n_embd = 6656
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 52
llama_model_load_internal: n_layer = 60
llama_model_load_internal: n_rot = 128
llama_model_load_internal: f16 = 2
llama_model_load_internal: n_ff = 17920
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 30B
llama_model_load_internal: ggml ctx size = 110.30 KB
llama_model_load_internal: mem required = 21695.46 MB (+ 3124.00 MB per state)
Traceback (most recent call last):
File "koboldcpp.py", line 401, in
File "koboldcpp.py", line 352, in main
File "koboldcpp.py", line 66, in load_model
OSError: [WinError -1073741795] Windows Error 0xc000001d
[8944] Failed to execute script 'koboldcpp' due to unhandled exception!

AMD FX8350, Win10.
sha256 hash:
2267ed1dc0bf0d6d300ba292c25083c7fa5395f3726c7c68a49b2be19a64b349 alpaca-33b-ggml-q4_0-lora-merged/ggml-model-q4_0.bin

[LostRuins]

1. Does it happen in the previous version?
2. Does it happen in the standard llama.cpp exe?

[prostoman]

Screenshots

[ManOfFlash]

From https://github.com/ggerganov/llama.cpp/releases/tag/master-8b67998 variant llama-master-8b67998-bin-win-avx-x64 successfully run.
P.S. Text _System Info: AVX = 1 | AVX2 = 1 _ is confusing. This info about compilation parameters, not your system.

[rabidcopy]

ggml-org#917 should fix this when it gets merged here. ggml-org@e7f6997

[Des-75]

Identified as LLAMA model: (ver 3)
Attempting to Load...

System Info: AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |
llama.cpp: loading model from C:\LLaMA-ggml-4bit_2023-03-31\llama-13b-ggml-q4_0\ggml-model-q4_0.bin
llama_model_load_internal: format     = ggjt v1 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 2048
llama_model_load_internal: n_embd     = 5120
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 40
llama_model_load_internal: n_layer    = 40
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: f16        = 2
llama_model_load_internal: n_ff       = 13824
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 13B
llama_model_load_internal: ggml ctx size =  73.73 KB
llama_model_load_internal: mem required  = 9807.47 MB (+ 1608.00 MB per state)
Traceback (most recent call last):
  File "koboldcpp.py", line 401, in <module>
  File "koboldcpp.py", line 352, in main
  File "koboldcpp.py", line 66, in load_model
OSError: [WinError -1073741795] Windows Error 0xc000001d
[9816] Failed to execute script 'koboldcpp' due to unhandled exception!

koboldcpp-1.4
Intel(R) Xeon(R) CPU E5-2670 v2 / 32 Gb / Win10
SHA256: 9cd4d6c1f5f42d5abf529c51bde3303991fba912ab8ed452adfd7c97a4be77d7 *ggml-model-q4_0.bin