Skip to content

whisper_backend_init_gpu: failed to initialize Metal backend #3009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
7k50 opened this issue Apr 5, 2025 · 3 comments
Open

whisper_backend_init_gpu: failed to initialize Metal backend #3009

7k50 opened this issue Apr 5, 2025 · 3 comments

Comments

@7k50
Copy link

7k50 commented Apr 5, 2025

On MacOS, I'm suddenly having pecuilar issues when running, e.g.:

./build/bin/whisper-cli -f samples/jfk.wav

or

./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wav

The main issue seems to be:

ggml_backend_metal_device_init: error: failed to allocate context
whisper_backend_init_gpu: failed to initialize Metal backend
whisper_backend_init: using BLAS backend

Giving lots of errors like:

program_source:396:38: error: unknown type name 'block_iq2_xxs'
void dequantize_iq2_xxs(device const block_iq2_xxs * xb, short il, thread type4x4 & reg) {
                                     ^
program_source:408:52: error: use of undeclared identifier 'iq2xxs_grid'
    constant uint8_t * grid = (constant uint8_t *)(iq2xxs_grid + aux8[2*il+0]);

I'm having this problem both in whisper.cpp v.1.7.4 & v.1.7.5.
I've tried reinstalling both from scratch (using Python 3.10 & 3.11 respectively).
I've also tried on different Macs, running MacOS 15.4 & 15.3.2.
Xcode version is however the same regardless of MacOS version (Xcode 16.3 released a few days ago), since it's updated via App Store.

xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.4.sdk

I don't have an idea about what the problem is, or how to resolve.
@7k50
Copy link
Author

7k50 commented Apr 5, 2025 •
edited
Loading

Full example output (divided into 2 comment posts) from

./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wav:

whisper_init_from_file_with_params_no_state: loading model from 'models/ggml-base.en.bin'
whisper_init_with_params_no_state: use gpu    = 1
whisper_init_with_params_no_state: flash attn = 0
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw        = 0
whisper_init_with_params_no_state: devices    = 3
whisper_init_with_params_no_state: backends   = 3
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
whisper_default_buffer_type: using device Metal (Apple M1 Max)
whisper_model_load:    Metal total size =   147.37 MB
whisper_model_load: model size    =  147.37 MB
whisper_backend_init_gpu: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M1 Max
ggml_metal_init: picking default device: Apple M1 Max
ggml_metal_init: using embedded metal library
ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:61:35: error: unknown type name 'block_q4_0'
void dequantize_q4_0(device const block_q4_0 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:80:38: error: unknown type name 'block_q4_0'
void dequantize_q4_0_t4(device const block_q4_0 * xb, short il, thread type4 & reg) {
                                     ^
program_source:95:35: error: unknown type name 'block_q4_1'
void dequantize_q4_1(device const block_q4_1 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:114:38: error: unknown type name 'block_q4_1'
void dequantize_q4_1_t4(device const block_q4_1 * xb, short il, thread type4 & reg) {
                                     ^
program_source:129:35: error: unknown type name 'block_q5_0'
void dequantize_q5_0(device const block_q5_0 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:161:38: error: unknown type name 'block_q5_0'
void dequantize_q5_0_t4(device const block_q5_0 * xb, short il, thread type4 & reg) {
                                     ^
program_source:191:35: error: unknown type name 'block_q5_1'
void dequantize_q5_1(device const block_q5_1 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:223:38: error: unknown type name 'block_q5_1'
void dequantize_q5_1_t4(device const block_q5_1 * xb, short il, thread type4 & reg) {
                                     ^
program_source:253:35: error: unknown type name 'block_q8_0'
void dequantize_q8_0(device const block_q8_0 *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:267:38: error: unknown type name 'block_q8_0'
void dequantize_q8_0_t4(device const block_q8_0 *xb, short il, thread type4 & reg) {
                                     ^
program_source:277:35: error: unknown type name 'block_q2_K'
void dequantize_q2_K(device const block_q2_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:296:35: error: unknown type name 'block_q3_K'
void dequantize_q3_K(device const block_q3_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:330:35: error: unknown type name 'block_q4_K'
void dequantize_q4_K(device const block_q4_K * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:349:35: error: unknown type name 'block_q5_K'
void dequantize_q5_K(device const block_q5_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:372:35: error: unknown type name 'block_q6_K'
void dequantize_q6_K(device const block_q6_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:396:38: error: unknown type name 'block_iq2_xxs'
void dequantize_iq2_xxs(device const block_iq2_xxs * xb, short il, thread type4x4 & reg) {
                                     ^
program_source:408:52: error: use of undeclared identifier 'iq2xxs_grid'
    constant uint8_t * grid = (constant uint8_t *)(iq2xxs_grid + aux8[2*il+0]);
                                                   ^
program_source:409:21: error: use of undeclared identifier 'ksigns_iq2xs'
    uint8_t signs = ksigns_iq2xs[(aux32_s >> 14*il) & 127];
                    ^
program_source:411:49: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                ^
program_source:413:33: error: use of undeclared identifier 'iq2xxs_grid'
    grid = (constant uint8_t *)(iq2xxs_grid + aux8[2*il+1]);
                                ^
program_source:414:13: error: use of undeclared identifier 'ksigns_iq2xs'
    signs = ksigns_iq2xs[(aux32_s >> (14*il+7)) & 127];
            ^
program_source:416:51: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2+i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                  ^
program_source:421:37: error: unknown type name 'block_iq2_xs'
void dequantize_iq2_xs(device const block_iq2_xs * xb, short il, thread type4x4 & reg) {
                                    ^
program_source:429:52: error: use of undeclared identifier 'iq2xs_grid'
    constant uint8_t * grid = (constant uint8_t *)(iq2xs_grid + (q2[2*il+0] & 511));
                                                   ^
program_source:430:21: error: use of undeclared identifier 'ksigns_iq2xs'
    uint8_t signs = ksigns_iq2xs[q2[2*il+0] >> 9];
                    ^
program_source:432:49: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                ^
program_source:434:33: error: use of undeclared identifier 'iq2xs_grid'
    grid = (constant uint8_t *)(iq2xs_grid + (q2[2*il+1] & 511));
                                ^
program_source:435:13: error: use of undeclared identifier 'ksigns_iq2xs'
    signs = ksigns_iq2xs[q2[2*il+1] >> 9];
            ^
program_source:437:51: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2+i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                  ^
program_source:442:38: error: unknown type name 'block_iq3_xxs'
void dequantize_iq3_xxs(device const block_iq3_xxs * xb, short il, thread type4x4 & reg) {
                                     ^
program_source:449:70: error: use of undeclared identifier 'QK_K'
    device const uint16_t * gas = (device const uint16_t *)(xb->qs + QK_K/4) + 2*ib32;
                                                                     ^
program_source:452:53: error: use of undeclared identifier 'iq3xxs_grid'
    constant uint8_t * grid1 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+0]);
                                                    ^
program_source:453:53: error: use of undeclared identifier 'iq3xxs_grid'
    constant uint8_t * grid2 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+1]);
                                                    ^
program_source:454:21: error: use of undeclared identifier 'ksigns_iq2xs'
    uint8_t signs = ksigns_iq2xs[(aux32 >> 14*il) & 127];
                    ^
program_source:456:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[0][i] = dl * grid1[i] * (signs & kmask_iq2xs[i+0] ? -1.f : 1.f);
                                             ^
program_source:457:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[1][i] = dl * grid2[i] * (signs & kmask_iq2xs[i+4] ? -1.f : 1.f);
                                             ^
program_source:459:34: error: use of undeclared identifier 'iq3xxs_grid'
    grid1 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+2]);
                                 ^
program_source:460:34: error: use of undeclared identifier 'iq3xxs_grid'
    grid2 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+3]);
                                 ^
program_source:461:13: error: use of undeclared identifier 'ksigns_iq2xs'
    signs = ksigns_iq2xs[(aux32 >> (14*il+7)) & 127];
            ^
program_source:463:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2][i] = dl * grid1[i] * (signs & kmask_iq2xs[i+0] ? -1.f : 1.f);
                                             ^
program_source:464:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[3][i] = dl * grid2[i] * (signs & kmask_iq2xs[i+4] ? -1.f : 1.f);
                                             ^
program_source:469:36: error: unknown type name 'block_iq3_s'
void dequantize_iq3_s(device const block_iq3_s * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:479:53: error: use of undeclared identifier 'iq3s_grid'
    constant uint8_t * grid1 = (constant uint8_t *)(iq3s_grid + (qs[4*il+0] | ((qh << 8) & 256)));
                                                    ^
program_source:480:53: error: use of undeclared identifier 'iq3s_grid'
    constant uint8_t * grid2 = (constant uint8_t *)(iq3s_grid + (qs[4*il+1] | ((qh << 7) & 256)));
                                                    ^
program_source:482:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[0][i] = dl * grid1[i] * select(1, -1, signs[0] & kmask_iq2xs[i+0]);
                                                             ^
program_source:483:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[1][i] = dl * grid2[i] * select(1, -1, signs[0] & kmask_iq2xs[i+4]);
                                                             ^
program_source:485:34: error: use of undeclared identifier 'iq3s_grid'
    grid1 = (constant uint8_t *)(iq3s_grid + (qs[4*il+2] | ((qh << 6) & 256)));
                                 ^
program_source:486:34: error: use of undeclared identifier 'iq3s_grid'
    grid2 = (constant uint8_t *)(iq3s_grid + (qs[4*il+3] | ((qh << 5) & 256)));
                                 ^
program_source:488:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2][i] = dl * grid1[i] * select(1, -1, signs[1] & kmask_iq2xs[i+0]);
                                                             ^
program_source:489:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[3][i] = dl * grid2[i] * select(1, -1, signs[1] & kmask_iq2xs[i+4]);
                                                             ^
program_source:494:36: error: unknown type name 'block_iq2_s'
void dequantize_iq2_s(device const block_iq2_s * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:501:41: error: use of undeclared identifier 'QK_K'
    device const uint8_t * signs = qs + QK_K/8;
                                        ^
program_source:504:53: error: use of undeclared identifier 'iq2s_grid'
    constant uint8_t * grid1 = (constant uint8_t *)(iq2s_grid + (qs[0] | ((qh << 8) & 0x300)));
                                                    ^
program_source:505:53: error: use of undeclared identifier 'iq2s_grid'
    constant uint8_t * grid2 = (constant uint8_t *)(iq2s_grid + (qs[1] | ((qh << 6) & 0x300)));
                                                    ^
program_source:507:68: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4+0][i%4] = dl * grid1[i] * select(1, -1, signs[0] & kmask_iq2xs[i]);
                                                                   ^
program_source:508:68: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4+2][i%4] = dl * grid2[i] * select(1, -1, signs[1] & kmask_iq2xs[i]);
                                                                   ^
program_source:513:36: error: unknown type name 'block_iq1_s'
void dequantize_iq1_s(device const block_iq1_s * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:521:53: error: use of undeclared identifier 'IQ1S_DELTA'
    const float ml = dl * (qh[ib32] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA);
                                                    ^
program_source:521:71: error: use of undeclared identifier 'IQ1S_DELTA'
    const float ml = dl * (qh[ib32] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA);
                                                                      ^
program_source:523:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((h << 8) & 0x700)));
                                                    ^
program_source:524:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((h << 5) & 0x700)));
                                                    ^
program_source:534:36: error: unknown type name 'block_iq1_m'
void dequantize_iq1_m(device const block_iq1_m * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:540:5: error: unknown type name 'iq1m_scale_t'
    iq1m_scale_t scale;
    ^
program_source:548:49: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml1 = dl * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                ^
program_source:548:67: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml1 = dl * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                  ^
program_source:549:49: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml2 = dl * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                ^
program_source:549:67: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml2 = dl * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                  ^
program_source:550:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((qh[0] << 8) & 0x700)));
                                                    ^
program_source:551:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((qh[0] << 4) & 0x700)));
                                                    ^
program_source:561:37: error: unknown type name 'block_iq4_nl'
void dequantize_iq4_nl(device const block_iq4_nl * xb, short il, thread type4x4 & reg) {
                                    ^
program_source:576:40: error: unknown type name 'block_iq4_nl'
void dequantize_iq4_nl_t4(device const block_iq4_nl * xb, short il, thread type4 & reg) {
                                       ^
program_source:589:37: error: unknown type name 'block_iq4_xs'
void dequantize_iq4_xs(device const block_iq4_xs * xb, short il, thread type4x4 & reg) {
                                    ^
program_source:617:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:643:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:669:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:695:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:722:18: error: unknown type name 'ggml_metal_kargs_repeat'
        constant ggml_metal_kargs_repeat & args,
                 ^
program_source:745:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_repeat<float>) kernel_repeat_t;
                 ^~~~~~~~~~~~~~~~~~~~
program_source:721:13: note: possible target for call
kernel void kernel_repeat(
            ^
program_source:747:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_f32")]] kernel kernel_repeat_t kernel_repeat<float>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:748:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_f16")]] kernel kernel_repeat_t kernel_repeat<half>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:749:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_i32")]] kernel kernel_repeat_t kernel_repeat<int>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:750:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_i16")]] kernel kernel_repeat_t kernel_repeat<short>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:755:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:765:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:775:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:785:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:1425:18: error: unknown type name 'ggml_metal_kargs_norm'
        constant ggml_metal_kargs_norm & args,
                 ^
program_source:1492:18: error: unknown type name 'ggml_metal_kargs_rms_norm'
        constant ggml_metal_kargs_rms_norm & args,
                 ^
program_source:1627:43: error: unknown type name 'block_q4_0'
inline float block_q_n_dot_y(device const block_q4_0 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1648:43: error: unknown type name 'block_q4_1'
inline float block_q_n_dot_y(device const block_q4_1 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1670:43: error: unknown type name 'block_q5_0'
inline float block_q_n_dot_y(device const block_q5_0 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1681:71: error: use of undeclared identifier 'QK5_0'
        acc[2] += yl[i + 8] * ((qs[i / 2] & 0x00F0) | ((qh >> (i+0+il+QK5_0/2) << 8 ) & 0x00100));
                                                                      ^
program_source:1682:71: error: use of undeclared identifier 'QK5_0'
        acc[3] += yl[i + 9] * ((qs[i / 2] & 0xF000) | ((qh >> (i+1+il+QK5_0/2) << 16) & 0x10000));
                                                                      ^
program_source:1692:43: error: unknown type name 'block_q5_1'
inline float block_q_n_dot_y(device const block_q5_1 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1704:71: error: use of undeclared identifier 'QK5_0'
        acc[2] += yl[i + 8] * ((qs[i / 2] & 0x00F0) | ((qh >> (i+0+il+QK5_0/2) << 8 ) & 0x00100));
                                                                      ^
program_source:1705:71: error: use of undeclared identifier 'QK5_0'
        acc[3] += yl[i + 9] * ((qs[i / 2] & 0xF000) | ((qh >> (i+1+il+QK5_0/2) << 16) & 0x10000));
                                                                      ^
program_source:1728:30: error: use of undeclared identifier 'QK4_0'
    const int nb = args.ne00/QK4_0;
                             ^
program_source:1759:38: error: use of undeclared identifier 'QK4_0'
    device const float * yb = y + ix*QK4_0 + il;
                                     ^
program_source:1781:15: error: use of undeclared identifier 'QK4_0'
        yb += QK4_0 * 16;
              ^
program_source:1796:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1803:26: error: use of undeclared identifier 'block_q4_0'
    mul_vec_q_n_f32_impl<block_q4_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                         ^
program_source:1807:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1814:27: error: use of undeclared identifier 'block_q4_1'
     mul_vec_q_n_f32_impl<block_q4_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                          ^
program_source:1818:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1825:26: error: use of undeclared identifier 'block_q5_0'
    mul_vec_q_n_f32_impl<block_q5_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                         ^
program_source:1829:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1836:26: error: use of undeclared identifier 'block_q5_1'
    mul_vec_q_n_f32_impl<block_q5_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                         ^
program_source:1855:30: error: use of undeclared identifier 'QK8_0'
    const int nb = args.ne00/QK8_0;
                             ^
program_source:1872:18: error: unknown type name 'block_q8_0'
    device const block_q8_0 * ax[nr];
                 ^
program_source:1876:33: error: unknown type name 'block_q8_0'
        ax[row] = (device const block_q8_0 *) ((device char *) src0 + offset0);
                                ^
program_source:1885:38: error: use of undeclared identifier 'QK8_0'
    device const float * yb = y + ix*QK8_0 + il*NB_Q8_0;
                                     ^
program_source:1918:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1925:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q8_0_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:1932:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2033:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2138:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2155:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2170:53: error: use of undeclared identifier 'block_q8_0'
typedef decltype(kernel_mul_mv_ext_q4_f32_disp  <2, block_q8_0, 32,  dequantize_q8_0_t4>) mul_mv_ext_q4_f32_t;
                                                    ^
program_source:2171:53: error: use of undeclared identifier 'block_q4_K'
typedef decltype(kernel_mul_mv_ext_q4x4_f32_disp<2, block_q4_K, 256, dequantize_q4_K>)    mul_mv_ext_q4x4_f32_t;
                                                    ^
program_source:2173:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_2")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2174:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_3")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2175:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_4")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2176:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_5")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2178:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2179:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2180:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2181:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2183:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2184:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2185:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2186:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2188:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2189:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2190:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2191:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2193:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2194:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2195:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2196:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2198:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2199:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2200:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2201:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2203:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_2")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2204:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_3")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2205:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_4")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2206:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_5")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2208:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_2")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<2, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2209:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_3")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<3, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2210:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_4")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<4, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2211:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_5")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<5, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2213:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_2")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<2, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2214:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_3")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<3, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2215:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_4")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<4, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2216:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_5")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<5, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2218:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_2")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<2, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2219:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_3")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<3, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2220:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_4")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<4, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2221:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_5")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<5, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2296:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:2302:51: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_impl<T0, T04, T1, T14, constant ggml_metal_kargs_mul_mv &>(
                                                  ^
program_source:2311:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv<half, half4, half, half4>) mul_mv_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:2295:13: note: possible target for call
kernel void kernel_mul_mv(
            ^
program_source:2313:67: error: explicit instantiation of 'kernel_mul_mv' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f32_f32")]]   kernel mul_mv_t kernel_mul_mv<float,  float4,  float,  float4>;
                                                                  ^
program_source:2295:13: note: explicit instantiation refers here
kernel void kernel_mul_mv(
            ^
program_source:2314:67: error: explicit instantiation of 'kernel_mul_mv' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f32")]]   kernel mul_mv_t kernel_mul_mv<half,   half4,   float,  float4>;
                                                                  ^
program_source:2295:13: note: explicit instantiation refers here
kernel void kernel_mul_mv(
            ^
program_source:2315:67: error: explicit instantiation of 'kernel_mul_mv' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f16")]]   kernel mul_mv_t kernel_mul_mv<half,   half4,   half,   half4>;
                                                                  ^
program_source:2295:13: note: explicit instantiation refers here
kernel void kernel_mul_mv(
            ^
program_source:2323:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:2371:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv_1row<half, half4>) mul_mv_1row_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:2322:13: note: possible target for call
kernel void kernel_mul_mv_1row(
            ^
program_source:2373:76: error: explicit instantiation of 'kernel_mul_mv_1row' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f32_1row")]]  kernel mul_mv_1row_t kernel_mul_mv_1row<half,   half4>;
                                                                           ^
program_source:2322:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_1row(
            ^
program_source:2381:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:2418:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv_l4<half, half4>) mul_mv_l4_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:2380:13: note: possible target for call
kernel void kernel_mul_mv_l4(
            ^
program_source:2420:72: error: explicit instantiation of 'kernel_mul_mv_l4' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f32_l4")]]  kernel mul_mv_l4_t kernel_mul_mv_l4<half, half4>;
                                                                       ^
program_source:2380:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_l4(
            ^
program_source:2465:18: error: unknown type name 'ggml_metal_kargs_rope'
        constant ggml_metal_kargs_rope & args,
                 ^
program_source:2518:18: error: unknown type name 'ggml_metal_kargs_rope'
        constant ggml_metal_kargs_rope & args,
                 ^
program_source:2569:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_rope_norm<float>) kernel_rope_norm_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~
program_source:2464:13: note: possible target for call
kernel void kernel_rope_norm(
            ^
program_source:2570:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_rope_neox<float>) kernel_rope_neox_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~
program_source:2517:13: note: possible target for call
kernel void kernel_rope_neox(
            ^
program_source:2572:74: error: explicit instantiation of 'kernel_rope_norm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_norm_f32")]] kernel kernel_rope_norm_t kernel_rope_norm<float>;
                                                                         ^
program_source:2464:13: note: explicit instantiation refers here
kernel void kernel_rope_norm(
            ^
program_source:2573:74: error: explicit instantiation of 'kernel_rope_norm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_norm_f16")]] kernel kernel_rope_norm_t kernel_rope_norm<half>;
                                                                         ^
program_source:2464:13: note: explicit instantiation refers here
kernel void kernel_rope_norm(
            ^
program_source:2575:74: error: explicit instantiation of 'kernel_rope_neox' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_neox_f32")]] kernel kernel_rope_neox_t kernel_rope_neox<float>;
                                                                         ^
program_source:2517:13: note: explicit instantiation refers here
kernel void kernel_rope_neox(
            ^
program_source:2576:74: error: explicit instantiation of 'kernel_rope_neox' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_neox_f16")]] kernel kernel_rope_neox_t kernel_rope_neox<half>;
                                                                         ^
program_source:2517:13: note: explicit instantiation refers here
kernel void kernel_rope_neox(
            ^
program_source:3093:18: error: unknown type name 'ggml_metal_kargs_flash_attn_ext'
        constant ggml_metal_kargs_flash_attn_ext & args,
                 ^
program_source:3513:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_flash_attn_ext<FA_TYPES, half4x4, 1, dequantize_f16, half4x4, 1, dequantize_f16, 64>) flash_attn_ext_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3092:13: note: possible target for call
kernel void kernel_flash_attn_ext(
            ^
program_source:3515:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h64" )]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  64>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3516:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h80" )]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  80>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3517:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h96" )]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  96>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3518:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h112")]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  112>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3519:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h128")]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  128>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3520:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h256")]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  256>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3531:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 64>;
                                                                                                                  ^
program_source:3532:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 80>;
                                                                                                                  ^
program_source:3533:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 96>;
                                                                                                                  ^
program_source:3534:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 112>;
                                                                                                                  ^
program_source:3535:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 128>;
                                                                                                                  ^
program_source:3536:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 256>;
                                                                                                                  ^
program_source:3538:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 64>;
                                                                                                                  ^
program_source:3539:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 80>;
                                                                                                                  ^
program_source:3540:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 96>;
                                                                                                                  ^
program_source:3541:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 112>;
                                                                                                                  ^
program_source:3542:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 128>;
                                                                                                                  ^
program_source:3543:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 256>;
                                                                                                                  ^
program_source:3545:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 64>;
                                                                                                                  ^
program_source:3546:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 80>;
                                                                                                                  ^
program_source:3547:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 96>;
                                                                                                                  ^
program_source:3548:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 112>;
                                                                                                                  ^
program_source:3549:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 128>;
                                                                                                                  ^
program_source:3550:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 256>;
                                                                                                                  ^
program_source:3552:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 64>;
                                                                                                                  ^
program_source:3553:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 80>;
                                                                                                                  ^
program_source:3554:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 96>;
                                                                                                                  ^
program_source:3555:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 112>;
                                                                                                                  ^
program_source:3556:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 128>;
                                                                                                                  ^
program_source:3557:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 256>;
                                                                                                                  ^
program_source:3559:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 64>;
                                                                                                                  ^
program_source:3560:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 80>;
                                                                                                                  ^
program_source:3561:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 96>;
                                                                                                                  ^
program_source:3562:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 112>;
                                                                                                                  ^
program_source:3563:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 128>;
                                                                                                                  ^
program_source:3564:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 256>;
                                                                                                                  ^
program_source:3588:18: error: unknown type name 'ggml_metal_kargs_flash_attn_ext'
        constant ggml_metal_kargs_flash_attn_ext & args,
                 ^
program_source:3904:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_flash_attn_ext_vec<FA_TYPES, half4x4, 1, dequantize_f16, half4x4, 1, dequantize_f16, 128>) flash_attn_ext_vec_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3587:13: note: possible target for call
kernel void kernel_flash_attn_ext_vec(
            ^
program_source:3906:91: error: explicit instantiation of 'kernel_flash_attn_ext_vec' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_vec_f16_h128")]]  kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,     1, dequantize_f16,  128>;
                                                                                          ^
program_source:3587:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext_vec(
            ^
program_source:3910:127: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_vec_q4_0_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0,  2, dequantize_q4_0, 128>;
                                                                                                                              ^
program_source:3911:127: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_vec_q4_1_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1,  2, dequantize_q4_1, 128>;
                                                                                                                              ^
program_source:3912:127: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_vec_q5_0_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0,  2, dequantize_q5_0, 128>;
                                                                                                                              ^
program_source:3913:127: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_vec_q5_1_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1,  2, dequantize_q5_1, 128>;
                                                                                                                              ^
program_source:3914:127: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_vec_q8_0_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0,  2, dequantize_q8_0, 128>;
                                                                                                                              ^
program_source:3916:91: error: explicit instantiation of 'kernel_flash_attn_ext_vec' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_vec_f16_h256")]]  kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,     1, dequantize_f16,  256>;
                                                                                          ^
program_source:3587:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext_vec(
            ^
program_source:3920:127: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_vec_q4_0_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0,  2, dequantize_q4_0, 256>;
                                                                                                                              ^
program_source:3921:127: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_vec_q4_1_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1,  2, dequantize_q4_1, 256>;
                                                                                                                              ^
program_source:3922:127: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_vec_q5_0_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0,  2, dequantize_q5_0, 256>;
                                                                                                                              ^
program_source:3923:127: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_vec_q5_1_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1,  2, dequantize_q5_1, 256>;
                                                                                                                              ^
program_source:3924:127: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_vec_q8_0_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0,  2, dequantize_q8_0, 256>;
                                                                                                                              ^
program_source:3930:14: error: unknown type name 'ggml_metal_kargs_set'
    constant ggml_metal_kargs_set & args,
             ^
program_source:3955:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_set<float>) kernel_set_t;
                 ^~~~~~~~~~~~~~~~~
program_source:3929:13: note: possible target for call
kernel void kernel_set(
            ^
program_source:3957:62: error: explicit instantiation of 'kernel_set' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_set_f32")]] kernel kernel_set_t kernel_set<float>;
                                                             ^
program_source:3929:13: note: explicit instantiation refers here
kernel void kernel_set(
            ^
program_source:3958:62: error: explicit instantiation of 'kernel_set' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_set_i32")]] kernel kernel_set_t kernel_set<int32_t>;
                                                             ^
program_source:3929:13: note: explicit instantiation refers here
kernel void kernel_set(
            ^
program_source:3962:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:3987:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_cpy<float, float>) kernel_cpy_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~
program_source:3961:13: note: possible target for call
kernel void kernel_cpy(
            ^
program_source:3989:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f32_f32")]]   kernel kernel_cpy_t kernel_cpy<float,  float>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:3990:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f32_f16")]]   kernel kernel_cpy_t kernel_cpy<float,  half>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:3994:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f16_f32")]]   kernel kernel_cpy_t kernel_cpy<half,   float>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:3995:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f16_f16")]]   kernel kernel_cpy_t kernel_cpy<half,   half>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:4002:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4017:97: error: use of undeclared identifier 'QK8_0'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK8_0;
                                                                                                ^
program_source:4019:12: error: unknown type name 'block_q8_0'
    device block_q8_0 * dst_data = (device block_q8_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4019:44: error: unknown type name 'block_q8_0'
    device block_q8_0 * dst_data = (device block_q8_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4021:32: error: use of undeclared identifier 'QK8_0'
    for (int64_t i00 = tpitg.x*QK8_0; i00 < args.ne00; i00 += ntg.x*QK8_0) {
                               ^
program_source:4021:69: error: use of undeclared identifier 'QK8_0'
    for (int64_t i00 = tpitg.x*QK8_0; i00 < args.ne00; i00 += ntg.x*QK8_0) {
                                                                    ^
program_source:4026:29: error: use of undeclared identifier 'QK8_0'
        for (int j = 0; j < QK8_0; j++) {
                            ^
program_source:4034:22: error: use of undeclared identifier 'QK8_0'
        dst_data[i00/QK8_0].d = d;
                     ^
program_source:4036:29: error: use of undeclared identifier 'QK8_0'
        for (int j = 0; j < QK8_0; ++j) {
                            ^
program_source:4039:26: error: use of undeclared identifier 'QK8_0'
            dst_data[i00/QK8_0].qs[j] = round(x0);
                         ^
program_source:4045:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4060:97: error: use of undeclared identifier 'QK4_0'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK4_0;
                                                                                                ^
program_source:4062:12: error: unknown type name 'block_q4_0'
    device block_q4_0 * dst_data = (device block_q4_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4062:44: error: unknown type name 'block_q4_0'
    device block_q4_0 * dst_data = (device block_q4_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4064:32: error: use of undeclared identifier 'QK4_0'
    for (int64_t i00 = tpitg.x*QK4_0; i00 < args.ne00; i00 += ntg.x*QK4_0) {
                               ^
program_source:4064:69: error: use of undeclared identifier 'QK4_0'
    for (int64_t i00 = tpitg.x*QK4_0; i00 < args.ne00; i00 += ntg.x*QK4_0) {
                                                                    ^
program_source:4070:29: error: use of undeclared identifier 'QK4_0'
        for (int j = 0; j < QK4_0; j++) {
                            ^
program_source:4081:22: error: use of undeclared identifier 'QK4_0'
        dst_data[i00/QK4_0].d = d;
                     ^
program_source:4083:29: error: use of undeclared identifier 'QK4_0'
        for (int j = 0; j < QK4_0/2; ++j) {
                            ^
program_source:4085:34: error: use of undeclared identifier 'QK4_0'
            const float x1 = src[QK4_0/2 + j]*id;
                                 ^
program_source:4090:26: error: use of undeclared identifier 'QK4_0'
            dst_data[i00/QK4_0].qs[j]  = xi0;
                         ^
program_source:4091:26: error: use of undeclared identifier 'QK4_0'
            dst_data[i00/QK4_0].qs[j] |= xi1 << 4;
                         ^
program_source:4097:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4112:97: error: use of undeclared identifier 'QK4_1'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK4_1;
                                                                                                ^
program_source:4114:12: error: unknown type name 'block_q4_1'
    device block_q4_1 * dst_data = (device block_q4_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4114:44: error: unknown type name 'block_q4_1'
    device block_q4_1 * dst_data = (device block_q4_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4116:32: error: use of undeclared identifier 'QK4_1'
    for (int64_t i00 = tpitg.x*QK4_1; i00 < args.ne00; i00 += ntg.x*QK4_1) {
                               ^
program_source:4116:69: error: use of undeclared identifier 'QK4_1'
    for (int64_t i00 = tpitg.x*QK4_1; i00 < args.ne00; i00 += ntg.x*QK4_1) {
                                                                    ^
program_source:4122:29: error: use of undeclared identifier 'QK4_1'
        for (int j = 0; j < QK4_1; j++) {
                            ^
program_source:4131:22: error: use of undeclared identifier 'QK4_1'
        dst_data[i00/QK4_1].d = d;
                     ^
program_source:4132:22: error: use of undeclared identifier 'QK4_1'
        dst_data[i00/QK4_1].m = min;
                     ^
program_source:4134:29: error: use of undeclared identifier 'QK4_1'
        for (int j = 0; j < QK4_1/2; ++j) {
                            ^
program_source:4136:35: error: use of undeclared identifier 'QK4_1'
            const float x1 = (src[QK4_1/2 + j] - min)*id;
                                  ^
program_source:4141:26: error: use of undeclared identifier 'QK4_1'
            dst_data[i00/QK4_1].qs[j]  = xi0;
                         ^
program_source:4142:26: error: use of undeclared identifier 'QK4_1'
            dst_data[i00/QK4_1].qs[j] |= xi1 << 4;
                         ^
program_source:4148:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4163:97: error: use of undeclared identifier 'QK5_0'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK5_0;
                                                                                                ^
program_source:4165:12: error: unknown type name 'block_q5_0'
    device block_q5_0 * dst_data = (device block_q5_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4165:44: error: unknown type name 'block_q5_0'
    device block_q5_0 * dst_data = (device block_q5_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4167:32: error: use of undeclared identifier 'QK5_0'
    for (int64_t i00 = tpitg.x*QK5_0; i00 < args.ne00; i00 += ntg.x*QK5_0) {
                               ^
program_source:4167:69: error: use of undeclared identifier 'QK5_0'
    for (int64_t i00 = tpitg.x*QK5_0; i00 < args.ne00; i00 += ntg.x*QK5_0) {
                                                                    ^
program_source:4173:29: error: use of undeclared identifier 'QK5_0'
        for (int j = 0; j < QK5_0; j++) {
                            ^
program_source:4184:22: error: use of undeclared identifier 'QK5_0'
        dst_data[i00/QK5_0].d = d;
                     ^
program_source:4187:29: error: use of undeclared identifier 'QK5_0'
        for (int j = 0; j < QK5_0/2; ++j) {
                            ^
program_source:4189:34: error: use of undeclared identifier 'QK5_0'
            const float x1 = src[QK5_0/2 + j]*id;
                                 ^
program_source:4194:26: error: use of undeclared identifier 'QK5_0'
            dst_data[i00/QK5_0].qs[j] = (xi0 & 0xf) | ((xi1 & 0xf) << 4);
                         ^
program_source:4196:48: error: use of undeclared identifier 'QK5_0'
            qh |= ((xi1 & 0x10u) >> 4) << (j + QK5_0/2);
                                               ^
program_source:4200:26: error: use of undeclared identifier 'QK5_0'
            dst_data[i00/QK5_0].qh[j] = qh8[j];
                         ^
program_source:4206:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4221:97: error: use of undeclared identifier 'QK5_1'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK5_1;
                                                                                                ^
program_source:4223:12: error: unknown type name 'block_q5_1'
    device block_q5_1 * dst_data = (device block_q5_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4223:44: error: unknown type name 'block_q5_1'
    device block_q5_1 * dst_data = (device block_q5_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4225:32: error: use of undeclared identifier 'QK5_1'
    for (int64_t i00 = tpitg.x*QK5_1; i00 < args.ne00; i00 += ntg.x*QK5_1) {
                               ^
program_source:4225:69: error: use of undeclared identifier 'QK5_1'
    for (int64_t i00 = tpitg.x*QK5_1; i00 < args.ne00; i00 += ntg.x*QK5_1) {
                                                                    ^
program_source:4231:29: error: use of undeclared identifier 'QK5_1'
        for (int j = 1; j < QK5_1; j++) {
                            ^
program_source:4240:22: error: use of undeclared identifier 'QK5_1'
        dst_data[i00/QK5_1].d = d;
                     ^
program_source:4241:22: error: use of undeclared identifier 'QK5_1'
        dst_data[i00/QK5_1].m = min;
                     ^
program_source:4244:29: error: use of undeclared identifier 'QK5_1'
        for (int j = 0; j < QK5_1/2; ++j) {
                            ^
program_source:4246:35: error: use of undeclared identifier 'QK5_1'
            const float x1 = (src[QK5_1/2 + j] - min)*id;
                                  ^
program_source:4251:26: error: use of undeclared identifier 'QK5_1'
            dst_data[i00/QK5_1].qs[j] = (xi0 & 0xf) | ((xi1 & 0xf) << 4);
                         ^
program_source:4253:48: error: use of undeclared identifier 'QK5_1'
            qh |= ((xi1 & 0x10u) >> 4) << (j + QK5_1/2);
                                               ^
program_source:4257:26: error: use of undeclared identifier 'QK5_1'
            dst_data[i00/QK5_1].qh[j] = qh8[j];
                         ^
program_source:4274:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4289:97: error: use of undeclared identifier 'QK4_NL'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK4_NL;
                                                                                                ^
program_source:4291:12: error: unknown type name 'block_iq4_nl'
    device block_iq4_nl * dst_data = (device block_iq4_nl *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4291:46: error: unknown type name 'block_iq4_nl'
    device block_iq4_nl * dst_data = (device block_iq4_nl *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                             ^
program_source:4293:32: error: use of undeclared identifier 'QK4_NL'
    for (int64_t i00 = tpitg.x*QK4_NL; i00 < args.ne00; i00 += ntg.x*QK4_NL) {
                               ^
program_source:4293:70: error: use of undeclared identifier 'QK4_NL'
    for (int64_t i00 = tpitg.x*QK4_NL; i00 < args.ne00; i00 += ntg.x*QK4_NL) {
                                                                     ^
program_source:4299:29: error: use of undeclared identifier 'QK4_0'
        for (int j = 0; j < QK4_0; j++) {
                            ^
program_source:4311:29: error: use of undeclared identifier 'QK4_NL'
        for (int j = 0; j < QK4_NL/2; ++j) {
                            ^
program_source:4313:34: error: use of undeclared identifier 'QK4_NL'
            const float x1 = src[QK4_NL/2 + j]*id;
                                 ^
program_source:4318:26: error: use of undeclared identifier 'QK4_NL'
            dst_data[i00/QK4_NL].qs[j] = xi0 | (xi1 << 4);
                         ^
program_source:4323:34: error: use of undeclared identifier 'QK4_NL'
            const float w1 = src[QK4_NL/2 + j]*src[QK4_NL/2 + j];
                                 ^
program_source:4323:52: error: use of undeclared identifier 'QK4_NL'
            const float w1 = src[QK4_NL/2 + j]*src[QK4_NL/2 + j];
                                                   ^
program_source:4324:47: error: use of undeclared identifier 'QK4_NL'
            sumqx += w0*v0*src[j] + w1*v1*src[QK4_NL/2 + j];
                                              ^
program_source:4329:22: error: use of undeclared identifier 'QK4_NL'
        dst_data[i00/QK4_NL].d = sumq2 > 0 ? sumqx/sumq2 : d;
                     ^
program_source:4334:14: error: unknown type name 'ggml_metal_kargs_concat'
    constant ggml_metal_kargs_concat & args,
             ^
program_source:4375:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4388:18: error: unknown type name 'block_q2_K'
    device const block_q2_K * x = (device const block_q2_K *) (src0 + offset0);
                 ^
program_source:4388:49: error: unknown type name 'block_q2_K'
    device const block_q2_K * x = (device const block_q2_K *) (src0 + offset0);
                                                ^
program_source:4400:40: error: use of undeclared identifier 'QK_K'
    device const float * y4 = y + ix * QK_K + 128 * iq + 8 * ir;
                                       ^
program_source:4443:19: error: use of undeclared identifier 'QK_K'
        y4 += 4 * QK_K;
                  ^
program_source:4458:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4466:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q2_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4480:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4494:18: error: unknown type name 'block_q3_K'
    device const block_q3_K * x = (device const block_q3_K *) (src0 + offset0);
                 ^
program_source:4494:49: error: unknown type name 'block_q3_K'
    device const block_q3_K * x = (device const block_q3_K *) (src0 + offset0);
                                                ^
program_source:4536:39: error: use of undeclared identifier 'QK_K'
    device const float * y1 = yy + ix*QK_K + y_offset;
                                      ^
program_source:4603:19: error: use of undeclared identifier 'QK_K'
        y1 += 4 * QK_K;
                  ^
program_source:4622:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4630:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q3_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4653:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4666:18: error: unknown type name 'block_q4_K'
    device const block_q4_K * x = (device const block_q4_K *) (src0 + offset0);
                 ^
program_source:4666:49: error: unknown type name 'block_q4_K'
    device const block_q4_K * x = (device const block_q4_K *) (src0 + offset0);
                                                ^
program_source:4673:40: error: use of undeclared identifier 'QK_K'
    device const float * y4 = y + ix * QK_K + 64 * iq + 8 * ir;
                                       ^
program_source:4725:19: error: use of undeclared identifier 'QK_K'
        y4 += 4 * QK_K;
                  ^
program_source:4740:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4748:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q4_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4762:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4776:18: error: unknown type name 'block_q5_K'
    device const block_q5_K * x = (device const block_q5_K *) (src0 + offset0);
                 ^
program_source:4776:49: error: unknown type name 'block_q5_K'
    device const block_q5_K * x = (device const block_q5_K *) (src0 + offset0);
                                                ^
program_source:4805:39: error: use of undeclared identifier 'QK_K'
    device const float * y1 = yy + ix*QK_K + y_offset;
                                      ^
program_source:4857:19: error: use of undeclared identifier 'QK_K'
        y1 += 4 * QK_K;
                  ^
program_source:4872:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4880:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q5_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4899:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4913:18: error: unknown type name 'block_q6_K'
    device const block_q6_K * x = (device const block_q6_K *) (src0 + offset0);
                 ^
program_source:4913:49: error: unknown type name 'block_q6_K'
    device const block_q6_K * x = (device const block_q6_K *) (src0 + offset0);
                                                ^
program_source:4936:43: error: use of undeclared identifier 'QK_K'
        device const float * y = yy + i * QK_K + y_offset;
                                          ^
program_source:4962:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4970:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q6_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4986:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4999:18: error: unknown type name 'block_iq2_xxs'
    device const block_iq2_xxs * x = (device const block_iq2_xxs *) (src0 + offset0);
                 ^
program_source:4999:52: error: unknown type name 'block_iq2_xxs'
    device const block_iq2_xxs * x = (device const block_iq2_xxs *) (src0 + offset0);
                                                   ^
program_source:5005:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5012:59: error: use of undeclared identifier 'iq2xxs_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq2xxs_grid[pos + i];
                                                          ^
program_source:5015:56: error: use of undeclared identifier 'ksigns_iq2xs'
        for (int i = 0; i < nval; ++i) ssigns[pos+i] = ksigns_iq2xs[pos+i];
                                                       ^
program_source:5029:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5030:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5032:22: error: unknown type name 'block_iq2_xxs'
        device const block_iq2_xxs * xr = x + ibl;
                     ^
program_source:5048:61: error: use of undeclared identifier 'kmask_iq2xs'
                    sum += yl[8*l + j] * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f);
                                                            ^
program_source:5072:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5080:45: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq2_xxs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                            ^
program_source:5094:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5107:18: error: unknown type name 'block_iq2_xs'
    device const block_iq2_xs * x = (device const block_iq2_xs *) (src0 + offset0);
                 ^
program_source:5107:51: error: unknown type name 'block_iq2_xs'
    device const block_iq2_xs * x = (device const block_iq2_xs *) (src0 + offset0);
                                                  ^
program_source:5113:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5120:59: error: use of undeclared identifier 'iq2xs_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq2xs_grid[pos + i];
                                                          ^
program_source:5123:56: error: use of undeclared identifier 'ksigns_iq2xs'
        for (int i = 0; i < nval; ++i) ssigns[pos+i] = ksigns_iq2xs[pos+i];
                                                       ^
program_source:5137:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5138:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5140:22: error: unknown type name 'block_iq2_xs'
        device const block_iq2_xs * xr = x + ibl;
                     ^
program_source:5158:62: error: use of undeclared identifier 'kmask_iq2xs'
                    sum1 += yl[8*l + j] * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f);
                                                             ^
program_source:5165:62: error: use of undeclared identifier 'kmask_iq2xs'
                    sum2 += yl[8*l + j] * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f);
                                                             ^
program_source:5190:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5199:44: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq2_xs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                           ^
program_source:5213:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5226:18: error: unknown type name 'block_iq3_xxs'
    device const block_iq3_xxs * x = (device const block_iq3_xxs *) (src0 + offset0);
                 ^
program_source:5226:52: error: unknown type name 'block_iq3_xxs'
    device const block_iq3_xxs * x = (device const block_iq3_xxs *) (src0 + offset0);
                                                   ^
program_source:5232:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5239:59: error: use of undeclared identifier 'iq3xxs_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq3xxs_grid[pos + i];
                                                          ^
program_source:5242:56: error: use of undeclared identifier 'ksigns_iq2xs'
        for (int i = 0; i < nval; ++i) ssigns[pos+i] = ksigns_iq2xs[pos+i];
                                                       ^
program_source:5255:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5256:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5258:22: error: unknown type name 'block_iq3_xxs'
        device const block_iq3_xxs * xr = x + ibl;
                     ^
program_source:5260:74: error: use of undeclared identifier 'QK_K'
        device const uint16_t * gas = (device const uint16_t *)(xr->qs + QK_K/4) + 2 * ib;
                                                                         ^
program_source:5274:69: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[0] += yl[8*l + j + 0] * grid1[j] * (signs & kmask_iq2xs[j+0] ? -1.f : 1.f);
                                                                    ^
program_source:5275:69: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[1] += yl[8*l + j + 4] * grid2[j] * (signs & kmask_iq2xs[j+4] ? -1.f : 1.f);
                                                                    ^
program_source:5300:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5309:45: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq3_xxs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                            ^
program_source:5323:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5336:18: error: unknown type name 'block_iq3_s'
    device const block_iq3_s * x = (device const block_iq3_s *) (src0 + offset0);
                 ^
program_source:5336:50: error: unknown type name 'block_iq3_s'
    device const block_iq3_s * x = (device const block_iq3_s *) (src0 + offset0);
                                                 ^
program_source:5342:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5348:59: error: use of undeclared identifier 'iq3s_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq3s_grid[pos + i];
                                                          ^
program_source:5362:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5363:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5365:22: error: unknown type name 'block_iq3_s'
        device const block_iq3_s * xr = x + ibl;
                     ^
program_source:5379:63: error: use of undeclared identifier 'kmask_iq2xs'
                const threadgroup uint32_t * table1 = qh[0] & kmask_iq2xs[2*l+0] ? svalues + 256 : svalues;
                                                              ^
program_source:5380:63: error: use of undeclared identifier 'kmask_iq2xs'
                const threadgroup uint32_t * table2 = qh[0] & kmask_iq2xs[2*l+1] ? svalues + 256 : svalues;
                                                              ^
program_source:5384:85: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[0] += yl[8*l + j + 0] * grid1[j] * select(1, -1, signs[l] & kmask_iq2xs[j+0]);
                                                                                    ^
program_source:5385:85: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[1] += yl[8*l + j + 4] * grid2[j] * select(1, -1, signs[l] & kmask_iq2xs[j+4]);
                                                                                    ^
program_source:5412:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5421:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq3_s_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                          ^
program_source:5435:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5448:18: error: unknown type name 'block_iq2_s'
    device const block_iq2_s * x = (device const block_iq2_s *) (src0 + offset0);
                 ^
program_source:5448:50: error: unknown type name 'block_iq2_s'
    device const block_iq2_s * x = (device const block_iq2_s *) (src0 + offset0);
                                                 ^
program_source:5454:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5474:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5475:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5477:22: error: unknown type name 'block_iq2_s'
        device const block_iq2_s * xr = x + ibl;
                     ^
program_source:5481:45: error: use of undeclared identifier 'QK_K'
        device const uint8_t * signs = qs + QK_K/8;
                                            ^
program_source:5494:65: error: use of undeclared identifier 'iq2s_grid'
                constant uint8_t * grid1 = (constant uint8_t *)(iq2s_grid + (qs[l+0] | ((qh[0] << (8-2*l)) & 0x300)));
                                                                ^
program_source:5495:65: error: use of undeclared identifier 'iq2s_grid'
                constant uint8_t * grid2 = (constant uint8_t *)(iq2s_grid + (qs[l+2] | ((qh[0] << (4-2*l)) & 0x300)));
                                                                ^
program_source:5497:88: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[0] += yl[8*l + j +  0] * grid1[j] * select(1, -1, signs[l+0] & kmask_iq2xs[j]);
                                                                                       ^
program_source:5498:88: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[1] += yl[8*l + j + 16] * grid2[j] * select(1, -1, signs[l+2] & kmask_iq2xs[j]);
                                                                                       ^
program_source:5525:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5534:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq2_s_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                          ^
program_source:5548:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5561:18: error: unknown type name 'block_iq1_s'
    device const block_iq1_s * x = (device const block_iq1_s *) (src0 + offset0);
                 ^
program_source:5561:50: error: unknown type name 'block_iq1_s'
    device const block_iq1_s * x = (device const block_iq1_s *) (src0 + offset0);
                                                 ^
program_source:5567:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5581:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5582:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5584:22: error: unknown type name 'block_iq1_s'
        device const block_iq1_s * xr = x + ibl;
                     ^
program_source:5591:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((qh[0] << 8) & 0x700)));
                                                            ^
program_source:5592:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((qh[0] << 5) & 0x700)));
                                                            ^
program_source:5593:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid3 = (constant uint8_t *)(iq1s_grid_gpu + (qs[2] | ((qh[0] << 2) & 0x700)));
                                                            ^
program_source:5594:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid4 = (constant uint8_t *)(iq1s_grid_gpu + (qs[3] | ((qh[0] >> 1) & 0x700)));
                                                            ^
program_source:5603:78: error: use of undeclared identifier 'IQ1S_DELTA'
            sumf[row] += (float)dh[0] * (sum + sumy * (qh[0] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA)) * (2*((qh[0] >> 12) & 7) + 1);
                                                                             ^
program_source:5603:96: error: use of undeclared identifier 'IQ1S_DELTA'
            sumf[row] += (float)dh[0] * (sum + sumy * (qh[0] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA)) * (2*((qh[0] >> 12) & 7) + 1);
                                                                                               ^
program_source:5634:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5647:18: error: unknown type name 'block_iq1_m'
    device const block_iq1_m * x = (device const block_iq1_m *) (src0 + offset0);
                 ^
program_source:5647:50: error: unknown type name 'block_iq1_m'
    device const block_iq1_m * x = (device const block_iq1_m *) (src0 + offset0);
                                                 ^
program_source:5653:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5659:5: error: unknown type name 'iq1m_scale_t'
    iq1m_scale_t scale;
    ^
program_source:5671:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5672:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5674:22: error: unknown type name 'block_iq1_m'
        device const block_iq1_m * xr = x + ibl;
                     ^
program_source:5682:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((qh[0] << 8) & 0x700)));
                                                            ^
program_source:5683:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((qh[0] << 4) & 0x700)));
                                                            ^
program_source:5684:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid3 = (constant uint8_t *)(iq1s_grid_gpu + (qs[2] | ((qh[1] << 8) & 0x700)));
                                                            ^
program_source:5685:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid4 = (constant uint8_t *)(iq1s_grid_gpu + (qs[3] | ((qh[1] << 4) & 0x700)));
                                                            ^
program_source:5694:65: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                ^
program_source:5694:83: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                  ^
program_source:5694:128: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                               ^
program_source:5694:146: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                                                 ^
program_source:5695:65: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                ^
program_source:5695:83: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                  ^
program_source:5695:128: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                               ^
program_source:5695:146: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                                                 ^
program_source:5730:30: error: use of undeclared identifier 'QK4_NL'
    const int nb = args.ne00/QK4_NL;
                             ^
program_source:5742:18: error: unknown type name 'block_iq4_nl'
    device const block_iq4_nl * x = (device const block_iq4_nl *) (src0 + offset0);
                 ^
program_source:5742:51: error: unknown type name 'block_iq4_nl'
    device const block_iq4_nl * x = (device const block_iq4_nl *) (src0 + offset0);
                                                  ^
program_source:5754:40: error: use of undeclared identifier 'QK4_NL'
    device const float * yb = y + ix * QK4_NL + it * 8;
                                       ^
program_source:5768:26: error: unknown type name 'block_iq4_nl'
            device const block_iq4_nl & xb = x[row*nb + ib];
                         ^
program_source:5795:20: error: use of undeclared identifier 'QK4_NL'
        yb += 16 * QK4_NL;
                   ^
program_source:5820:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5832:18: error: unknown type name 'block_iq4_xs'
    device const block_iq4_xs * x = (device const block_iq4_xs *) (src0 + offset0);
                 ^
program_source:5832:51: error: unknown type name 'block_iq4_xs'
    device const block_iq4_xs * x = (device const block_iq4_xs *) (src0 + offset0);
                                                  ^
program_source:5846:40: error: use of undeclared identifier 'QK_K'
    device const float * yb = y + ix * QK_K + ib * 32 + il * 8;
                                       ^
program_source:5858:26: error: unknown type name 'block_iq4_xs'
            device const block_iq4_xs & xb = x[row*nb + ibl];
                         ^
program_source:5884:19: error: use of undeclared identifier 'QK_K'
        yb += 2 * QK_K;
                  ^
program_source:5899:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5907:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq1_s_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                          ^
program_source:5912:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5920:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq1_m_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                          ^
program_source:5925:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5934:44: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq4_nl_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                           ^
program_source:5939:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5948:44: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq4_xs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                           ^
program_source:6053:18: error: unknown type name 'ggml_metal_kargs_mul_mm'
        constant ggml_metal_kargs_mul_mm & args,
                 ^
program_source:6335:18: error: unknown type name 'ggml_metal_kargs_mul_mm_id'
        constant ggml_metal_kargs_mul_mm_id & args,
                 ^
program_source:6407:36: error: use of undeclared identifier 'block_q4_0'
typedef decltype(kernel_get_rows_q<block_q4_0, 2, dequantize_q4_0>) get_rows_q_t;
                                   ^
program_source:6409:89: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_get_rows_q4_0")]]    kernel get_rows_q_t kernel_get_rows_q<block_q4_0,    2, dequantize_q4_0>;
                                                                                        ^
program_source:6410:89: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_get_rows_q4_1")]]    kernel get_rows_q_t kernel_get_rows_q<block_q4_1,    2, dequantize_q4_1>;
                                                                                        ^
program_source:6411:89: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_get_rows_q5_0")]]    kernel get_rows_q_t kernel_get_rows_q<block_q5_0,    2, dequantize_q5_0>;
                                                                                        ^
program_source:6412:89: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_get_rows_q5_1")]]    kernel get_rows_q_t kernel_get_rows_q<block_q5_1,    2, dequantize_q5_1>;
                                                                                        ^
program_source:6413:89: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_get_rows_q8_0")]]    kernel get_rows_q_t kernel_get_rows_q<block_q8_0,    2, dequantize_q8_0>;
                                                                                        ^
program_source:6414:89: error: use of undeclared identifier 'block_q2_K'
template [[host_name("kernel_get_rows_q2_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q2_K,    QK_NL, dequantize_q2_K>;
                                                                                        ^
program_source:6415:89: error: use of undeclared identifier 'block_q3_K'
template [[host_name("kernel_get_rows_q3_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q3_K,    QK_NL, dequantize_q3_K>;
                                                                                        ^
program_source:6416:89: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_get_rows_q4_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q4_K,    QK_NL, dequantize_q4_K>;
                                                                                        ^
program_source:6417:89: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_get_rows_q5_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q5_K,    QK_NL, dequantize_q5_K>;
                                                                                        ^
program_source:6418:89: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_get_rows_q6_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q6_K,    QK_NL, dequantize_q6_K>;
                                                                                        ^
program_source:6419:89: error: use of undeclared identifier 'block_iq2_xxs'
template [[host_name("kernel_get_rows_iq2_xxs")]] kernel get_rows_q_t kernel_get_rows_q<block_iq2_xxs, QK_NL, dequantize_iq2_xxs>;
                                                                                        ^
program_source:6420:89: error: use of undeclared identifier 'block_iq2_xs'
template [[host_name("kernel_get_rows_iq2_xs")]]  kernel get_rows_q_t kernel_get_rows_q<block_iq2_xs,  QK_NL, dequantize_iq2_xs>;
                                                                                        ^
program_source:6421:89: error: use of undeclared identifier 'block_iq3_xxs'
template [[host_name("kernel_get_rows_iq3_xxs")]] kernel get_rows_q_t kernel_get_rows_q<block_iq3_xxs, QK_NL, dequantize_iq3_xxs>;
                                                                                        ^
program_source:6422:89: error: use of undeclared identifier 'block_iq3_s'
template [[host_name("kernel_get_rows_iq3_s")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq3_s,   QK_NL, dequantize_iq3_s>;
                                                                                        ^
program_source:6423:89: error: use of undeclared identifier 'block_iq2_s'
template [[host_name("kernel_get_rows_iq2_s")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq2_s,   QK_NL, dequantize_iq2_s>;
                                                                                        ^
program_source:6424:89: error: use of undeclared identifier 'block_iq1_s'
template [[host_name("kernel_get_rows_iq1_s")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq1_s,   QK_NL, dequantize_iq1_s>;
                                                                                        ^
program_source:6425:89: error: use of undeclared identifier 'block_iq1_m'
template [[host_name("kernel_get_rows_iq1_m")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq1_m,   QK_NL, dequantize_iq1_m>;
                                                                                        ^
program_source:6426:89: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_get_rows_iq4_nl")]]  kernel get_rows_q_t kernel_get_rows_q<block_iq4_nl,  2,     dequantize_iq4_nl>;
                                                                                        ^
program_source:6427:89: error: use of undeclared identifier 'block_iq4_xs'
template [[host_name("kernel_get_rows_iq4_xs")]]  kernel get_rows_q_t kernel_get_rows_q<block_iq4_xs,  QK_NL, dequantize_iq4_xs>;
                                                                                        ^
program_source:6433:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mm<half, half4x4, simdgroup_half8x8, float4x4, 1, dequantize_f32>) mat_mm_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

@7k50
Copy link
Author

7k50 commented Apr 5, 2025

cont.:

program_source:6052:13: note: possible target for call
kernel void kernel_mul_mm(
            ^
program_source:6435:69: error: explicit instantiation of 'kernel_mul_mm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_f32_f32")]]     kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   float4x4,      1,     dequantize_f32>;
                                                                    ^
program_source:6052:13: note: explicit instantiation refers here
kernel void kernel_mul_mm(
            ^
program_source:6436:69: error: explicit instantiation of 'kernel_mul_mm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_f16_f32")]]     kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   half4x4,       1,     dequantize_f16>;
                                                                    ^
program_source:6052:13: note: explicit instantiation refers here
kernel void kernel_mul_mm(
            ^
program_source:6440:123: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mm_q4_0_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q4_0,    2,     dequantize_q4_0>;
                                                                                                                          ^
program_source:6441:123: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mm_q4_1_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q4_1,    2,     dequantize_q4_1>;
                                                                                                                          ^
program_source:6442:123: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mm_q5_0_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q5_0,    2,     dequantize_q5_0>;
                                                                                                                          ^
program_source:6443:123: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mm_q5_1_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q5_1,    2,     dequantize_q5_1>;
                                                                                                                          ^
program_source:6444:123: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mm_q8_0_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q8_0,    2,     dequantize_q8_0>;
                                                                                                                          ^
program_source:6445:123: error: use of undeclared identifier 'block_q2_K'
template [[host_name("kernel_mul_mm_q2_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q2_K,    QK_NL, dequantize_q2_K>;
                                                                                                                          ^
program_source:6446:123: error: use of undeclared identifier 'block_q3_K'
template [[host_name("kernel_mul_mm_q3_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q3_K,    QK_NL, dequantize_q3_K>;
                                                                                                                          ^
program_source:6447:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mm_q4_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q4_K,    QK_NL, dequantize_q4_K>;
                                                                                                                          ^
program_source:6448:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mm_q5_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q5_K,    QK_NL, dequantize_q5_K>;
                                                                                                                          ^
program_source:6449:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mm_q6_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q6_K,    QK_NL, dequantize_q6_K>;
                                                                                                                          ^
program_source:6450:123: error: use of undeclared identifier 'block_iq2_xxs'
template [[host_name("kernel_mul_mm_iq2_xxs_f32")]] kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq2_xxs, QK_NL, dequantize_iq2_xxs>;
                                                                                                                          ^
program_source:6451:123: error: use of undeclared identifier 'block_iq2_xs'
template [[host_name("kernel_mul_mm_iq2_xs_f32")]]  kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq2_xs,  QK_NL, dequantize_iq2_xs>;
                                                                                                                          ^
program_source:6452:123: error: use of undeclared identifier 'block_iq3_xxs'
template [[host_name("kernel_mul_mm_iq3_xxs_f32")]] kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq3_xxs, QK_NL, dequantize_iq3_xxs>;
                                                                                                                          ^
program_source:6453:123: error: use of undeclared identifier 'block_iq3_s'
template [[host_name("kernel_mul_mm_iq3_s_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq3_s,   QK_NL, dequantize_iq3_s>;
                                                                                                                          ^
program_source:6454:123: error: use of undeclared identifier 'block_iq2_s'
template [[host_name("kernel_mul_mm_iq2_s_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq2_s,   QK_NL, dequantize_iq2_s>;
                                                                                                                          ^
program_source:6455:123: error: use of undeclared identifier 'block_iq1_s'
template [[host_name("kernel_mul_mm_iq1_s_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq1_s,   QK_NL, dequantize_iq1_s>;
                                                                                                                          ^
program_source:6456:123: error: use of undeclared identifier 'block_iq1_m'
template [[host_name("kernel_mul_mm_iq1_m_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq1_m,   QK_NL, dequantize_iq1_m>;
                                                                                                                          ^
program_source:6457:123: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mm_iq4_nl_f32")]]  kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq4_nl,  2,     dequantize_iq4_nl>;
                                                                                                                          ^
program_source:6458:123: error: use of undeclared identifier 'block_iq4_xs'
template [[host_name("kernel_mul_mm_iq4_xs_f32")]]  kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq4_xs,  QK_NL, dequantize_iq4_xs>;
                                                                                                                          ^
program_source:6464:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mm_id<float4x4, 1, dequantize_f32>) mat_mm_id_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:6334:13: note: possible target for call
kernel void kernel_mul_mm_id(
            ^
program_source:6466:75: error: explicit instantiation of 'kernel_mul_mm_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_id_f32_f32")]]     kernel mat_mm_id_t kernel_mul_mm_id<float4x4,      1,     dequantize_f32>;
                                                                          ^
program_source:6334:13: note: explicit instantiation refers here
kernel void kernel_mul_mm_id(
            ^
program_source:6467:75: error: explicit instantiation of 'kernel_mul_mm_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_id_f16_f32")]]     kernel mat_mm_id_t kernel_mul_mm_id<half4x4,       1,     dequantize_f16>;
                                                                          ^
program_source:6334:13: note: explicit instantiation refers here
kernel void kernel_mul_mm_id(
            ^
program_source:6471:92: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mm_id_q4_0_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q4_0,    2,     dequantize_q4_0>;
                                                                                           ^
program_source:6472:92: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mm_id_q4_1_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q4_1,    2,     dequantize_q4_1>;
                                                                                           ^
program_source:6473:92: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mm_id_q5_0_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q5_0,    2,     dequantize_q5_0>;
                                                                                           ^
program_source:6474:92: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mm_id_q5_1_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q5_1,    2,     dequantize_q5_1>;
                                                                                           ^
program_source:6475:92: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mm_id_q8_0_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q8_0,    2,     dequantize_q8_0>;
                                                                                           ^
program_source:6476:92: error: use of undeclared identifier 'block_q2_K'
template [[host_name("kernel_mul_mm_id_q2_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q2_K,    QK_NL, dequantize_q2_K>;
                                                                                           ^
program_source:6477:92: error: use of undeclared identifier 'block_q3_K'
template [[host_name("kernel_mul_mm_id_q3_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q3_K,    QK_NL, dequantize_q3_K>;
                                                                                           ^
program_source:6478:92: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mm_id_q4_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q4_K,    QK_NL, dequantize_q4_K>;
                                                                                           ^
program_source:6479:92: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mm_id_q5_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q5_K,    QK_NL, dequantize_q5_K>;
                                                                                           ^
program_source:6480:92: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mm_id_q6_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q6_K,    QK_NL, dequantize_q6_K>;
                                                                                           ^
program_source:6481:92: error: use of undeclared identifier 'block_iq2_xxs'
template [[host_name("kernel_mul_mm_id_iq2_xxs_f32")]] kernel mat_mm_id_t kernel_mul_mm_id<block_iq2_xxs, QK_NL, dequantize_iq2_xxs>;
                                                                                           ^
program_source:6482:92: error: use of undeclared identifier 'block_iq2_xs'
template [[host_name("kernel_mul_mm_id_iq2_xs_f32")]]  kernel mat_mm_id_t kernel_mul_mm_id<block_iq2_xs,  QK_NL, dequantize_iq2_xs>;
                                                                                           ^
program_source:6483:92: error: use of undeclared identifier 'block_iq3_xxs'
template [[host_name("kernel_mul_mm_id_iq3_xxs_f32")]] kernel mat_mm_id_t kernel_mul_mm_id<block_iq3_xxs, QK_NL, dequantize_iq3_xxs>;
                                                                                           ^
program_source:6484:92: error: use of undeclared identifier 'block_iq3_s'
template [[host_name("kernel_mul_mm_id_iq3_s_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq3_s,   QK_NL, dequantize_iq3_s>;
                                                                                           ^
program_source:6485:92: error: use of undeclared identifier 'block_iq2_s'
template [[host_name("kernel_mul_mm_id_iq2_s_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq2_s,   QK_NL, dequantize_iq2_s>;
                                                                                           ^
program_source:6486:92: error: use of undeclared identifier 'block_iq1_s'
template [[host_name("kernel_mul_mm_id_iq1_s_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq1_s,   QK_NL, dequantize_iq1_s>;
                                                                                           ^
program_source:6487:92: error: use of undeclared identifier 'block_iq1_m'
template [[host_name("kernel_mul_mm_id_iq1_m_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq1_m,   QK_NL, dequantize_iq1_m>;
                                                                                           ^
program_source:6488:92: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mm_id_iq4_nl_f32")]]  kernel mat_mm_id_t kernel_mul_mm_id<block_iq4_nl,  2,     dequantize_iq4_nl>;
                                                                                           ^
program_source:6489:92: error: use of undeclared identifier 'block_iq4_xs'
template [[host_name("kernel_mul_mm_id_iq4_xs_f32")]]  kernel mat_mm_id_t kernel_mul_mm_id<block_iq4_xs,  QK_NL, dequantize_iq4_xs>;
                                                                                           ^
program_source:6496:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6504:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6515:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6529:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6541:70: error: use of undeclared identifier 'ggml_metal_kargs_mul_mv'
typedef decltype(mmv_fn<kernel_mul_mv_impl<half, half4, half, half4, ggml_metal_kargs_mul_mv>>) mul_mv_impl_fn_t;
                                                                     ^
program_source:6545:18: error: unknown type name 'ggml_metal_kargs_mul_mv_id'
        constant ggml_metal_kargs_mul_mv_id & args,
                 ^
program_source:6573:5: error: unknown type name 'ggml_metal_kargs_mul_mv'
    ggml_metal_kargs_mul_mv args0 = {
    ^
program_source:6606:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv_id<mmv_fn<kernel_mul_mv_impl<float, float4, float, float4>>>) kernel_mul_mv_id_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:6544:13: note: possible target for call
kernel void kernel_mul_mv_id(
            ^
program_source:6608:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_f32_f32")]]     kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_impl<float, float4, float, float4>>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6609:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_f16_f32")]]     kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_impl<half, half4, float, float4>>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6613:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q8_0_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q8_0_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6614:127: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_id_q4_0_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q4_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6615:127: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_id_q4_1_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q4_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6616:127: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_id_q5_0_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q5_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6617:127: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_id_q5_1_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q5_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6618:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q2_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q2_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6619:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q3_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q3_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6620:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q4_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q4_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6621:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q5_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q5_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6622:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q6_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q6_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6623:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq1_s_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq1_s_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6624:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq1_m_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq1_m_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6625:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq2_xxs_f32")]] kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq2_xxs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6626:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq2_xs_f32")]]  kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq2_xs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6627:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq3_xxs_f32")]] kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq3_xxs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6628:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq3_s_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq3_s_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6629:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq2_s_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq2_s_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6630:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq4_nl_f32")]]  kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq4_nl_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6631:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq4_xs_f32")]]  kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq4_xs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
" UserInfo={NSLocalizedDescription=program_source:61:35: error: unknown type name 'block_q4_0'
void dequantize_q4_0(device const block_q4_0 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:80:38: error: unknown type name 'block_q4_0'
void dequantize_q4_0_t4(device const block_q4_0 * xb, short il, thread type4 & reg) {
                                     ^
program_source:95:35: error: unknown type name 'block_q4_1'
void dequantize_q4_1(device const block_q4_1 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:114:38: error: unknown type name 'block_q4_1'
void dequantize_q4_1_t4(device const block_q4_1 * xb, short il, thread type4 & reg) {
                                     ^
program_source:129:35: error: unknown type name 'block_q5_0'
void dequantize_q5_0(device const block_q5_0 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:161:38: error: unknown type name 'block_q5_0'
void dequantize_q5_0_t4(device const block_q5_0 * xb, short il, thread type4 & reg) {
                                     ^
program_source:191:35: error: unknown type name 'block_q5_1'
void dequantize_q5_1(device const block_q5_1 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:223:38: error: unknown type name 'block_q5_1'
void dequantize_q5_1_t4(device const block_q5_1 * xb, short il, thread type4 & reg) {
                                     ^
program_source:253:35: error: unknown type name 'block_q8_0'
void dequantize_q8_0(device const block_q8_0 *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:267:38: error: unknown type name 'block_q8_0'
void dequantize_q8_0_t4(device const block_q8_0 *xb, short il, thread type4 & reg) {
                                     ^
program_source:277:35: error: unknown type name 'block_q2_K'
void dequantize_q2_K(device const block_q2_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:296:35: error: unknown type name 'block_q3_K'
void dequantize_q3_K(device const block_q3_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:330:35: error: unknown type name 'block_q4_K'
void dequantize_q4_K(device const block_q4_K * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:349:35: error: unknown type name 'block_q5_K'
void dequantize_q5_K(device const block_q5_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:372:35: error: unknown type name 'block_q6_K'
void dequantize_q6_K(device const block_q6_K *xb, short il, thread type4x4 & reg) {
                                  ^
program_source:396:38: error: unknown type name 'block_iq2_xxs'
void dequantize_iq2_xxs(device const block_iq2_xxs * xb, short il, thread type4x4 & reg) {
                                     ^
program_source:408:52: error: use of undeclared identifier 'iq2xxs_grid'
    constant uint8_t * grid = (constant uint8_t *)(iq2xxs_grid + aux8[2*il+0]);
                                                   ^
program_source:409:21: error: use of undeclared identifier 'ksigns_iq2xs'
    uint8_t signs = ksigns_iq2xs[(aux32_s >> 14*il) & 127];
                    ^
program_source:411:49: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                ^
program_source:413:33: error: use of undeclared identifier 'iq2xxs_grid'
    grid = (constant uint8_t *)(iq2xxs_grid + aux8[2*il+1]);
                                ^
program_source:414:13: error: use of undeclared identifier 'ksigns_iq2xs'
    signs = ksigns_iq2xs[(aux32_s >> (14*il+7)) & 127];
            ^
program_source:416:51: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2+i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                  ^
program_source:421:37: error: unknown type name 'block_iq2_xs'
void dequantize_iq2_xs(device const block_iq2_xs * xb, short il, thread type4x4 & reg) {
                                    ^
program_source:429:52: error: use of undeclared identifier 'iq2xs_grid'
    constant uint8_t * grid = (constant uint8_t *)(iq2xs_grid + (q2[2*il+0] & 511));
                                                   ^
program_source:430:21: error: use of undeclared identifier 'ksigns_iq2xs'
    uint8_t signs = ksigns_iq2xs[q2[2*il+0] >> 9];
                    ^
program_source:432:49: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                ^
program_source:434:33: error: use of undeclared identifier 'iq2xs_grid'
    grid = (constant uint8_t *)(iq2xs_grid + (q2[2*il+1] & 511));
                                ^
program_source:435:13: error: use of undeclared identifier 'ksigns_iq2xs'
    signs = ksigns_iq2xs[q2[2*il+1] >> 9];
            ^
program_source:437:51: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2+i/4][i%4] = dl * grid[i] * (signs & kmask_iq2xs[i] ? -1.f : 1.f);
                                                  ^
program_source:442:38: error: unknown type name 'block_iq3_xxs'
void dequantize_iq3_xxs(device const block_iq3_xxs * xb, short il, thread type4x4 & reg) {
                                     ^
program_source:449:70: error: use of undeclared identifier 'QK_K'
    device const uint16_t * gas = (device const uint16_t *)(xb->qs + QK_K/4) + 2*ib32;
                                                                     ^
program_source:452:53: error: use of undeclared identifier 'iq3xxs_grid'
    constant uint8_t * grid1 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+0]);
                                                    ^
program_source:453:53: error: use of undeclared identifier 'iq3xxs_grid'
    constant uint8_t * grid2 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+1]);
                                                    ^
program_source:454:21: error: use of undeclared identifier 'ksigns_iq2xs'
    uint8_t signs = ksigns_iq2xs[(aux32 >> 14*il) & 127];
                    ^
program_source:456:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[0][i] = dl * grid1[i] * (signs & kmask_iq2xs[i+0] ? -1.f : 1.f);
                                             ^
program_source:457:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[1][i] = dl * grid2[i] * (signs & kmask_iq2xs[i+4] ? -1.f : 1.f);
                                             ^
program_source:459:34: error: use of undeclared identifier 'iq3xxs_grid'
    grid1 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+2]);
                                 ^
program_source:460:34: error: use of undeclared identifier 'iq3xxs_grid'
    grid2 = (constant uint8_t *)(iq3xxs_grid + q3[4*il+3]);
                                 ^
program_source:461:13: error: use of undeclared identifier 'ksigns_iq2xs'
    signs = ksigns_iq2xs[(aux32 >> (14*il+7)) & 127];
            ^
program_source:463:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2][i] = dl * grid1[i] * (signs & kmask_iq2xs[i+0] ? -1.f : 1.f);
                                             ^
program_source:464:46: error: use of undeclared identifier 'kmask_iq2xs'
        reg[3][i] = dl * grid2[i] * (signs & kmask_iq2xs[i+4] ? -1.f : 1.f);
                                             ^
program_source:469:36: error: unknown type name 'block_iq3_s'
void dequantize_iq3_s(device const block_iq3_s * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:479:53: error: use of undeclared identifier 'iq3s_grid'
    constant uint8_t * grid1 = (constant uint8_t *)(iq3s_grid + (qs[4*il+0] | ((qh << 8) & 256)));
                                                    ^
program_source:480:53: error: use of undeclared identifier 'iq3s_grid'
    constant uint8_t * grid2 = (constant uint8_t *)(iq3s_grid + (qs[4*il+1] | ((qh << 7) & 256)));
                                                    ^
program_source:482:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[0][i] = dl * grid1[i] * select(1, -1, signs[0] & kmask_iq2xs[i+0]);
                                                             ^
program_source:483:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[1][i] = dl * grid2[i] * select(1, -1, signs[0] & kmask_iq2xs[i+4]);
                                                             ^
program_source:485:34: error: use of undeclared identifier 'iq3s_grid'
    grid1 = (constant uint8_t *)(iq3s_grid + (qs[4*il+2] | ((qh << 6) & 256)));
                                 ^
program_source:486:34: error: use of undeclared identifier 'iq3s_grid'
    grid2 = (constant uint8_t *)(iq3s_grid + (qs[4*il+3] | ((qh << 5) & 256)));
                                 ^
program_source:488:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[2][i] = dl * grid1[i] * select(1, -1, signs[1] & kmask_iq2xs[i+0]);
                                                             ^
program_source:489:62: error: use of undeclared identifier 'kmask_iq2xs'
        reg[3][i] = dl * grid2[i] * select(1, -1, signs[1] & kmask_iq2xs[i+4]);
                                                             ^
program_source:494:36: error: unknown type name 'block_iq2_s'
void dequantize_iq2_s(device const block_iq2_s * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:501:41: error: use of undeclared identifier 'QK_K'
    device const uint8_t * signs = qs + QK_K/8;
                                        ^
program_source:504:53: error: use of undeclared identifier 'iq2s_grid'
    constant uint8_t * grid1 = (constant uint8_t *)(iq2s_grid + (qs[0] | ((qh << 8) & 0x300)));
                                                    ^
program_source:505:53: error: use of undeclared identifier 'iq2s_grid'
    constant uint8_t * grid2 = (constant uint8_t *)(iq2s_grid + (qs[1] | ((qh << 6) & 0x300)));
                                                    ^
program_source:507:68: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4+0][i%4] = dl * grid1[i] * select(1, -1, signs[0] & kmask_iq2xs[i]);
                                                                   ^
program_source:508:68: error: use of undeclared identifier 'kmask_iq2xs'
        reg[i/4+2][i%4] = dl * grid2[i] * select(1, -1, signs[1] & kmask_iq2xs[i]);
                                                                   ^
program_source:513:36: error: unknown type name 'block_iq1_s'
void dequantize_iq1_s(device const block_iq1_s * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:521:53: error: use of undeclared identifier 'IQ1S_DELTA'
    const float ml = dl * (qh[ib32] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA);
                                                    ^
program_source:521:71: error: use of undeclared identifier 'IQ1S_DELTA'
    const float ml = dl * (qh[ib32] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA);
                                                                      ^
program_source:523:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((h << 8) & 0x700)));
                                                    ^
program_source:524:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((h << 5) & 0x700)));
                                                    ^
program_source:534:36: error: unknown type name 'block_iq1_m'
void dequantize_iq1_m(device const block_iq1_m * xb, short il, thread type4x4 & reg) {
                                   ^
program_source:540:5: error: unknown type name 'iq1m_scale_t'
    iq1m_scale_t scale;
    ^
program_source:548:49: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml1 = dl * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                ^
program_source:548:67: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml1 = dl * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                  ^
program_source:549:49: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml2 = dl * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                ^
program_source:549:67: error: use of undeclared identifier 'IQ1M_DELTA'
    const float ml2 = dl * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                  ^
program_source:550:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((qh[0] << 8) & 0x700)));
                                                    ^
program_source:551:53: error: use of undeclared identifier 'iq1s_grid_gpu'
    constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((qh[0] << 4) & 0x700)));
                                                    ^
program_source:561:37: error: unknown type name 'block_iq4_nl'
void dequantize_iq4_nl(device const block_iq4_nl * xb, short il, thread type4x4 & reg) {
                                    ^
program_source:576:40: error: unknown type name 'block_iq4_nl'
void dequantize_iq4_nl_t4(device const block_iq4_nl * xb, short il, thread type4 & reg) {
                                       ^
program_source:589:37: error: unknown type name 'block_iq4_xs'
void dequantize_iq4_xs(device const block_iq4_xs * xb, short il, thread type4x4 & reg) {
                                    ^
program_source:617:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:643:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:669:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:695:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:722:18: error: unknown type name 'ggml_metal_kargs_repeat'
        constant ggml_metal_kargs_repeat & args,
                 ^
program_source:745:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_repeat<float>) kernel_repeat_t;
                 ^~~~~~~~~~~~~~~~~~~~
program_source:721:13: note: possible target for call
kernel void kernel_repeat(
            ^
program_source:747:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_f32")]] kernel kernel_repeat_t kernel_repeat<float>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:748:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_f16")]] kernel kernel_repeat_t kernel_repeat<half>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:749:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_i32")]] kernel kernel_repeat_t kernel_repeat<int>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:750:68: error: explicit instantiation of 'kernel_repeat' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_repeat_i16")]] kernel kernel_repeat_t kernel_repeat<short>;
                                                                   ^
program_source:721:13: note: explicit instantiation refers here
kernel void kernel_repeat(
            ^
program_source:755:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:765:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:775:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:785:18: error: unknown type name 'ggml_metal_kargs_bin'
        constant ggml_metal_kargs_bin & args,
                 ^
program_source:1425:18: error: unknown type name 'ggml_metal_kargs_norm'
        constant ggml_metal_kargs_norm & args,
                 ^
program_source:1492:18: error: unknown type name 'ggml_metal_kargs_rms_norm'
        constant ggml_metal_kargs_rms_norm & args,
                 ^
program_source:1627:43: error: unknown type name 'block_q4_0'
inline float block_q_n_dot_y(device const block_q4_0 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1648:43: error: unknown type name 'block_q4_1'
inline float block_q_n_dot_y(device const block_q4_1 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1670:43: error: unknown type name 'block_q5_0'
inline float block_q_n_dot_y(device const block_q5_0 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1681:71: error: use of undeclared identifier 'QK5_0'
        acc[2] += yl[i + 8] * ((qs[i / 2] & 0x00F0) | ((qh >> (i+0+il+QK5_0/2) << 8 ) & 0x00100));
                                                                      ^
program_source:1682:71: error: use of undeclared identifier 'QK5_0'
        acc[3] += yl[i + 9] * ((qs[i / 2] & 0xF000) | ((qh >> (i+1+il+QK5_0/2) << 16) & 0x10000));
                                                                      ^
program_source:1692:43: error: unknown type name 'block_q5_1'
inline float block_q_n_dot_y(device const block_q5_1 * qb_curr, float sumy, thread float * yl, int il) {
                                          ^
program_source:1704:71: error: use of undeclared identifier 'QK5_0'
        acc[2] += yl[i + 8] * ((qs[i / 2] & 0x00F0) | ((qh >> (i+0+il+QK5_0/2) << 8 ) & 0x00100));
                                                                      ^
program_source:1705:71: error: use of undeclared identifier 'QK5_0'
        acc[3] += yl[i + 9] * ((qs[i / 2] & 0xF000) | ((qh >> (i+1+il+QK5_0/2) << 16) & 0x10000));
                                                                      ^
program_source:1728:30: error: use of undeclared identifier 'QK4_0'
    const int nb = args.ne00/QK4_0;
                             ^
program_source:1759:38: error: use of undeclared identifier 'QK4_0'
    device const float * yb = y + ix*QK4_0 + il;
                                     ^
program_source:1781:15: error: use of undeclared identifier 'QK4_0'
        yb += QK4_0 * 16;
              ^
program_source:1796:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1803:26: error: use of undeclared identifier 'block_q4_0'
    mul_vec_q_n_f32_impl<block_q4_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                         ^
program_source:1807:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1814:27: error: use of undeclared identifier 'block_q4_1'
     mul_vec_q_n_f32_impl<block_q4_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                          ^
program_source:1818:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1825:26: error: use of undeclared identifier 'block_q5_0'
    mul_vec_q_n_f32_impl<block_q5_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                         ^
program_source:1829:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1836:26: error: use of undeclared identifier 'block_q5_1'
    mul_vec_q_n_f32_impl<block_q5_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH, constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                         ^
program_source:1855:30: error: use of undeclared identifier 'QK8_0'
    const int nb = args.ne00/QK8_0;
                             ^
program_source:1872:18: error: unknown type name 'block_q8_0'
    device const block_q8_0 * ax[nr];
                 ^
program_source:1876:33: error: unknown type name 'block_q8_0'
        ax[row] = (device const block_q8_0 *) ((device char *) src0 + offset0);
                                ^
program_source:1885:38: error: use of undeclared identifier 'QK8_0'
    device const float * yb = y + ix*QK8_0 + il*NB_Q8_0;
                                     ^
program_source:1918:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:1925:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q8_0_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:1932:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2033:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2138:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2155:18: error: unknown type name 'ggml_metal_kargs_mul_mv_ext'
        constant ggml_metal_kargs_mul_mv_ext & args,
                 ^
program_source:2170:53: error: use of undeclared identifier 'block_q8_0'
typedef decltype(kernel_mul_mv_ext_q4_f32_disp  <2, block_q8_0, 32,  dequantize_q8_0_t4>) mul_mv_ext_q4_f32_t;
                                                    ^
program_source:2171:53: error: use of undeclared identifier 'block_q4_K'
typedef decltype(kernel_mul_mv_ext_q4x4_f32_disp<2, block_q4_K, 256, dequantize_q4_K>)    mul_mv_ext_q4x4_f32_t;
                                                    ^
program_source:2173:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_2")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2174:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_3")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2175:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_4")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2176:88: error: explicit instantiation of 'kernel_mul_mv_ext_q4_f32_disp' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_ext_f16_f32_r1_5")]]    kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, half4,        4,  dequantize_f16_t4>;
                                                                                       ^
program_source:2137:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_ext_q4_f32_disp(
            ^
program_source:2178:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2179:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2180:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2181:121: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_ext_q4_0_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q4_0,   32, dequantize_q4_0_t4>;
                                                                                                                        ^
program_source:2183:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2184:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2185:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2186:121: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_ext_q4_1_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q4_1,   32, dequantize_q4_1_t4>;
                                                                                                                        ^
program_source:2188:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2189:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2190:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2191:121: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_ext_q5_0_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q5_0,   32, dequantize_q5_0_t4>;
                                                                                                                        ^
program_source:2193:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2194:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2195:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2196:121: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_ext_q5_1_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q5_1,   32, dequantize_q5_1_t4>;
                                                                                                                        ^
program_source:2198:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_2")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2199:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_3")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2200:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_4")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2201:121: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mv_ext_q8_0_f32_r1_5")]]   kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_q8_0,   32, dequantize_q8_0_t4>;
                                                                                                                        ^
program_source:2203:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_2")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<2, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2204:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_3")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<3, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2205:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_4")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<4, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2206:121: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mv_ext_iq4_nl_f32_r1_5")]] kernel mul_mv_ext_q4_f32_t kernel_mul_mv_ext_q4_f32_disp<5, block_iq4_nl, 32, dequantize_iq4_nl_t4>;
                                                                                                                        ^
program_source:2208:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_2")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<2, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2209:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_3")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<3, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2210:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_4")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<4, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2211:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mv_ext_q4_K_f32_r1_5")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<5, block_q4_K, 256, dequantize_q4_K>;
                                                                                                                          ^
program_source:2213:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_2")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<2, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2214:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_3")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<3, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2215:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_4")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<4, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2216:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mv_ext_q5_K_f32_r1_5")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<5, block_q5_K, 256, dequantize_q5_K>;
                                                                                                                          ^
program_source:2218:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_2")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<2, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2219:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_3")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<3, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2220:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_4")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<4, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2221:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mv_ext_q6_K_f32_r1_5")]] kernel mul_mv_ext_q4x4_f32_t kernel_mul_mv_ext_q4x4_f32_disp<5, block_q6_K, 256, dequantize_q6_K>;
                                                                                                                          ^
program_source:2296:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:2302:51: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_impl<T0, T04, T1, T14, constant ggml_metal_kargs_mul_mv &>(
                                                  ^
program_source:2311:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv<half, half4, half, half4>) mul_mv_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:2295:13: note: possible target for call
kernel void kernel_mul_mv(
            ^
program_source:2313:67: error: explicit instantiation of 'kernel_mul_mv' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f32_f32")]]   kernel mul_mv_t kernel_mul_mv<float,  float4,  float,  float4>;
                                                                  ^
program_source:2295:13: note: explicit instantiation refers here
kernel void kernel_mul_mv(
            ^
program_source:2314:67: error: explicit instantiation of 'kernel_mul_mv' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f32")]]   kernel mul_mv_t kernel_mul_mv<half,   half4,   float,  float4>;
                                                                  ^
program_source:2295:13: note: explicit instantiation refers here
kernel void kernel_mul_mv(
            ^
program_source:2315:67: error: explicit instantiation of 'kernel_mul_mv' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f16")]]   kernel mul_mv_t kernel_mul_mv<half,   half4,   half,   half4>;
                                                                  ^
program_source:2295:13: note: explicit instantiation refers here
kernel void kernel_mul_mv(
            ^
program_source:2323:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:2371:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv_1row<half, half4>) mul_mv_1row_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:2322:13: note: possible target for call
kernel void kernel_mul_mv_1row(
            ^
program_source:2373:76: error: explicit instantiation of 'kernel_mul_mv_1row' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f32_1row")]]  kernel mul_mv_1row_t kernel_mul_mv_1row<half,   half4>;
                                                                           ^
program_source:2322:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_1row(
            ^
program_source:2381:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:2418:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv_l4<half, half4>) mul_mv_l4_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:2380:13: note: possible target for call
kernel void kernel_mul_mv_l4(
            ^
program_source:2420:72: error: explicit instantiation of 'kernel_mul_mv_l4' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_f16_f32_l4")]]  kernel mul_mv_l4_t kernel_mul_mv_l4<half, half4>;
                                                                       ^
program_source:2380:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_l4(
            ^
program_source:2465:18: error: unknown type name 'ggml_metal_kargs_rope'
        constant ggml_metal_kargs_rope & args,
                 ^
program_source:2518:18: error: unknown type name 'ggml_metal_kargs_rope'
        constant ggml_metal_kargs_rope & args,
                 ^
program_source:2569:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_rope_norm<float>) kernel_rope_norm_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~
program_source:2464:13: note: possible target for call
kernel void kernel_rope_norm(
            ^
program_source:2570:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_rope_neox<float>) kernel_rope_neox_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~
program_source:2517:13: note: possible target for call
kernel void kernel_rope_neox(
            ^
program_source:2572:74: error: explicit instantiation of 'kernel_rope_norm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_norm_f32")]] kernel kernel_rope_norm_t kernel_rope_norm<float>;
                                                                         ^
program_source:2464:13: note: explicit instantiation refers here
kernel void kernel_rope_norm(
            ^
program_source:2573:74: error: explicit instantiation of 'kernel_rope_norm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_norm_f16")]] kernel kernel_rope_norm_t kernel_rope_norm<half>;
                                                                         ^
program_source:2464:13: note: explicit instantiation refers here
kernel void kernel_rope_norm(
            ^
program_source:2575:74: error: explicit instantiation of 'kernel_rope_neox' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_neox_f32")]] kernel kernel_rope_neox_t kernel_rope_neox<float>;
                                                                         ^
program_source:2517:13: note: explicit instantiation refers here
kernel void kernel_rope_neox(
            ^
program_source:2576:74: error: explicit instantiation of 'kernel_rope_neox' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_rope_neox_f16")]] kernel kernel_rope_neox_t kernel_rope_neox<half>;
                                                                         ^
program_source:2517:13: note: explicit instantiation refers here
kernel void kernel_rope_neox(
            ^
program_source:3093:18: error: unknown type name 'ggml_metal_kargs_flash_attn_ext'
        constant ggml_metal_kargs_flash_attn_ext & args,
                 ^
program_source:3513:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_flash_attn_ext<FA_TYPES, half4x4, 1, dequantize_f16, half4x4, 1, dequantize_f16, 64>) flash_attn_ext_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3092:13: note: possible target for call
kernel void kernel_flash_attn_ext(
            ^
program_source:3515:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h64" )]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  64>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3516:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h80" )]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  80>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3517:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h96" )]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  96>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3518:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h112")]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  112>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3519:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h128")]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  128>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3520:83: error: explicit instantiation of 'kernel_flash_attn_ext' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_f16_h256")]]  kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,    1, dequantize_f16,  256>;
                                                                                  ^
program_source:3092:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext(
            ^
program_source:3531:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 64>;
                                                                                                                  ^
program_source:3532:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 80>;
                                                                                                                  ^
program_source:3533:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 96>;
                                                                                                                  ^
program_source:3534:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 112>;
                                                                                                                  ^
program_source:3535:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 128>;
                                                                                                                  ^
program_source:3536:115: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_q4_0_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0, 2, dequantize_q4_0, 256>;
                                                                                                                  ^
program_source:3538:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 64>;
                                                                                                                  ^
program_source:3539:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 80>;
                                                                                                                  ^
program_source:3540:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 96>;
                                                                                                                  ^
program_source:3541:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 112>;
                                                                                                                  ^
program_source:3542:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 128>;
                                                                                                                  ^
program_source:3543:115: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_q4_1_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1, 2, dequantize_q4_1, 256>;
                                                                                                                  ^
program_source:3545:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 64>;
                                                                                                                  ^
program_source:3546:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 80>;
                                                                                                                  ^
program_source:3547:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 96>;
                                                                                                                  ^
program_source:3548:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 112>;
                                                                                                                  ^
program_source:3549:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 128>;
                                                                                                                  ^
program_source:3550:115: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_q5_0_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0, 2, dequantize_q5_0, 256>;
                                                                                                                  ^
program_source:3552:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 64>;
                                                                                                                  ^
program_source:3553:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 80>;
                                                                                                                  ^
program_source:3554:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 96>;
                                                                                                                  ^
program_source:3555:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 112>;
                                                                                                                  ^
program_source:3556:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 128>;
                                                                                                                  ^
program_source:3557:115: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_q5_1_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1, 2, dequantize_q5_1, 256>;
                                                                                                                  ^
program_source:3559:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h64" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 64>;
                                                                                                                  ^
program_source:3560:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h80" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 80>;
                                                                                                                  ^
program_source:3561:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h96" )]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 96>;
                                                                                                                  ^
program_source:3562:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h112")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 112>;
                                                                                                                  ^
program_source:3563:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h128")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 128>;
                                                                                                                  ^
program_source:3564:115: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_q8_0_h256")]] kernel flash_attn_ext_t kernel_flash_attn_ext<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0, 2, dequantize_q8_0, 256>;
                                                                                                                  ^
program_source:3588:18: error: unknown type name 'ggml_metal_kargs_flash_attn_ext'
        constant ggml_metal_kargs_flash_attn_ext & args,
                 ^
program_source:3904:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_flash_attn_ext_vec<FA_TYPES, half4x4, 1, dequantize_f16, half4x4, 1, dequantize_f16, 128>) flash_attn_ext_vec_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3587:13: note: possible target for call
kernel void kernel_flash_attn_ext_vec(
            ^
program_source:3906:91: error: explicit instantiation of 'kernel_flash_attn_ext_vec' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_vec_f16_h128")]]  kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,     1, dequantize_f16,  128>;
                                                                                          ^
program_source:3587:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext_vec(
            ^
program_source:3910:127: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_vec_q4_0_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0,  2, dequantize_q4_0, 128>;
                                                                                                                              ^
program_source:3911:127: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_vec_q4_1_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1,  2, dequantize_q4_1, 128>;
                                                                                                                              ^
program_source:3912:127: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_vec_q5_0_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0,  2, dequantize_q5_0, 128>;
                                                                                                                              ^
program_source:3913:127: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_vec_q5_1_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1,  2, dequantize_q5_1, 128>;
                                                                                                                              ^
program_source:3914:127: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_vec_q8_0_h128")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0,  2, dequantize_q8_0, 128>;
                                                                                                                              ^
program_source:3916:91: error: explicit instantiation of 'kernel_flash_attn_ext_vec' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_flash_attn_ext_vec_f16_h256")]]  kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, half4x4,    1, dequantize_f16,  half4x4,     1, dequantize_f16,  256>;
                                                                                          ^
program_source:3587:13: note: explicit instantiation refers here
kernel void kernel_flash_attn_ext_vec(
            ^
program_source:3920:127: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_flash_attn_ext_vec_q4_0_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_0, 2, dequantize_q4_0, block_q4_0,  2, dequantize_q4_0, 256>;
                                                                                                                              ^
program_source:3921:127: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_flash_attn_ext_vec_q4_1_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q4_1, 2, dequantize_q4_1, block_q4_1,  2, dequantize_q4_1, 256>;
                                                                                                                              ^
program_source:3922:127: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_flash_attn_ext_vec_q5_0_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_0, 2, dequantize_q5_0, block_q5_0,  2, dequantize_q5_0, 256>;
                                                                                                                              ^
program_source:3923:127: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_flash_attn_ext_vec_q5_1_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q5_1, 2, dequantize_q5_1, block_q5_1,  2, dequantize_q5_1, 256>;
                                                                                                                              ^
program_source:3924:127: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_flash_attn_ext_vec_q8_0_h256")]] kernel flash_attn_ext_vec_t kernel_flash_attn_ext_vec<FA_TYPES, block_q8_0, 2, dequantize_q8_0, block_q8_0,  2, dequantize_q8_0, 256>;
                                                                                                                              ^
program_source:3930:14: error: unknown type name 'ggml_metal_kargs_set'
    constant ggml_metal_kargs_set & args,
             ^
program_source:3955:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_set<float>) kernel_set_t;
                 ^~~~~~~~~~~~~~~~~
program_source:3929:13: note: possible target for call
kernel void kernel_set(
            ^
program_source:3957:62: error: explicit instantiation of 'kernel_set' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_set_f32")]] kernel kernel_set_t kernel_set<float>;
                                                             ^
program_source:3929:13: note: explicit instantiation refers here
kernel void kernel_set(
            ^
program_source:3958:62: error: explicit instantiation of 'kernel_set' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_set_i32")]] kernel kernel_set_t kernel_set<int32_t>;
                                                             ^
program_source:3929:13: note: explicit instantiation refers here
kernel void kernel_set(
            ^
program_source:3962:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:3987:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_cpy<float, float>) kernel_cpy_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~
program_source:3961:13: note: possible target for call
kernel void kernel_cpy(
            ^
program_source:3989:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f32_f32")]]   kernel kernel_cpy_t kernel_cpy<float,  float>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:3990:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f32_f16")]]   kernel kernel_cpy_t kernel_cpy<float,  half>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:3994:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f16_f32")]]   kernel kernel_cpy_t kernel_cpy<half,   float>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:3995:68: error: explicit instantiation of 'kernel_cpy' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_cpy_f16_f16")]]   kernel kernel_cpy_t kernel_cpy<half,   half>;
                                                                   ^
program_source:3961:13: note: explicit instantiation refers here
kernel void kernel_cpy(
            ^
program_source:4002:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4017:97: error: use of undeclared identifier 'QK8_0'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK8_0;
                                                                                                ^
program_source:4019:12: error: unknown type name 'block_q8_0'
    device block_q8_0 * dst_data = (device block_q8_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4019:44: error: unknown type name 'block_q8_0'
    device block_q8_0 * dst_data = (device block_q8_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4021:32: error: use of undeclared identifier 'QK8_0'
    for (int64_t i00 = tpitg.x*QK8_0; i00 < args.ne00; i00 += ntg.x*QK8_0) {
                               ^
program_source:4021:69: error: use of undeclared identifier 'QK8_0'
    for (int64_t i00 = tpitg.x*QK8_0; i00 < args.ne00; i00 += ntg.x*QK8_0) {
                                                                    ^
program_source:4026:29: error: use of undeclared identifier 'QK8_0'
        for (int j = 0; j < QK8_0; j++) {
                            ^
program_source:4034:22: error: use of undeclared identifier 'QK8_0'
        dst_data[i00/QK8_0].d = d;
                     ^
program_source:4036:29: error: use of undeclared identifier 'QK8_0'
        for (int j = 0; j < QK8_0; ++j) {
                            ^
program_source:4039:26: error: use of undeclared identifier 'QK8_0'
            dst_data[i00/QK8_0].qs[j] = round(x0);
                         ^
program_source:4045:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4060:97: error: use of undeclared identifier 'QK4_0'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK4_0;
                                                                                                ^
program_source:4062:12: error: unknown type name 'block_q4_0'
    device block_q4_0 * dst_data = (device block_q4_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4062:44: error: unknown type name 'block_q4_0'
    device block_q4_0 * dst_data = (device block_q4_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4064:32: error: use of undeclared identifier 'QK4_0'
    for (int64_t i00 = tpitg.x*QK4_0; i00 < args.ne00; i00 += ntg.x*QK4_0) {
                               ^
program_source:4064:69: error: use of undeclared identifier 'QK4_0'
    for (int64_t i00 = tpitg.x*QK4_0; i00 < args.ne00; i00 += ntg.x*QK4_0) {
                                                                    ^
program_source:4070:29: error: use of undeclared identifier 'QK4_0'
        for (int j = 0; j < QK4_0; j++) {
                            ^
program_source:4081:22: error: use of undeclared identifier 'QK4_0'
        dst_data[i00/QK4_0].d = d;
                     ^
program_source:4083:29: error: use of undeclared identifier 'QK4_0'
        for (int j = 0; j < QK4_0/2; ++j) {
                            ^
program_source:4085:34: error: use of undeclared identifier 'QK4_0'
            const float x1 = src[QK4_0/2 + j]*id;
                                 ^
program_source:4090:26: error: use of undeclared identifier 'QK4_0'
            dst_data[i00/QK4_0].qs[j]  = xi0;
                         ^
program_source:4091:26: error: use of undeclared identifier 'QK4_0'
            dst_data[i00/QK4_0].qs[j] |= xi1 << 4;
                         ^
program_source:4097:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4112:97: error: use of undeclared identifier 'QK4_1'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK4_1;
                                                                                                ^
program_source:4114:12: error: unknown type name 'block_q4_1'
    device block_q4_1 * dst_data = (device block_q4_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4114:44: error: unknown type name 'block_q4_1'
    device block_q4_1 * dst_data = (device block_q4_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4116:32: error: use of undeclared identifier 'QK4_1'
    for (int64_t i00 = tpitg.x*QK4_1; i00 < args.ne00; i00 += ntg.x*QK4_1) {
                               ^
program_source:4116:69: error: use of undeclared identifier 'QK4_1'
    for (int64_t i00 = tpitg.x*QK4_1; i00 < args.ne00; i00 += ntg.x*QK4_1) {
                                                                    ^
program_source:4122:29: error: use of undeclared identifier 'QK4_1'
        for (int j = 0; j < QK4_1; j++) {
                            ^
program_source:4131:22: error: use of undeclared identifier 'QK4_1'
        dst_data[i00/QK4_1].d = d;
                     ^
program_source:4132:22: error: use of undeclared identifier 'QK4_1'
        dst_data[i00/QK4_1].m = min;
                     ^
program_source:4134:29: error: use of undeclared identifier 'QK4_1'
        for (int j = 0; j < QK4_1/2; ++j) {
                            ^
program_source:4136:35: error: use of undeclared identifier 'QK4_1'
            const float x1 = (src[QK4_1/2 + j] - min)*id;
                                  ^
program_source:4141:26: error: use of undeclared identifier 'QK4_1'
            dst_data[i00/QK4_1].qs[j]  = xi0;
                         ^
program_source:4142:26: error: use of undeclared identifier 'QK4_1'
            dst_data[i00/QK4_1].qs[j] |= xi1 << 4;
                         ^
program_source:4148:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4163:97: error: use of undeclared identifier 'QK5_0'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK5_0;
                                                                                                ^
program_source:4165:12: error: unknown type name 'block_q5_0'
    device block_q5_0 * dst_data = (device block_q5_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4165:44: error: unknown type name 'block_q5_0'
    device block_q5_0 * dst_data = (device block_q5_0 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4167:32: error: use of undeclared identifier 'QK5_0'
    for (int64_t i00 = tpitg.x*QK5_0; i00 < args.ne00; i00 += ntg.x*QK5_0) {
                               ^
program_source:4167:69: error: use of undeclared identifier 'QK5_0'
    for (int64_t i00 = tpitg.x*QK5_0; i00 < args.ne00; i00 += ntg.x*QK5_0) {
                                                                    ^
program_source:4173:29: error: use of undeclared identifier 'QK5_0'
        for (int j = 0; j < QK5_0; j++) {
                            ^
program_source:4184:22: error: use of undeclared identifier 'QK5_0'
        dst_data[i00/QK5_0].d = d;
                     ^
program_source:4187:29: error: use of undeclared identifier 'QK5_0'
        for (int j = 0; j < QK5_0/2; ++j) {
                            ^
program_source:4189:34: error: use of undeclared identifier 'QK5_0'
            const float x1 = src[QK5_0/2 + j]*id;
                                 ^
program_source:4194:26: error: use of undeclared identifier 'QK5_0'
            dst_data[i00/QK5_0].qs[j] = (xi0 & 0xf) | ((xi1 & 0xf) << 4);
                         ^
program_source:4196:48: error: use of undeclared identifier 'QK5_0'
            qh |= ((xi1 & 0x10u) >> 4) << (j + QK5_0/2);
                                               ^
program_source:4200:26: error: use of undeclared identifier 'QK5_0'
            dst_data[i00/QK5_0].qh[j] = qh8[j];
                         ^
program_source:4206:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4221:97: error: use of undeclared identifier 'QK5_1'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK5_1;
                                                                                                ^
program_source:4223:12: error: unknown type name 'block_q5_1'
    device block_q5_1 * dst_data = (device block_q5_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4223:44: error: unknown type name 'block_q5_1'
    device block_q5_1 * dst_data = (device block_q5_1 *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                           ^
program_source:4225:32: error: use of undeclared identifier 'QK5_1'
    for (int64_t i00 = tpitg.x*QK5_1; i00 < args.ne00; i00 += ntg.x*QK5_1) {
                               ^
program_source:4225:69: error: use of undeclared identifier 'QK5_1'
    for (int64_t i00 = tpitg.x*QK5_1; i00 < args.ne00; i00 += ntg.x*QK5_1) {
                                                                    ^
program_source:4231:29: error: use of undeclared identifier 'QK5_1'
        for (int j = 1; j < QK5_1; j++) {
                            ^
program_source:4240:22: error: use of undeclared identifier 'QK5_1'
        dst_data[i00/QK5_1].d = d;
                     ^
program_source:4241:22: error: use of undeclared identifier 'QK5_1'
        dst_data[i00/QK5_1].m = min;
                     ^
program_source:4244:29: error: use of undeclared identifier 'QK5_1'
        for (int j = 0; j < QK5_1/2; ++j) {
                            ^
program_source:4246:35: error: use of undeclared identifier 'QK5_1'
            const float x1 = (src[QK5_1/2 + j] - min)*id;
                                  ^
program_source:4251:26: error: use of undeclared identifier 'QK5_1'
            dst_data[i00/QK5_1].qs[j] = (xi0 & 0xf) | ((xi1 & 0xf) << 4);
                         ^
program_source:4253:48: error: use of undeclared identifier 'QK5_1'
            qh |= ((xi1 & 0x10u) >> 4) << (j + QK5_1/2);
                                               ^
program_source:4257:26: error: use of undeclared identifier 'QK5_1'
            dst_data[i00/QK5_1].qh[j] = qh8[j];
                         ^
program_source:4274:18: error: unknown type name 'ggml_metal_kargs_cpy'
        constant ggml_metal_kargs_cpy & args,
                 ^
program_source:4289:97: error: use of undeclared identifier 'QK4_NL'
    const int64_t i0 = (n - i3*args.ne2*args.ne1*args.ne0 - i2*args.ne1*args.ne0 - i1*args.ne0)/QK4_NL;
                                                                                                ^
program_source:4291:12: error: unknown type name 'block_iq4_nl'
    device block_iq4_nl * dst_data = (device block_iq4_nl *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
           ^
program_source:4291:46: error: unknown type name 'block_iq4_nl'
    device block_iq4_nl * dst_data = (device block_iq4_nl *) (dst + i3*args.nb3 + i2*args.nb2 + i1*args.nb1 + i0*args.nb0);
                                             ^
program_source:4293:32: error: use of undeclared identifier 'QK4_NL'
    for (int64_t i00 = tpitg.x*QK4_NL; i00 < args.ne00; i00 += ntg.x*QK4_NL) {
                               ^
program_source:4293:70: error: use of undeclared identifier 'QK4_NL'
    for (int64_t i00 = tpitg.x*QK4_NL; i00 < args.ne00; i00 += ntg.x*QK4_NL) {
                                                                     ^
program_source:4299:29: error: use of undeclared identifier 'QK4_0'
        for (int j = 0; j < QK4_0; j++) {
                            ^
program_source:4311:29: error: use of undeclared identifier 'QK4_NL'
        for (int j = 0; j < QK4_NL/2; ++j) {
                            ^
program_source:4313:34: error: use of undeclared identifier 'QK4_NL'
            const float x1 = src[QK4_NL/2 + j]*id;
                                 ^
program_source:4318:26: error: use of undeclared identifier 'QK4_NL'
            dst_data[i00/QK4_NL].qs[j] = xi0 | (xi1 << 4);
                         ^
program_source:4323:34: error: use of undeclared identifier 'QK4_NL'
            const float w1 = src[QK4_NL/2 + j]*src[QK4_NL/2 + j];
                                 ^
program_source:4323:52: error: use of undeclared identifier 'QK4_NL'
            const float w1 = src[QK4_NL/2 + j]*src[QK4_NL/2 + j];
                                                   ^
program_source:4324:47: error: use of undeclared identifier 'QK4_NL'
            sumqx += w0*v0*src[j] + w1*v1*src[QK4_NL/2 + j];
                                              ^
program_source:4329:22: error: use of undeclared identifier 'QK4_NL'
        dst_data[i00/QK4_NL].d = sumq2 > 0 ? sumqx/sumq2 : d;
                     ^
program_source:4334:14: error: unknown type name 'ggml_metal_kargs_concat'
    constant ggml_metal_kargs_concat & args,
             ^
program_source:4375:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4388:18: error: unknown type name 'block_q2_K'
    device const block_q2_K * x = (device const block_q2_K *) (src0 + offset0);
                 ^
program_source:4388:49: error: unknown type name 'block_q2_K'
    device const block_q2_K * x = (device const block_q2_K *) (src0 + offset0);
                                                ^
program_source:4400:40: error: use of undeclared identifier 'QK_K'
    device const float * y4 = y + ix * QK_K + 128 * iq + 8 * ir;
                                       ^
program_source:4443:19: error: use of undeclared identifier 'QK_K'
        y4 += 4 * QK_K;
                  ^
program_source:4458:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4466:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q2_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4480:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4494:18: error: unknown type name 'block_q3_K'
    device const block_q3_K * x = (device const block_q3_K *) (src0 + offset0);
                 ^
program_source:4494:49: error: unknown type name 'block_q3_K'
    device const block_q3_K * x = (device const block_q3_K *) (src0 + offset0);
                                                ^
program_source:4536:39: error: use of undeclared identifier 'QK_K'
    device const float * y1 = yy + ix*QK_K + y_offset;
                                      ^
program_source:4603:19: error: use of undeclared identifier 'QK_K'
        y1 += 4 * QK_K;
                  ^
program_source:4622:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4630:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q3_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4653:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4666:18: error: unknown type name 'block_q4_K'
    device const block_q4_K * x = (device const block_q4_K *) (src0 + offset0);
                 ^
program_source:4666:49: error: unknown type name 'block_q4_K'
    device const block_q4_K * x = (device const block_q4_K *) (src0 + offset0);
                                                ^
program_source:4673:40: error: use of undeclared identifier 'QK_K'
    device const float * y4 = y + ix * QK_K + 64 * iq + 8 * ir;
                                       ^
program_source:4725:19: error: use of undeclared identifier 'QK_K'
        y4 += 4 * QK_K;
                  ^
program_source:4740:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4748:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q4_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4762:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4776:18: error: unknown type name 'block_q5_K'
    device const block_q5_K * x = (device const block_q5_K *) (src0 + offset0);
                 ^
program_source:4776:49: error: unknown type name 'block_q5_K'
    device const block_q5_K * x = (device const block_q5_K *) (src0 + offset0);
                                                ^
program_source:4805:39: error: use of undeclared identifier 'QK_K'
    device const float * y1 = yy + ix*QK_K + y_offset;
                                      ^
program_source:4857:19: error: use of undeclared identifier 'QK_K'
        y1 += 4 * QK_K;
                  ^
program_source:4872:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4880:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q5_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4899:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4913:18: error: unknown type name 'block_q6_K'
    device const block_q6_K * x = (device const block_q6_K *) (src0 + offset0);
                 ^
program_source:4913:49: error: unknown type name 'block_q6_K'
    device const block_q6_K * x = (device const block_q6_K *) (src0 + offset0);
                                                ^
program_source:4936:43: error: use of undeclared identifier 'QK_K'
        device const float * y = yy + i * QK_K + y_offset;
                                          ^
program_source:4962:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:4970:42: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_q6_K_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                         ^
program_source:4986:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:4999:18: error: unknown type name 'block_iq2_xxs'
    device const block_iq2_xxs * x = (device const block_iq2_xxs *) (src0 + offset0);
                 ^
program_source:4999:52: error: unknown type name 'block_iq2_xxs'
    device const block_iq2_xxs * x = (device const block_iq2_xxs *) (src0 + offset0);
                                                   ^
program_source:5005:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5012:59: error: use of undeclared identifier 'iq2xxs_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq2xxs_grid[pos + i];
                                                          ^
program_source:5015:56: error: use of undeclared identifier 'ksigns_iq2xs'
        for (int i = 0; i < nval; ++i) ssigns[pos+i] = ksigns_iq2xs[pos+i];
                                                       ^
program_source:5029:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5030:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5032:22: error: unknown type name 'block_iq2_xxs'
        device const block_iq2_xxs * xr = x + ibl;
                     ^
program_source:5048:61: error: use of undeclared identifier 'kmask_iq2xs'
                    sum += yl[8*l + j] * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f);
                                                            ^
program_source:5072:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5080:45: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq2_xxs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                            ^
program_source:5094:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5107:18: error: unknown type name 'block_iq2_xs'
    device const block_iq2_xs * x = (device const block_iq2_xs *) (src0 + offset0);
                 ^
program_source:5107:51: error: unknown type name 'block_iq2_xs'
    device const block_iq2_xs * x = (device const block_iq2_xs *) (src0 + offset0);
                                                  ^
program_source:5113:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5120:59: error: use of undeclared identifier 'iq2xs_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq2xs_grid[pos + i];
                                                          ^
program_source:5123:56: error: use of undeclared identifier 'ksigns_iq2xs'
        for (int i = 0; i < nval; ++i) ssigns[pos+i] = ksigns_iq2xs[pos+i];
                                                       ^
program_source:5137:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5138:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5140:22: error: unknown type name 'block_iq2_xs'
        device const block_iq2_xs * xr = x + ibl;
                     ^
program_source:5158:62: error: use of undeclared identifier 'kmask_iq2xs'
                    sum1 += yl[8*l + j] * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f);
                                                             ^
program_source:5165:62: error: use of undeclared identifier 'kmask_iq2xs'
                    sum2 += yl[8*l + j] * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f);
                                                             ^
program_source:5190:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5199:44: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq2_xs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                           ^
program_source:5213:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5226:18: error: unknown type name 'block_iq3_xxs'
    device const block_iq3_xxs * x = (device const block_iq3_xxs *) (src0 + offset0);
                 ^
program_source:5226:52: error: unknown type name 'block_iq3_xxs'
    device const block_iq3_xxs * x = (device const block_iq3_xxs *) (src0 + offset0);
                                                   ^
program_source:5232:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5239:59: error: use of undeclared identifier 'iq3xxs_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq3xxs_grid[pos + i];
                                                          ^
program_source:5242:56: error: use of undeclared identifier 'ksigns_iq2xs'
        for (int i = 0; i < nval; ++i) ssigns[pos+i] = ksigns_iq2xs[pos+i];
                                                       ^
program_source:5255:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5256:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5258:22: error: unknown type name 'block_iq3_xxs'
        device const block_iq3_xxs * xr = x + ibl;
                     ^
program_source:5260:74: error: use of undeclared identifier 'QK_K'
        device const uint16_t * gas = (device const uint16_t *)(xr->qs + QK_K/4) + 2 * ib;
                                                                         ^
program_source:5274:69: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[0] += yl[8*l + j + 0] * grid1[j] * (signs & kmask_iq2xs[j+0] ? -1.f : 1.f);
                                                                    ^
program_source:5275:69: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[1] += yl[8*l + j + 4] * grid2[j] * (signs & kmask_iq2xs[j+4] ? -1.f : 1.f);
                                                                    ^
program_source:5300:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5309:45: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq3_xxs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                            ^
program_source:5323:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5336:18: error: unknown type name 'block_iq3_s'
    device const block_iq3_s * x = (device const block_iq3_s *) (src0 + offset0);
                 ^
program_source:5336:50: error: unknown type name 'block_iq3_s'
    device const block_iq3_s * x = (device const block_iq3_s *) (src0 + offset0);
                                                 ^
program_source:5342:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5348:59: error: use of undeclared identifier 'iq3s_grid'
        for (int i = 0; i < nval; ++i) svalues[pos + i] = iq3s_grid[pos + i];
                                                          ^
program_source:5362:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5363:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5365:22: error: unknown type name 'block_iq3_s'
        device const block_iq3_s * xr = x + ibl;
                     ^
program_source:5379:63: error: use of undeclared identifier 'kmask_iq2xs'
                const threadgroup uint32_t * table1 = qh[0] & kmask_iq2xs[2*l+0] ? svalues + 256 : svalues;
                                                              ^
program_source:5380:63: error: use of undeclared identifier 'kmask_iq2xs'
                const threadgroup uint32_t * table2 = qh[0] & kmask_iq2xs[2*l+1] ? svalues + 256 : svalues;
                                                              ^
program_source:5384:85: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[0] += yl[8*l + j + 0] * grid1[j] * select(1, -1, signs[l] & kmask_iq2xs[j+0]);
                                                                                    ^
program_source:5385:85: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[1] += yl[8*l + j + 4] * grid2[j] * select(1, -1, signs[l] & kmask_iq2xs[j+4]);
                                                                                    ^
program_source:5412:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5421:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq3_s_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                          ^
program_source:5435:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5448:18: error: unknown type name 'block_iq2_s'
    device const block_iq2_s * x = (device const block_iq2_s *) (src0 + offset0);
                 ^
program_source:5448:50: error: unknown type name 'block_iq2_s'
    device const block_iq2_s * x = (device const block_iq2_s *) (src0 + offset0);
                                                 ^
program_source:5454:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5474:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5475:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5477:22: error: unknown type name 'block_iq2_s'
        device const block_iq2_s * xr = x + ibl;
                     ^
program_source:5481:45: error: use of undeclared identifier 'QK_K'
        device const uint8_t * signs = qs + QK_K/8;
                                            ^
program_source:5494:65: error: use of undeclared identifier 'iq2s_grid'
                constant uint8_t * grid1 = (constant uint8_t *)(iq2s_grid + (qs[l+0] | ((qh[0] << (8-2*l)) & 0x300)));
                                                                ^
program_source:5495:65: error: use of undeclared identifier 'iq2s_grid'
                constant uint8_t * grid2 = (constant uint8_t *)(iq2s_grid + (qs[l+2] | ((qh[0] << (4-2*l)) & 0x300)));
                                                                ^
program_source:5497:88: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[0] += yl[8*l + j +  0] * grid1[j] * select(1, -1, signs[l+0] & kmask_iq2xs[j]);
                                                                                       ^
program_source:5498:88: error: use of undeclared identifier 'kmask_iq2xs'
                    sum[1] += yl[8*l + j + 16] * grid2[j] * select(1, -1, signs[l+2] & kmask_iq2xs[j]);
                                                                                       ^
program_source:5525:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5534:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq2_s_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                          ^
program_source:5548:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5561:18: error: unknown type name 'block_iq1_s'
    device const block_iq1_s * x = (device const block_iq1_s *) (src0 + offset0);
                 ^
program_source:5561:50: error: unknown type name 'block_iq1_s'
    device const block_iq1_s * x = (device const block_iq1_s *) (src0 + offset0);
                                                 ^
program_source:5567:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5581:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5582:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5584:22: error: unknown type name 'block_iq1_s'
        device const block_iq1_s * xr = x + ibl;
                     ^
program_source:5591:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((qh[0] << 8) & 0x700)));
                                                            ^
program_source:5592:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((qh[0] << 5) & 0x700)));
                                                            ^
program_source:5593:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid3 = (constant uint8_t *)(iq1s_grid_gpu + (qs[2] | ((qh[0] << 2) & 0x700)));
                                                            ^
program_source:5594:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid4 = (constant uint8_t *)(iq1s_grid_gpu + (qs[3] | ((qh[0] >> 1) & 0x700)));
                                                            ^
program_source:5603:78: error: use of undeclared identifier 'IQ1S_DELTA'
            sumf[row] += (float)dh[0] * (sum + sumy * (qh[0] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA)) * (2*((qh[0] >> 12) & 7) + 1);
                                                                             ^
program_source:5603:96: error: use of undeclared identifier 'IQ1S_DELTA'
            sumf[row] += (float)dh[0] * (sum + sumy * (qh[0] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA)) * (2*((qh[0] >> 12) & 7) + 1);
                                                                                               ^
program_source:5634:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5647:18: error: unknown type name 'block_iq1_m'
    device const block_iq1_m * x = (device const block_iq1_m *) (src0 + offset0);
                 ^
program_source:5647:50: error: unknown type name 'block_iq1_m'
    device const block_iq1_m * x = (device const block_iq1_m *) (src0 + offset0);
                                                 ^
program_source:5653:28: error: use of undeclared identifier 'QK_K'
    const int nb32 = nb * (QK_K / 32);
                           ^
program_source:5659:5: error: unknown type name 'iq1m_scale_t'
    iq1m_scale_t scale;
    ^
program_source:5671:33: error: use of undeclared identifier 'QK_K'
        const int ibl = ib32 / (QK_K / 32);
                                ^
program_source:5672:33: error: use of undeclared identifier 'QK_K'
        const int ib  = ib32 % (QK_K / 32);
                                ^
program_source:5674:22: error: unknown type name 'block_iq1_m'
        device const block_iq1_m * xr = x + ibl;
                     ^
program_source:5682:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid1 = (constant uint8_t *)(iq1s_grid_gpu + (qs[0] | ((qh[0] << 8) & 0x700)));
                                                            ^
program_source:5683:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid2 = (constant uint8_t *)(iq1s_grid_gpu + (qs[1] | ((qh[0] << 4) & 0x700)));
                                                            ^
program_source:5684:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid3 = (constant uint8_t *)(iq1s_grid_gpu + (qs[2] | ((qh[1] << 8) & 0x700)));
                                                            ^
program_source:5685:61: error: use of undeclared identifier 'iq1s_grid_gpu'
            constant uint8_t * grid4 = (constant uint8_t *)(iq1s_grid_gpu + (qs[3] | ((qh[1] << 4) & 0x700)));
                                                            ^
program_source:5694:65: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                ^
program_source:5694:83: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                  ^
program_source:5694:128: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                               ^
program_source:5694:146: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta1 = sumy[0] * (qh[0] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[1] * (qh[0] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                                                 ^
program_source:5695:65: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                ^
program_source:5695:83: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                  ^
program_source:5695:128: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                               ^
program_source:5695:146: error: use of undeclared identifier 'IQ1M_DELTA'
            const float delta2 = sumy[2] * (qh[1] & 0x08 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA) + sumy[3] * (qh[1] & 0x80 ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA);
                                                                                                                                                 ^
program_source:5730:30: error: use of undeclared identifier 'QK4_NL'
    const int nb = args.ne00/QK4_NL;
                             ^
program_source:5742:18: error: unknown type name 'block_iq4_nl'
    device const block_iq4_nl * x = (device const block_iq4_nl *) (src0 + offset0);
                 ^
program_source:5742:51: error: unknown type name 'block_iq4_nl'
    device const block_iq4_nl * x = (device const block_iq4_nl *) (src0 + offset0);
                                                  ^
program_source:5754:40: error: use of undeclared identifier 'QK4_NL'
    device const float * yb = y + ix * QK4_NL + it * 8;
                                       ^
program_source:5768:26: error: unknown type name 'block_iq4_nl'
            device const block_iq4_nl & xb = x[row*nb + ib];
                         ^
program_source:5795:20: error: use of undeclared identifier 'QK4_NL'
        yb += 16 * QK4_NL;
                   ^
program_source:5820:30: error: use of undeclared identifier 'QK_K'
    const int nb = args.ne00/QK_K;
                             ^
program_source:5832:18: error: unknown type name 'block_iq4_xs'
    device const block_iq4_xs * x = (device const block_iq4_xs *) (src0 + offset0);
                 ^
program_source:5832:51: error: unknown type name 'block_iq4_xs'
    device const block_iq4_xs * x = (device const block_iq4_xs *) (src0 + offset0);
                                                  ^
program_source:5846:40: error: use of undeclared identifier 'QK_K'
    device const float * yb = y + ix * QK_K + ib * 32 + il * 8;
                                       ^
program_source:5858:26: error: unknown type name 'block_iq4_xs'
            device const block_iq4_xs & xb = x[row*nb + ibl];
                         ^
program_source:5884:19: error: use of undeclared identifier 'QK_K'
        yb += 2 * QK_K;
                  ^
program_source:5899:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5907:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq1_s_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                          ^
program_source:5912:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5920:43: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq1_m_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, nullptr, tgpig, tiisg, sgitg);
                                          ^
program_source:5925:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5934:44: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq4_nl_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                           ^
program_source:5939:18: error: unknown type name 'ggml_metal_kargs_mul_mv'
        constant ggml_metal_kargs_mul_mv & args,
                 ^
program_source:5948:44: error: unknown type name 'ggml_metal_kargs_mul_mv'
    kernel_mul_mv_iq4_xs_f32_impl<constant ggml_metal_kargs_mul_mv &>(args, src0, src1, dst, shmem, tgpig, tiisg, sgitg);
                                           ^
program_source:6053:18: error: unknown type name 'ggml_metal_kargs_mul_mm'
        constant ggml_metal_kargs_mul_mm & args,
                 ^
program_source:6335:18: error: unknown type name 'ggml_metal_kargs_mul_mm_id'
        constant ggml_metal_kargs_mul_mm_id & args,
                 ^
program_source:6407:36: error: use of undeclared identifier 'block_q4_0'
typedef decltype(kernel_get_rows_q<block_q4_0, 2, dequantize_q4_0>) get_rows_q_t;
                                   ^
program_source:6409:89: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_get_rows_q4_0")]]    kernel get_rows_q_t kernel_get_rows_q<block_q4_0,    2, dequantize_q4_0>;
                                                                                        ^
program_source:6410:89: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_get_rows_q4_1")]]    kernel get_rows_q_t kernel_get_rows_q<block_q4_1,    2, dequantize_q4_1>;
                                                                                        ^
program_source:6411:89: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_get_rows_q5_0")]]    kernel get_rows_q_t kernel_get_rows_q<block_q5_0,    2, dequantize_q5_0>;
                                                                                        ^
program_source:6412:89: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_get_rows_q5_1")]]    kernel get_rows_q_t kernel_get_rows_q<block_q5_1,    2, dequantize_q5_1>;
                                                                                        ^
program_source:6413:89: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_get_rows_q8_0")]]    kernel get_rows_q_t kernel_get_rows_q<block_q8_0,    2, dequantize_q8_0>;
                                                                                        ^
program_source:6414:89: error: use of undeclared identifier 'block_q2_K'
template [[host_name("kernel_get_rows_q2_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q2_K,    QK_NL, dequantize_q2_K>;
                                                                                        ^
program_source:6415:89: error: use of undeclared identifier 'block_q3_K'
template [[host_name("kernel_get_rows_q3_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q3_K,    QK_NL, dequantize_q3_K>;
                                                                                        ^
program_source:6416:89: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_get_rows_q4_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q4_K,    QK_NL, dequantize_q4_K>;
                                                                                        ^
program_source:6417:89: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_get_rows_q5_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q5_K,    QK_NL, dequantize_q5_K>;
                                                                                        ^
program_source:6418:89: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_get_rows_q6_K")]]    kernel get_rows_q_t kernel_get_rows_q<block_q6_K,    QK_NL, dequantize_q6_K>;
                                                                                        ^
program_source:6419:89: error: use of undeclared identifier 'block_iq2_xxs'
template [[host_name("kernel_get_rows_iq2_xxs")]] kernel get_rows_q_t kernel_get_rows_q<block_iq2_xxs, QK_NL, dequantize_iq2_xxs>;
                                                                                        ^
program_source:6420:89: error: use of undeclared identifier 'block_iq2_xs'
template [[host_name("kernel_get_rows_iq2_xs")]]  kernel get_rows_q_t kernel_get_rows_q<block_iq2_xs,  QK_NL, dequantize_iq2_xs>;
                                                                                        ^
program_source:6421:89: error: use of undeclared identifier 'block_iq3_xxs'
template [[host_name("kernel_get_rows_iq3_xxs")]] kernel get_rows_q_t kernel_get_rows_q<block_iq3_xxs, QK_NL, dequantize_iq3_xxs>;
                                                                                        ^
program_source:6422:89: error: use of undeclared identifier 'block_iq3_s'
template [[host_name("kernel_get_rows_iq3_s")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq3_s,   QK_NL, dequantize_iq3_s>;
                                                                                        ^
program_source:6423:89: error: use of undeclared identifier 'block_iq2_s'
template [[host_name("kernel_get_rows_iq2_s")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq2_s,   QK_NL, dequantize_iq2_s>;
                                                                                        ^
program_source:6424:89: error: use of undeclared identifier 'block_iq1_s'
template [[host_name("kernel_get_rows_iq1_s")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq1_s,   QK_NL, dequantize_iq1_s>;
                                                                                        ^
program_source:6425:89: error: use of undeclared identifier 'block_iq1_m'
template [[host_name("kernel_get_rows_iq1_m")]]   kernel get_rows_q_t kernel_get_rows_q<block_iq1_m,   QK_NL, dequantize_iq1_m>;
                                                                                        ^
program_source:6426:89: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_get_rows_iq4_nl")]]  kernel get_rows_q_t kernel_get_rows_q<block_iq4_nl,  2,     dequantize_iq4_nl>;
                                                                                        ^
program_source:6427:89: error: use of undeclared identifier 'block_iq4_xs'
template [[host_name("kernel_get_rows_iq4_xs")]]  kernel get_rows_q_t kernel_get_rows_q<block_iq4_xs,  QK_NL, dequantize_iq4_xs>;
                                                                                        ^
program_source:6433:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mm<half, half4x4, simdgroup_half8x8, float4x4, 1, dequantize_f32>) mat_mm_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:6052:13: note: possible target for call
kernel void kernel_mul_mm(
            ^
program_source:6435:69: error: explicit instantiation of 'kernel_mul_mm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_f32_f32")]]     kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   float4x4,      1,     dequantize_f32>;
                                                                    ^
program_source:6052:13: note: explicit instantiation refers here
kernel void kernel_mul_mm(
            ^
program_source:6436:69: error: explicit instantiation of 'kernel_mul_mm' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_f16_f32")]]     kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   half4x4,       1,     dequantize_f16>;
                                                                    ^
program_source:6052:13: note: explicit instantiation refers here
kernel void kernel_mul_mm(
            ^
program_source:6440:123: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mm_q4_0_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q4_0,    2,     dequantize_q4_0>;
                                                                                                                          ^
program_source:6441:123: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mm_q4_1_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q4_1,    2,     dequantize_q4_1>;
                                                                                                                          ^
program_source:6442:123: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mm_q5_0_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q5_0,    2,     dequantize_q5_0>;
                                                                                                                          ^
program_source:6443:123: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mm_q5_1_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q5_1,    2,     dequantize_q5_1>;
                                                                                                                          ^
program_source:6444:123: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mm_q8_0_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q8_0,    2,     dequantize_q8_0>;
                                                                                                                          ^
program_source:6445:123: error: use of undeclared identifier 'block_q2_K'
template [[host_name("kernel_mul_mm_q2_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q2_K,    QK_NL, dequantize_q2_K>;
                                                                                                                          ^
program_source:6446:123: error: use of undeclared identifier 'block_q3_K'
template [[host_name("kernel_mul_mm_q3_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q3_K,    QK_NL, dequantize_q3_K>;
                                                                                                                          ^
program_source:6447:123: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mm_q4_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q4_K,    QK_NL, dequantize_q4_K>;
                                                                                                                          ^
program_source:6448:123: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mm_q5_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q5_K,    QK_NL, dequantize_q5_K>;
                                                                                                                          ^
program_source:6449:123: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mm_q6_K_f32")]]    kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_q6_K,    QK_NL, dequantize_q6_K>;
                                                                                                                          ^
program_source:6450:123: error: use of undeclared identifier 'block_iq2_xxs'
template [[host_name("kernel_mul_mm_iq2_xxs_f32")]] kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq2_xxs, QK_NL, dequantize_iq2_xxs>;
                                                                                                                          ^
program_source:6451:123: error: use of undeclared identifier 'block_iq2_xs'
template [[host_name("kernel_mul_mm_iq2_xs_f32")]]  kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq2_xs,  QK_NL, dequantize_iq2_xs>;
                                                                                                                          ^
program_source:6452:123: error: use of undeclared identifier 'block_iq3_xxs'
template [[host_name("kernel_mul_mm_iq3_xxs_f32")]] kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq3_xxs, QK_NL, dequantize_iq3_xxs>;
                                                                                                                          ^
program_source:6453:123: error: use of undeclared identifier 'block_iq3_s'
template [[host_name("kernel_mul_mm_iq3_s_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq3_s,   QK_NL, dequantize_iq3_s>;
                                                                                                                          ^
program_source:6454:123: error: use of undeclared identifier 'block_iq2_s'
template [[host_name("kernel_mul_mm_iq2_s_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq2_s,   QK_NL, dequantize_iq2_s>;
                                                                                                                          ^
program_source:6455:123: error: use of undeclared identifier 'block_iq1_s'
template [[host_name("kernel_mul_mm_iq1_s_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq1_s,   QK_NL, dequantize_iq1_s>;
                                                                                                                          ^
program_source:6456:123: error: use of undeclared identifier 'block_iq1_m'
template [[host_name("kernel_mul_mm_iq1_m_f32")]]   kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq1_m,   QK_NL, dequantize_iq1_m>;
                                                                                                                          ^
program_source:6457:123: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mm_iq4_nl_f32")]]  kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq4_nl,  2,     dequantize_iq4_nl>;
                                                                                                                          ^
program_source:6458:123: error: use of undeclared identifier 'block_iq4_xs'
template [[host_name("kernel_mul_mm_iq4_xs_f32")]]  kernel mat_mm_t kernel_mul_mm<half,   half4x4,   simdgroup_half8x8,   block_iq4_xs,  QK_NL, dequantize_iq4_xs>;
                                                                                                                          ^
program_source:6464:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mm_id<float4x4, 1, dequantize_f32>) mat_mm_id_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:6334:13: note: possible target for call
kernel void kernel_mul_mm_id(
            ^
program_source:6466:75: error: explicit instantiation of 'kernel_mul_mm_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_id_f32_f32")]]     kernel mat_mm_id_t kernel_mul_mm_id<float4x4,      1,     dequantize_f32>;
                                                                          ^
program_source:6334:13: note: explicit instantiation refers here
kernel void kernel_mul_mm_id(
            ^
program_source:6467:75: error: explicit instantiation of 'kernel_mul_mm_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mm_id_f16_f32")]]     kernel mat_mm_id_t kernel_mul_mm_id<half4x4,       1,     dequantize_f16>;
                                                                          ^
program_source:6334:13: note: explicit instantiation refers here
kernel void kernel_mul_mm_id(
            ^
program_source:6471:92: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mm_id_q4_0_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q4_0,    2,     dequantize_q4_0>;
                                                                                           ^
program_source:6472:92: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mm_id_q4_1_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q4_1,    2,     dequantize_q4_1>;
                                                                                           ^
program_source:6473:92: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mm_id_q5_0_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q5_0,    2,     dequantize_q5_0>;
                                                                                           ^
program_source:6474:92: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mm_id_q5_1_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q5_1,    2,     dequantize_q5_1>;
                                                                                           ^
program_source:6475:92: error: use of undeclared identifier 'block_q8_0'
template [[host_name("kernel_mul_mm_id_q8_0_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q8_0,    2,     dequantize_q8_0>;
                                                                                           ^
program_source:6476:92: error: use of undeclared identifier 'block_q2_K'
template [[host_name("kernel_mul_mm_id_q2_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q2_K,    QK_NL, dequantize_q2_K>;
                                                                                           ^
program_source:6477:92: error: use of undeclared identifier 'block_q3_K'
template [[host_name("kernel_mul_mm_id_q3_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q3_K,    QK_NL, dequantize_q3_K>;
                                                                                           ^
program_source:6478:92: error: use of undeclared identifier 'block_q4_K'
template [[host_name("kernel_mul_mm_id_q4_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q4_K,    QK_NL, dequantize_q4_K>;
                                                                                           ^
program_source:6479:92: error: use of undeclared identifier 'block_q5_K'
template [[host_name("kernel_mul_mm_id_q5_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q5_K,    QK_NL, dequantize_q5_K>;
                                                                                           ^
program_source:6480:92: error: use of undeclared identifier 'block_q6_K'
template [[host_name("kernel_mul_mm_id_q6_K_f32")]]    kernel mat_mm_id_t kernel_mul_mm_id<block_q6_K,    QK_NL, dequantize_q6_K>;
                                                                                           ^
program_source:6481:92: error: use of undeclared identifier 'block_iq2_xxs'
template [[host_name("kernel_mul_mm_id_iq2_xxs_f32")]] kernel mat_mm_id_t kernel_mul_mm_id<block_iq2_xxs, QK_NL, dequantize_iq2_xxs>;
                                                                                           ^
program_source:6482:92: error: use of undeclared identifier 'block_iq2_xs'
template [[host_name("kernel_mul_mm_id_iq2_xs_f32")]]  kernel mat_mm_id_t kernel_mul_mm_id<block_iq2_xs,  QK_NL, dequantize_iq2_xs>;
                                                                                           ^
program_source:6483:92: error: use of undeclared identifier 'block_iq3_xxs'
template [[host_name("kernel_mul_mm_id_iq3_xxs_f32")]] kernel mat_mm_id_t kernel_mul_mm_id<block_iq3_xxs, QK_NL, dequantize_iq3_xxs>;
                                                                                           ^
program_source:6484:92: error: use of undeclared identifier 'block_iq3_s'
template [[host_name("kernel_mul_mm_id_iq3_s_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq3_s,   QK_NL, dequantize_iq3_s>;
                                                                                           ^
program_source:6485:92: error: use of undeclared identifier 'block_iq2_s'
template [[host_name("kernel_mul_mm_id_iq2_s_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq2_s,   QK_NL, dequantize_iq2_s>;
                                                                                           ^
program_source:6486:92: error: use of undeclared identifier 'block_iq1_s'
template [[host_name("kernel_mul_mm_id_iq1_s_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq1_s,   QK_NL, dequantize_iq1_s>;
                                                                                           ^
program_source:6487:92: error: use of undeclared identifier 'block_iq1_m'
template [[host_name("kernel_mul_mm_id_iq1_m_f32")]]   kernel mat_mm_id_t kernel_mul_mm_id<block_iq1_m,   QK_NL, dequantize_iq1_m>;
                                                                                           ^
program_source:6488:92: error: use of undeclared identifier 'block_iq4_nl'
template [[host_name("kernel_mul_mm_id_iq4_nl_f32")]]  kernel mat_mm_id_t kernel_mul_mm_id<block_iq4_nl,  2,     dequantize_iq4_nl>;
                                                                                           ^
program_source:6489:92: error: use of undeclared identifier 'block_iq4_xs'
template [[host_name("kernel_mul_mm_id_iq4_xs_f32")]]  kernel mat_mm_id_t kernel_mul_mm_id<block_iq4_xs,  QK_NL, dequantize_iq4_xs>;
                                                                                           ^
program_source:6496:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6504:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6515:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6529:9: error: unknown type name 'ggml_metal_kargs_mul_mv'
        ggml_metal_kargs_mul_mv args,
        ^
program_source:6541:70: error: use of undeclared identifier 'ggml_metal_kargs_mul_mv'
typedef decltype(mmv_fn<kernel_mul_mv_impl<half, half4, half, half4, ggml_metal_kargs_mul_mv>>) mul_mv_impl_fn_t;
                                                                     ^
program_source:6545:18: error: unknown type name 'ggml_metal_kargs_mul_mv_id'
        constant ggml_metal_kargs_mul_mv_id & args,
                 ^
program_source:6573:5: error: unknown type name 'ggml_metal_kargs_mul_mv'
    ggml_metal_kargs_mul_mv args0 = {
    ^
program_source:6606:18: error: reference to overloaded function could not be resolved; did you mean to call it?
typedef decltype(kernel_mul_mv_id<mmv_fn<kernel_mul_mv_impl<float, float4, float, float4>>>) kernel_mul_mv_id_t;
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:6544:13: note: possible target for call
kernel void kernel_mul_mv_id(
            ^
program_source:6608:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_f32_f32")]]     kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_impl<float, float4, float, float4>>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6609:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_f16_f32")]]     kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_impl<half, half4, float, float4>>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6613:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q8_0_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q8_0_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6614:127: error: use of undeclared identifier 'block_q4_0'
template [[host_name("kernel_mul_mv_id_q4_0_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q4_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6615:127: error: use of undeclared identifier 'block_q4_1'
template [[host_name("kernel_mul_mv_id_q4_1_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q4_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6616:127: error: use of undeclared identifier 'block_q5_0'
template [[host_name("kernel_mul_mv_id_q5_0_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q5_0, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6617:127: error: use of undeclared identifier 'block_q5_1'
template [[host_name("kernel_mul_mv_id_q5_1_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<mul_vec_q_n_f32_impl<block_q5_1, N_DST, N_SIMDGROUP, N_SIMDWIDTH>>>;
                                                                                                                              ^
program_source:6618:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q2_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q2_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6619:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q3_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q3_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6620:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q4_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q4_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6621:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q5_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q5_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6622:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_q6_K_f32")]]    kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_q6_K_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6623:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq1_s_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq1_s_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6624:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq1_m_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq1_m_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6625:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq2_xxs_f32")]] kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq2_xxs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6626:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq2_xs_f32")]]  kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq2_xs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6627:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq3_xxs_f32")]] kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq3_xxs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6628:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq3_s_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq3_s_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6629:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq2_s_f32")]]   kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq2_s_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6630:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq4_nl_f32")]]  kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq4_nl_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
program_source:6631:82: error: explicit instantiation of 'kernel_mul_mv_id' does not refer to a function template, variable template, member function, member class, or static data member
template [[host_name("kernel_mul_mv_id_iq4_xs_f32")]]  kernel kernel_mul_mv_id_t kernel_mul_mv_id<mmv_fn<kernel_mul_mv_iq4_xs_f32_impl>>;
                                                                                 ^
program_source:6544:13: note: explicit instantiation refers here
kernel void kernel_mul_mv_id(
            ^
}
ggml_backend_metal_device_init: error: failed to allocate context
whisper_backend_init_gpu: failed to initialize Metal backend
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size  =    6.29 MB
whisper_init_state: kv cross size =   18.87 MB
whisper_init_state: kv pad  size  =    3.15 MB
whisper_init_state: compute buffer (conv)   =   16.26 MB
whisper_init_state: compute buffer (encode) =   85.86 MB
whisper_init_state: compute buffer (cross)  =    4.65 MB
whisper_init_state: compute buffer (decode) =   96.35 MB

system_info: n_threads = 4 / 10 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | COREML = 0 | OPENVINO = 0 | 

main: processing 'samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...


[00:00:00.000 --> 00:00:11.000]   And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.


whisper_print_timings:     load time =   128.42 ms
whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:      mel time =     4.70 ms
whisper_print_timings:   sample time =    33.85 ms /   131 runs (    0.26 ms per run)
whisper_print_timings:   encode time =   431.44 ms /     1 runs (  431.44 ms per run)
whisper_print_timings:   decode time =     5.28 ms /     2 runs (    2.64 ms per run)
whisper_print_timings:   batchd time =   143.34 ms /   125 runs (    1.15 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:    total time =   972.15 ms

@danbev
Copy link
Collaborator

danbev commented Apr 17, 2025 •
edited
Loading

@7k50 Thanks for reporting this. I'm currently updating to Xcode 16.3 and I'll see if I can reproduce this.

Could you share the build commands that you used to build whisper.cpp?
And just to double check as this is a late reply, are you still seeing this error?

I've tried this out after upgrading to Xcode 16.3, but I've got an M3 so slightly different environment. I was not able to reproduce this directly, but I could force the same error that you are seeing by modifying the path to ggml-common.h to something that does not exist. This will build successfully but at runtime I get a similar error to what you are seeing:

whisper_backend_init_gpu: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M3
ggml_metal_init: picking default device: Apple M3
ggml_metal_load_library: using embedded metal library
ggml_metal_load_library: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:662:35: error: unknown type name 'block_q4_0'
void dequantize_q4_0(device const block_q4_0 * xb, short il, thread type4x4 & reg) {
                                  ^
program_source:681:38: error: unknown type name 'block_q4_0'
void dequantize_q4_0_t4(device const block_q4_0 * xb, short il, thread type4 & reg) {
...

But if this header is not there the build would fail so that does not seem like it could be the issue here. The same error would happen if the sed command is failing. Any chance you have a different version of sed that then system default?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@danbev @7k50