Search results

metal: SSM_SCAN performance

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Status: Open (in progress).

ggml-org/llama.cppnumber 14743

#14743 In ggml-org/llama.cpp;

· gabe-l-hart opened on Jul 17, 2025

metal : reuse graphs

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

demoDemonstrate some concept or idea, not intended to be mergedDemonstrate some concept or idea, not intended to be merged

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Status: Draft (not ready).

ggml-org/llama.cppnumber 14570

#14570 In ggml-org/llama.cpp;

· ggerganov opened on Jul 7, 2025

llama : try loading tensors with pre-computed hashes

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Komputehttps://github.com/KomputeProject/kompute/https://github.com/KomputeProject/kompute/

Nvidia GPUIssues specific to Nvidia GPUsIssues specific to Nvidia GPUs

SYCLhttps://en.wikipedia.org/wiki/SYCL - GPU programming languagehttps://en.wikipedia.org/wiki/SYCL - GPU programming language

VulkanIssues specific to the Vulkan backendIssues specific to the Vulkan backend

Status: Open (in progress).

ggml-org/llama.cppnumber 13106

#13106 In ggml-org/llama.cpp;

· rgerganov opened on Apr 25, 2025

Metal TQ2_0

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Status: Open (in progress).

ggml-org/llama.cppnumber 12485

#12485 In ggml-org/llama.cpp;

· dmahurin opened on Mar 20, 2025

<code>tool-call</code>: Phi-4 support

androidIssues specific to AndroidIssues specific to Android

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

devopsimprovements to build systems and github actionsimprovements to build systems and github actions

documentationImprovements or additions to documentationImprovements or additions to documentation

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Nvidia GPUIssues specific to Nvidia GPUsIssues specific to Nvidia GPUs

pythonpython script changespython script changes

server

SYCLhttps://en.wikipedia.org/wiki/SYCL - GPU programming languagehttps://en.wikipedia.org/wiki/SYCL - GPU programming language

testingEverything test relatedEverything test related

VulkanIssues specific to the Vulkan backendIssues specific to the Vulkan backend

Status: Open (in progress).

ggml-org/llama.cppnumber 12288

#12288 In ggml-org/llama.cpp;

· jpohhhh opened on Mar 9, 2025

Attempt to add the <code>mllama</code> support

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Nvidia GPUIssues specific to Nvidia GPUsIssues specific to Nvidia GPUs

Status: Draft (not ready).

ggml-org/llama.cppnumber 11639

#11639 In ggml-org/llama.cpp;

· q82419 opened on Feb 4, 2025

ggml-cuda : add TQ2_0 kernels, for ternary inference on GPU

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

enhancementNew feature or requestNew feature or request

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Nvidia GPUIssues specific to Nvidia GPUsIssues specific to Nvidia GPUs

performanceSpeed related topicsSpeed related topics

pythonpython script changespython script changes

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

testingEverything test relatedEverything test related

Status: Open (in progress).

ggml-org/llama.cppnumber 11183

#11183 In ggml-org/llama.cpp;

· compilade opened on Jan 10, 2025

Add VisionOS compatibility by adding missing type definitions

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Status: Open (in progress).

ggml-org/llama.cppnumber 11019

#11019 In ggml-org/llama.cpp;

· sinkingsugar opened on Dec 30, 2024

Revert "ggml : remove OpenCL (#7735) + (#8235)"

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

buildCompilation issuesCompilation issues

devopsimprovements to build systems and github actionsimprovements to build systems and github actions

documentationImprovements or additions to documentationImprovements or additions to documentation

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

nixIssues specific to consuming flake.nix, or generally concerned with ❄ Nix-based llama.cpp deploymentIssues specific to consuming flake.nix, or generally concerned with ❄ Nix-based llama.cpp deployment

pythonpython script changespython script changes

scriptScript relatedScript related

SYCLhttps://en.wikipedia.org/wiki/SYCL - GPU programming languagehttps://en.wikipedia.org/wiki/SYCL - GPU programming language

Status: Draft (not ready).

ggml-org/llama.cppnumber 8986

#8986 In ggml-org/llama.cpp;

· okias opened on Aug 11, 2024

Added support to select GPU using metal on Apple Intel or Apple Silicon using --main-gpu index

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Review Complexity : LowTrivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fixTrivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix

Status: Open (in progress).

ggml-org/llama.cppnumber 8962

#8962 In ggml-org/llama.cpp;

· ifeanyipossibilities opened on Aug 10, 2024

Rebalancing Metal threads workload in dot product kernel kernel_mul_mv_f16_f32_l4

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

Review Complexity : MediumGenerally require more time to grok but manageable by beginner to medium expertise levelGenerally require more time to grok but manageable by beginner to medium expertise level

Status: Open (in progress).

ggml-org/llama.cppnumber 7522

#7522 In ggml-org/llama.cpp;

· izard opened on May 24, 2024