ggml-org llama.cpp General · Discussions · GitHub

Sort by: Latest activity

General Discussions

Chat about anything and everything here

Pinned to General

You must be logged in to vote

QLoRA 4-bit quantization
research 🔬
unbounded started May 25, 2023 in General

5

Discussions

You must be logged in to vote

Mac M1/M2 Speed Optimization 🔥

xISSAx started Apr 12, 2023 in General

1
You must be logged in to vote

Local MoE Inference over the network - viable or dumb?

kalomaze started Dec 23, 2023 in General

1
You must be logged in to vote

Questions about order of MoE matmul for CPU/GPU inference

kalomaze started Dec 25, 2023 in General

6
You must be logged in to vote

Mixtral 8x7b

ronangrant started Dec 19, 2023 in General

2
You must be logged in to vote

PowerInfer

logikstate started Dec 19, 2023 in General

1
You must be logged in to vote

llama.cpp and lora - forcing 16 bit or a merge is kind of defeating the purpose of a separate lora file ?

cmp-nct started Dec 4, 2023 in General

2
You must be logged in to vote

We need CogVLM - extremely good image and text analysis, feels like a multi generational step forward.

cmp-nct started Dec 6, 2023 in General

9
You must be logged in to vote

make LLAMA_CUBLAS=1 - nvcc fatal : Value 'native' is not defined for option 'gpu-architecture'

FileDotZip started Jul 8, 2023 in General

8
You must be logged in to vote

It’s crazy in these days, I think AI base system will soon be available

FNsi started Dec 13, 2023 in General

0
You must be logged in to vote

openai api compatibility

iplayfast started Oct 19, 2023 in General

16
You must be logged in to vote

Look that model Mixtral-8x7B-Instruct-v0.1-GGUF is crazy.

mirek190 started Dec 11, 2023 in General

0
You must be logged in to vote

Merge Kompute from GPT4ALL

sorasoras started Dec 11, 2023 in General

0
You must be logged in to vote

Google just shipped libggml from llama-cpp into its Android AICore

sorasoras started Dec 10, 2023 in General

0
You must be logged in to vote

Mistral AI released the 8x7B MOE model

czkoko started Dec 8, 2023 in General · Closed

1
You must be logged in to vote

Server web UI features and direction

crasm started Dec 10, 2023 in General · Closed

0
You must be logged in to vote

KeyError: ('torch._utils', '_rebuild_parameter') when convert CodeShell-7B-Chat-int4

workmanihcj started Nov 29, 2023 in General

0
You must be logged in to vote

Measured Quantization probability differences for Mistral 7b and Llama 13b

kalomaze started Nov 23, 2023 in General

2
You must be logged in to vote

Making source files small to be GPT-4 friendly

aniljava started Nov 23, 2023 in General

1
You must be logged in to vote

My output is too low

paulocoutinhox started Nov 22, 2023 in General

0
You must be logged in to vote

Undefined symbols: gguf_get_val_data

paulocoutinhox started Nov 22, 2023 in General · Closed

2
You must be logged in to vote

How do I use the GPU?

deversbpv started Oct 7, 2023 in General

4
You must be logged in to vote

Perplexity / PPL, as a quantization loss benchmark, is inaccurate - KL divergence seem to be a better data point

kalomaze started Nov 17, 2023 in General

8
You must be logged in to vote

xTTS2 - quite great speech

cmp-nct started Nov 18, 2023 in General · Closed

1
You must be logged in to vote

We've a new king of the hill on visual llms since today (Sphinx-MLLM)
enhancement New feature or request
cmp-nct started Nov 16, 2023 in General

2
You must be logged in to vote

is the grammer based approach best approach to get structured data out of llama 7b ?

vishugupta96 started Nov 16, 2023 in General

1