Skip to content

Conversation

@b4rtaz
Copy link
Owner

@b4rtaz b4rtaz commented Aug 14, 2025

1x RTX 3060 Ti 8 GB

🌋 Device: NVIDIA GeForce RTX 3060 Ti
🌋 DeviceApiVersion: 1.3.242
🌋 MaxComputeSharedMemory: 48 kB
🌋 NonCoherentAtomSize: 64 bytes
🌋 Heap[0]: 8192 MB
🌋 Heap[2]: 246 MB
Model Tokens/s (version 0.15.0) Tokens/s (version 0.15.2) Tokens/s (This PR)
lama3_1_8b_instruct_q40 13.46 15.96 17.91

4x RTX 3090 24 GB

🌋 Device: NVIDIA GeForce RTX 3090
🌋 DeviceApiVersion: 1.4.303
🌋 MaxComputeSharedMemory: 48 kB
🌋 NonCoherentAtomSize: 64 bytes
🌋 Heap[0]: 24576 MB
🌋 Heap[2]: 246 MB
Model Tokens/s (version 0.15.0) Tokens/s (version 0.15.2) Tokens/s (This PR)
llama3_3_70b_instruct_q40 - 4.22 4.35

@b4rtaz b4rtaz merged commit e35a2f3 into main Aug 16, 2025
3 checks passed
@b4rtaz b4rtaz deleted the feat/vulkan-optimization-2 branch August 18, 2025 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants