Skip to content

Commit 8e23a08

Browse files
committed
updated readme, memory detection prints
1 parent 11c4e7c commit 8e23a08

File tree

2 files changed

+5
-9
lines changed

2 files changed

+5
-9
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models
1919
- Includes multiple modes (chat, adventure, instruct, storywriter) and UI Themes (aesthetic roleplay, classic writer, corporate assistant, messsenger)
2020
- Supports loading Tavern Character Cards, importing many different data formats from various sites, reading or exporting JSON savefiles and persistent stories.
2121
- Many other features including new samplers, regex support, websearch, RAG via TextDB and more.
22-
- Ready-to-use binaries for Windows, MacOS, Linux, Android (via Termux), Colab, Docker, also supports other platforms if self-compiled (like Raspberry PI).
22+
- Ready-to-use binaries for Windows, MacOS, Linux. Runs directly with Colab, Docker, also supports other platforms if self-compiled (like Android (via Termux) and Raspberry PI).
2323
- [Need help finding a model? Read this!](https://github.com/LostRuins/koboldcpp/wiki#getting-an-ai-model-file)
2424

2525
## Windows Usage (Precompiled Binary, Recommended)
@@ -31,7 +31,7 @@ KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models
3131
- You can also run it using the command line. For info, please check `koboldcpp.exe --help`
3232

3333
## Linux Usage (Precompiled Binary, Recommended)
34-
On modern Linux systems, you should download the `koboldcpp-linux-x64-cuda1150` prebuilt PyInstaller binary on the **[releases page](https://github.com/LostRuins/koboldcpp/releases/latest)**. Simply download and run the binary (You may have to `chmod +x` it first).
34+
On modern Linux systems, you should download the `koboldcpp-linux-x64-cuda1150` prebuilt PyInstaller binary for greatest compatibility on the **[releases page](https://github.com/LostRuins/koboldcpp/releases/latest)**. Simply download and run the binary (You may have to `chmod +x` it first). If you have a newer device, you can also try the `koboldcpp-linux-x64-cuda1210` instead for better speeds.
3535

3636
Alternatively, you can also install koboldcpp to the current directory by running the following terminal command:
3737
```

koboldcpp.py

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -978,7 +978,6 @@ def fetch_gpu_properties(testCL,testCU,testVK):
978978
FetchedCUdevices = []
979979
FetchedCUdeviceMem = []
980980
FetchedCUfreeMem = []
981-
faileddetectvram = False
982981

983982
AMDgpu = None
984983
try: # Get NVIDIA GPU names
@@ -989,7 +988,6 @@ def fetch_gpu_properties(testCL,testCU,testVK):
989988
except Exception:
990989
FetchedCUdeviceMem = []
991990
FetchedCUfreeMem = []
992-
faileddetectvram = True
993991
pass
994992
if len(FetchedCUdevices)==0:
995993
try: # Get AMD ROCm GPU names
@@ -1011,7 +1009,6 @@ def fetch_gpu_properties(testCL,testCU,testVK):
10111009
except Exception:
10121010
FetchedCUdeviceMem = []
10131011
FetchedCUfreeMem = []
1014-
faileddetectvram = True
10151012
pass
10161013
lowestcumem = 0
10171014
lowestfreecumem = 0
@@ -1030,14 +1027,13 @@ def fetch_gpu_properties(testCL,testCU,testVK):
10301027
except Exception:
10311028
lowestcumem = 0
10321029
lowestfreecumem = 0
1033-
faileddetectvram = True
1034-
1035-
if faileddetectvram:
1036-
print("Unable to detect VRAM, please set layers manually.")
10371030

10381031
MaxMemory[0] = max(lowestcumem,MaxMemory[0])
10391032
MaxFreeMemory[0] = max(lowestfreecumem,MaxFreeMemory[0])
10401033

1034+
if MaxMemory[0] < (1024*1024*256):
1035+
print("Unable to detect VRAM, please set layers manually.")
1036+
10411037
if testVK:
10421038
try: # Get Vulkan names
10431039
foundVkGPU = False

0 commit comments

Comments
 (0)