This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Cortex.cpp: RAM & VRAM Management #1128

dan-menlo started this conversation in Architecture Specs

dan-menlo
Sep 6, 2024
Maintainer

Goal

Cortex.cpp should monitor free memory (e.g. RAM and VRAM)
Cortex.cpp should prevent user from ending up in OOM errors that hang their computer
e.g. OOM due to user loading model size > free memory

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment