Parallel execution graph #10043
mailonghua
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
When llama.cpp executes in a pure CPU environment, it runs the graph in a single-threaded manner but executes operators in a multi-threaded manner. Why not expand it to run the graph in multiple threads? Would this lead to an efficiency improvement? Are there any performance risks that prevent this approach from being currently implemented?
Beta Was this translation helpful? Give feedback.
All reactions