Read the text from a txt file as input

[Showwwwwwwww]

Hi,
I am currently working with llama.cpp on my macOS machine and attempting to use a sentence from another conversation as input continually for the llama model. Unfortunately, I have encountered difficulties in configuring the program to read the conversation text as input.

Could you possibly provide guidance or suggest a method that would allow me to successfully integrate the conversation text as input into the llama model?

[ianscrivener]

echo 'Please complete the following sentence:\n "Mary had a little lamb..."' > prompt.txt

./main \
  -t 8 \
  --model ./models/7B/llama-2-7b-chat.ggmlv3.q6_K.gguf \
  --file prompt.txt \
  --ctx-size 2048

[ianscrivener]

What about using llama-cpp-python? It is a python wrapper for llama.cpp
Easier (for most) than coding C++/C

[Showwwwwwwww]

Thanks for the information. I am trying to install it, however, I meet the error message for "FileNotFoundError: Shared library with base name 'llama' not found". Have quite confused about that.

[ianscrivener]

a few troubleshooting tips;
1) use python v3.10.12
2) force and download build llama-cpp-python;
eg CMAKE_ARGS="-DLLAMA_METAL=on" pip install -U llama-cpp-python --upgrade --no-cache-dir
CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" pip install llama-cpp-python --upgrade --no-cache-dir
3) Put the gguf model in the same directory as the python script

[Showwwwwwwww]

It still does nots works for me. It said that all the requirement already satisfied.

The error message still continue with:

Traceback (most recent call last):
File "", line 189, in _run_module_as_main
File "", line 112, in _get_module_details
File "/Users/chen/Desktop/robot/robot_research/llama2/llama-cpp-python/llama_cpp/init.py", line 1, in
from .llama_cpp import *
File "/Users/chen/Desktop/robot/robot_research/llama2/llama-cpp-python/llama_cpp/llama_cpp.py", line 80, in
_lib = _load_shared_library(_lib_base_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chen/Desktop/robot/robot_research/llama2/llama-cpp-python/llama_cpp/llama_cpp.py", line 71, in _load_shared_library
raise FileNotFoundError(
FileNotFoundError: Shared library with base name 'llama' not found

[ianscrivener]

As this is a llama-cpp-python issue you are better to post this on the llama-cpp-python Dicussion - they may have had this issue before.