Is it a specification that the response is very slow?

[RRX-JP]

It takes about 20 seconds from the time a character is sent to the time it is displayed.
Also, VRAM increases in GB for each sent.

mode: instruct

start_windows.bat
@echo off
powershell -Command "Start-Process powershell -ArgumentList '.\qdrant.exe' -WorkingDirectory '.\extensions\Memoir' -NoNewWindow"
setlocal enabledelayedexpansion
:
:

logs.
main: server is listening on http://127.0.0.1:55493 - starting the main loop 03:20:03-231406 INFO Loaded "umiyuki-Umievo-itr012-Gleipnir-7B-Q2_K.gguf" in 2.53 seconds. 03:20:03-233544 INFO LOADER: "llama.cpp" 03:20:03-234598 INFO TRUNCATION LENGTH: 8192 03:20:03-235627 INFO INSTRUCTION TEMPLATE: "Alpaca" COMMANDS:[] 2025-06-17T18:20:21.773440Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI 2025-06-17T18:20:21.774022Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI HTTP/1.1" 409 93 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000963 2025-06-17T18:20:24.082723Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.002733 RAG LIMIT:2 2025-06-17T18:20:30.590755Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI_rag_data 2025-06-17T18:20:30.593007Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI_rag_data HTTP/1.1" 409 98 "-" "qdrant-client/1.14.2 python/3.11.13" 0.002835 2025-06-17T18:20:32.661884Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI_rag_data/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000882 --------current context string input modifier--------------- hi --------/end context string input modifier--------------- 2025-06-17T18:20:39.381000Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI 2025-06-17T18:20:39.381475Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI HTTP/1.1" 409 93 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000612 prompt processing progress, n_past = 39, n_tokens = 39, progress = 1.000000 03:20:39-989989 INFO Output generated in 0.60 seconds (51.95 tokens/s, 31 tokens, context 39, seed 2136099510) COMMANDS:[] 2025-06-17T18:20:46.581138Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI 2025-06-17T18:20:46.582869Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI HTTP/1.1" 409 93 "-" "qdrant-client/1.14.2 python/3.11.13" 0.002006 2025-06-17T18:20:48.667763Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000958 RAG LIMIT:2 2025-06-17T18:20:55.179669Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI_rag_data 2025-06-17T18:20:55.182315Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI_rag_data HTTP/1.1" 409 98 "-" "qdrant-client/1.14.2 python/3.11.13" 0.002989 2025-06-17T18:20:57.257414Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI_rag_data/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.004607 COMMANDS:[] 2025-06-17T18:21:05.560383Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI 2025-06-17T18:21:05.560906Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI HTTP/1.1" 409 93 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000642 2025-06-17T18:21:07.635222Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.001444 RAG LIMIT:2 2025-06-17T18:21:14.242869Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI_rag_data 2025-06-17T18:21:14.243367Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI_rag_data HTTP/1.1" 409 97 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000659 2025-06-17T18:21:16.310335Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI_rag_data/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000880 --------current context string input modifier--------------- help ok? --------/end context string input modifier--------------- 2025-06-17T18:21:22.988745Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI 2025-06-17T18:21:22.989752Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI HTTP/1.1" 409 93 "-" "qdrant-client/1.14.2 python/3.11.13" 0.001191 --------------------------------------Enough memories for a dream... 2025-06-17T18:21:25.090147Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.001269 prompt processing progress, n_past = 279, n_tokens = 22, progress = 0.9964163 prompt eval time = 223.95 ms / 278 tokens ( 0.81 ms per token, 1241.37 tokens per second) eval time = 6253.45 ms / 331 tokens ( 18.89 ms per token, 52.93 tokens per second) total time = 6477.40 ms / 609 tokens 03:21:31-575535 INFO Output generated in 6.48 seconds (50.93 tokens/s, 330 tokens, context 279, seed 318987839) Summary passed checking 2025-06-17T18:21:34.393525Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI/points?wait=true HTTP/1.1" 200 81 "-" "qdrant-client/1.14.2 python/3.11.13" 0.331775 03:21:34-896484 INFO Output generated in 0.47 seconds (48.44 tokens/s, 23 tokens, context 86, seed 1267065335) COMMANDS:[] 2025-06-17T18:21:41.355411Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI 2025-06-17T18:21:41.355747Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI HTTP/1.1" 409 93 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000405 2025-06-17T18:21:43.441719Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI/points/search HTTP/1.1" 200 817 "-" "qdrant-client/1.14.2 python/3.11.13" 0.037758 RAG LIMIT:2 2025-06-17T18:21:49.927720Z INFO storage::content_manager::toc::collection_meta_ops: Creating collection AI_rag_data 2025-06-17T18:21:49.928021Z INFO actix_web::middleware::logger: 127.0.0.1 "PUT /collections/AI_rag_data HTTP/1.1" 409 97 "-" "qdrant-client/1.14.2 python/3.11.13" 0.000389 2025-06-17T18:21:52.008739Z INFO actix_web::middleware::logger: 127.0.0.1 "POST /collections/AI_rag_data/points/search HTTP/1.1" 200 53 "-" "qdrant-client/1.14.2 python/3.11.13" 0.002933

[brucepro]

When the memories are being formed you may see some increased lag. You can also disable the rag system and see if it helps with the speed. I need to do some logging and determine if the code has any bottlenecks in the qdrant lookup.

[RRX-JP]

Turning it OFF does not change the time.
Debugging parameters with RAG System OFF.

fulllog.txt