Skip to content

Vector-db-proxy queries same datasource 2x for every row #600

@tomlynchRNA

Description

@tomlynchRNA

If you look at the logs, you will see 2x as many prints as rows. E.g. when syncing a table with 1 row:

Datasource retrieved from Mongo: 66fcd0619953c0a0f29d8663
Datasource retrieved from Mongo: 66fcd0619953c0a0f29d8663

First one is from:
rabbit_consume -> process_message -> get_model

Second is from:
process_incoming_messages -> handle_embedding -> embed_text_construct_point -> embed_text -> get_model

Full backtraces after each print:
Datasource retrieved from Mongo: 66fcd0619953c0a0f29d8663
backtrace:    0: vector_db_proxy::adaptors::mongo::queries::get_model::{{closure}}
             at ./src/adaptors/mongo/queries.rs:41:39
   1: vector_db_proxy::messages::tasks::process_message::{{closure}}
             at ./src/messages/tasks.rs:67:67
   2: vector_db_proxy::adaptors::rabbitmq::models::rabbit_consume::{{closure}}
             at ./src/adaptors/rabbitmq/models.rs:105:38
   3: <vector_db_proxy::messages::models::QueueConnectionTypes as vector_db_proxy::messages::models::MessageQueue>::consume::{{closure}}
             at ./src/messages/models.rs:61:88
   4: vector_db_proxy::main::{{closure}}::{{closure}}
             at ./src/main.rs:128:14
   5: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /usr/src/debug/rust/rustc-1.81.0-src/library/core/src/future/future.rs:123:9
... removed non useful stack info

Datasource retrieved from Mongo: 66fcd0619953c0a0f29d8663
backtrace:    0: vector_db_proxy::adaptors::mongo::queries::get_model::{{closure}}
             at ./src/adaptors/mongo/queries.rs:41:39
   1: vector_db_proxy::embeddings::utils::embed_text::{{closure}}
             at ./src/embeddings/utils.rs:156:78
   2: vector_db_proxy::data::processing_incoming_messages::embed_text_construct_point::{{closure}}
             at ./src/data/processing_incoming_messages.rs:82:93
   3: vector_db_proxy::data::processing_incoming_messages::handle_embedding::{{closure}}
             at ./src/data/processing_incoming_messages.rs:132:6
   4: vector_db_proxy::data::processing_incoming_messages::process_incoming_messages::{{closure}}::{{closure}}
             at ./src/data/processing_incoming_messages.rs:240:42
   5: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /usr/src/debug/rust/rustc-1.81.0-src/library/core/src/future/future.rs:123:9
   6: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
... removed non useful stack info

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions