-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Improve data chunk #5082
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Improve data chunk #5082
Conversation
This PR removes the hand-written Clone implementation for Chunk in favor of #[derive(Clone)], relying on Rust's more efficient automatic derivation for Arc-based structs. Additionally, it fixes a typo in the iterator type DataChunkIterator and adds #[derive(Clone)] for the internal MaterializedLogRecord struct. This summary was automatically generated by @propel-code-bot |
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks
Head branch was pushed to by a user without write access
@@ -122,7 +122,7 @@ impl ChromaError for LogMaterializerError { | |||
/// Instead of cloning or holding references to log records/segment data, this struct contains owned values that can be resolved to the referenced data. | |||
/// E.x. `final_document_at_log_index: Option<usize>` is used instead of `final_document: Option<&str>` to avoid holding references to the data. | |||
/// This allows `MaterializedLogRecord` (and types above it) to be trivially Send'able. | |||
#[derive(Debug)] | |||
#[derive(Debug, Clone)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw this was added after I hit approve. We generally don't expose Clone
on types that we don't intend to clone, as a defensive measure. Was there a strong reason for this change, would you be ok to remove it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The MaterializeLogsResult
struct contains a Chunk<MaterializedLogRecord>
field (on line 481).
Without this commit, tests were failing with the following error:
Compiling chroma-segment v0.1.0 (/Users/erik/Code/Rust/chroma/rust/segment)
error[E0277]: the trait bound `MaterializedLogRecord: Clone` is not satisfied
--> rust/segment/src/types.rs:481:5
|
478 | #[derive(Debug, Clone)]
| ----- in this derive macro expansion
...
481 | materialized: Chunk<MaterializedLogRecord>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Clone` is not implemented for `MaterializedLogRecord`
|
= note: required for `Chunk<MaterializedLogRecord>` to implement `Clone`
help: consider annotating `MaterializedLogRecord` with `#[derive(Clone)]`
|
126 + #[derive(Clone)]
127 | struct MaterializedLogRecord {
|
For more information about this error, try `rustc --explain E0277`.
error: could not compile `chroma-segment` (lib) due to 1 previous error
error: command `/opt/homebrew/bin/cargo test --no-run --message-format json-render-diagnostics` exited with code 101
This patch fixes that problem so the tests all pass. (Sorry for not running the tests locally before submitting my initial patch.)
Oh whoops! I missed that it was an implicit use through Chunk. Nevermind
…On Sat, Jul 12, 2025 at 1:08 AM Erik Berlin ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In rust/segment/src/types.rs
<#5082 (comment)>:
> @@ -122,7 +122,7 @@ impl ChromaError for LogMaterializerError {
/// Instead of cloning or holding references to log records/segment data, this struct contains owned values that can be resolved to the referenced data.
/// E.x. `final_document_at_log_index: Option<usize>` is used instead of `final_document: Option<&str>` to avoid holding references to the data.
/// This allows `MaterializedLogRecord` (and types above it) to be trivially Send'able.
-#[derive(Debug)]
+#[derive(Debug, Clone)]
The MaterializeLogsResult struct contains a Chunk<MaterializedLogRecord>
field (on line 481). For Rust to automatically implement Clone for
MaterializeLogsResult, all of its fields must implement Clone.
Without this commit, tests were failing with the following error:
Compiling chroma-segment v0.1.0 (/Users/erik/Code/Rust/chroma/rust/segment)
error[E0277]: the trait bound `MaterializedLogRecord: Clone` is not satisfied
--> rust/segment/src/types.rs:481:5
|
478 | #[derive(Debug, Clone)]
| ----- in this derive macro expansion
...
481 | materialized: Chunk<MaterializedLogRecord>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Clone` is not implemented for `MaterializedLogRecord`
|
= note: required for `Chunk<MaterializedLogRecord>` to implement `Clone`
help: consider annotating `MaterializedLogRecord` with `#[derive(Clone)]`
|
126 + #[derive(Clone)]
127 | struct MaterializedLogRecord {
|
For more information about this error, try `rustc --explain E0277`.
error: could not compile `chroma-segment` (lib) due to 1 previous error
error: command `/opt/homebrew/bin/cargo test --no-run --message-format json-render-diagnostics` exited with code 101
This patch fixes that problem so the tests all pass.
—
Reply to this email directly, view it on GitHub
<#5082 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKW32KXKFNYA3SY4JJWMQL3IC7BJAVCNFSM6AAAAACBJ7OUUSVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTAMJSHAZDEMBQGU>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Clone
forChunk<T>
with#[derive(Clone)]
so the struct relies on the automatic, efficient clone implementation provided forArc
typesDataChunkIterator