Skip to content

Implement of searchv2 to fetch batch of blocks #35

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 29, 2025

Conversation

mike-petrov
Copy link
Contributor

closes #34

Signed-off-by: Mikhail Petrov mike@nspcc.ru

@mike-petrov mike-petrov self-assigned this May 13, 2025
@mike-petrov mike-petrov requested a review from roman-khimov as a code owner May 13, 2025 16:20
@mike-petrov mike-petrov changed the title Implement of searchv2 to fetch batch of blocks, Implement of searchv2 to fetch batch of blocks May 13, 2025
@mike-petrov mike-petrov force-pushed the feature/34-implement_searchv2 branch from 37400df to 10d4276 Compare May 13, 2025 16:24
@mike-petrov
Copy link
Contributor Author

While tetsing, I found an interesting case, some blocks have a duplicate:

image

@roman-khimov
Copy link
Member

It's known, there are duplicates. Code should handle this.

@mike-petrov mike-petrov force-pushed the feature/34-implement_searchv2 branch from 3e66a42 to bc9c3f0 Compare May 20, 2025 22:09
@mike-petrov
Copy link
Contributor Author

@roman-khimov everything works now, asynchronously, but because of duplicates I can't understand from which number I should request a search:

i.e. if I specify the interval 1000-2700, then we take first 1000 objects with 50 duplicates, and the second search request will start from 2000, while it should start from 1950, how to solve this situation?

@roman-khimov
Copy link
Member

By using the answer you have from the first request? What's the problem, really? You know the last block-oid pair after every request, you don't need to have perfect 1000 blocks.

@mike-petrov
Copy link
Contributor Author

Yes, I have 1000 blocks every time, but if there is a duplicate among them, I will not know about it at the asynchronous search stage and will skip it:

  • first searchv2 (get 0-1000 blocks)
  • iterating 0-500 block (get data)
  • second searchv2 (get 1000-2000 blocks)
  • iterating 500-1000 block (get data, but the 800 number is a duplicate, so we're skipping it)

At this point we missed block number 1000 because of the duplicate

  • iterating 1000-1500 block (get data)
    ...

@roman-khimov
Copy link
Member

second searchv2 (get 1000-2000 blocks)

get 995-1990, what's the problem? You know where to start from right after the first search.

@mike-petrov
Copy link
Contributor Author

If I immediately request the last block, then yes - I'll know which number is the last, is that what we want to do?

@mike-petrov mike-petrov force-pushed the feature/34-implement_searchv2 branch 2 times, most recently from df00c70 to 00bba73 Compare May 21, 2025 16:48
@mike-petrov
Copy link
Contributor Author

ok, now it works

@mike-petrov mike-petrov requested a review from roman-khimov May 21, 2025 16:49
Signed-off-by: Mikhail Petrov <mike@nspcc.ru>
@mike-petrov mike-petrov force-pushed the feature/34-implement_searchv2 branch from f453dfb to 3a07ad6 Compare May 26, 2025 14:26
@mike-petrov mike-petrov requested a review from roman-khimov May 26, 2025 14:30
@roman-khimov roman-khimov merged commit fa658e0 into master May 29, 2025
2 checks passed
@roman-khimov roman-khimov deleted the feature/34-implement_searchv2 branch May 29, 2025 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to use searchv2
2 participants