Skip to content

Commit 680722d

Browse files
authored
chore: time sensitive throttle for scan command (#4954)
The issue is that Scan has high latency (in seconds) when keys are large and there are no matches. Iterating 10k buckets and string matching each of the keys is potentially an expensive operation that depends on the keyspace and the number of actual matches. * replace heuristic in scan command to throttle based on time Signed-off-by: kostas <kostas@dragonflydb.io>
1 parent c06e154 commit 680722d

File tree

1 file changed

+5
-4
lines changed

1 file changed

+5
-4
lines changed

src/server/generic_family.cc

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -644,13 +644,14 @@ void OpScan(const OpArgs& op_args, const ScanOpts& scan_opts, uint64_t* cursor,
644644
PrimeTable::Cursor cur = *cursor;
645645
auto [prime_table, expire_table] = db_slice.GetTables(op_args.db_cntx.db_index);
646646

647-
size_t buckets_iterated = 0;
648-
// 10k Traverses
649-
const size_t limit = 10000;
647+
const auto start = absl::Now();
648+
// Don't allow it to monopolize cpu time.
649+
const absl::Duration timeout = absl::Milliseconds(10);
650+
650651
do {
651652
cur = prime_table->Traverse(
652653
cur, [&](PrimeIterator it) { cnt += ScanCb(op_args, it, scan_opts, vec); });
653-
} while (cur && cnt < scan_opts.limit && buckets_iterated++ < limit);
654+
} while (cur && cnt < scan_opts.limit && (absl::Now() - start) < timeout);
654655

655656
VLOG(1) << "OpScan " << db_slice.shard_id() << " cursor: " << cur.value();
656657
*cursor = cur.value();

0 commit comments

Comments
 (0)