Skip to content

feat: add parent selector with connection management and capacity control #1161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

SouthWest7
Copy link
Contributor

@SouthWest7 SouthWest7 commented May 27, 2025

Description

This PR significantly enhances the parent selector functionality with robust connection management, reference counting, and capacity control mechanisms to improve reliability and resource utilization.

Related Issue

Tracing

Config

  • model flie: Qwen3-0.6B
  • parent1: upload_rate_limit = 20MiB
  • parent2: upload_rate_limit = 10MiB
  • hsot: download_rate_limit = 30MiB

origin

1749442663893_6AFDD465-6B4A-457e-8A9F-872DD93A0CB1

parent selector

1749442657264_11BF947D-4F51-4c54-8962-E130B1BDDDBD

Changes

New Features

  • Bandwidth-based Selection: Selects optimal parent peers based on available upload bandwidth using weighted random selection.
  • Dynamic Weight Calculation: Automatically calculates parent weights from real-time network statistics.
  • Reference Counting: Automatic connection lifecycle management using RAII pattern with ConnectionGuard.

API Design

Core Methods

// Create new ParentSelector with configuration
pub fn new(config: Arc<Config>, id_generator: Arc<IDGenerator>) -> Result<ParentSelector>

// Register multiple parents for monitoring
pub async fn register_parents(&self, parents: &[CollectedParent]) -> Result<()>

// Get connection to specific parent with reference counting
pub async fn get_connection(&self, parent: &CollectedParent) -> Result<(ConnectionGuard, DfdaemonUploadClient)>

// Select optimal parent based on bandwidth weights
pub fn select_parent(&self, parents: Vec<CollectedParent>) -> Result<CollectedParent>

// Remove parents and cleanup connections
pub async fn unregister_parents(&self, parents: Vec<Peer>) -> Result<()>

Motivation and Context

Screenshots (if appropriate)

Signed-off-by: southwest <1403572259@qq.com>
…at/parent-selector

Signed-off-by: southwest <1403572259@qq.com>
Signed-off-by: southwest <1403572259@qq.com>
Signed-off-by: southwest <1403572259@qq.com>
Signed-off-by: southwest <1403572259@qq.com>
Copy link

codecov bot commented May 27, 2025

Codecov Report

Attention: Patch coverage is 52.20418% with 206 lines in your changes missing coverage. Please review.

Project coverage is 47.92%. Comparing base (d899d99) to head (a64205a).

Files with missing lines Patch % Lines
dragonfly-client/src/resource/parent_selector.rs 59.84% 151 Missing ⚠️
dragonfly-client/src/resource/task.rs 0.00% 22 Missing ⚠️
dragonfly-client/src/resource/piece_collector.rs 0.00% 19 Missing ⚠️
dragonfly-client/src/bin/dfdaemon/main.rs 0.00% 8 Missing ⚠️
dragonfly-client/src/grpc/dfdaemon_upload.rs 0.00% 6 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1161      +/-   ##
==========================================
+ Coverage   47.81%   47.92%   +0.11%     
==========================================
  Files          60       61       +1     
  Lines       14207    14629     +422     
==========================================
+ Hits         6793     7011     +218     
- Misses       7414     7618     +204     
Files with missing lines Coverage Δ
dragonfly-client/src/grpc/dfdaemon_upload.rs 0.00% <0.00%> (ø)
dragonfly-client/src/bin/dfdaemon/main.rs 0.00% <0.00%> (ø)
dragonfly-client/src/resource/piece_collector.rs 0.00% <0.00%> (ø)
dragonfly-client/src/resource/task.rs 8.50% <0.00%> (-0.32%) ⬇️
dragonfly-client/src/resource/parent_selector.rs 59.84% <59.84%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gaius-qi gaius-qi added the enhancement New feature or request label May 27, 2025
Signed-off-by: southwest <1403572259@qq.com>
Signed-off-by: southwest <1403572259@qq.com>
Signed-off-by: southwest <1403572259@qq.com>
Signed-off-by: southwest <1403572259@qq.com>
Signed-off-by: southwest <1403572259@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants