Skip to content

Conversation

@ChenZiHong-Gavin
Copy link
Collaborator

@ChenZiHong-Gavin ChenZiHong-Gavin commented Oct 11, 2025

This PR refactors the partitioner and generator system by decoupling graph partitioning from QA generation and creating a more modular architecture. The changes introduce new base classes and implementations for both partitioners and generators, moving away from the previous community detection approach to a more flexible partitioning system.

Introduces new partitioner classes (BFS, DFS, ECE, Leiden) with unified interfaces
Creates generator classes for different QA generation modes (Atomic, Aggregated, CoT, MultiHop)
Refactors the main generation workflow to use separate partition and generate steps

Key renames / removals in the ECE partitioner (originally traverse strategy for traverse_graph.py):

  • Removed max_depth.
  • Removed expand_method – expansion is always BFS (or DFS, depending on the concrete class).
  • Removed loss_strategy – nodes and edges are treated as generic “units”; the single switch unit_sampling (“random”, “min_loss”, “max_loss”) decides the order in which they are visited.
  • Removed bidirectional – graphs are always traversed in both directions.
  • max_extra_edges → max_units_per_community (total nodes + edges allowed in one community).
  • max_tokens → max_tokens_per_community (community-level token budget).

When unit_sampling="random" the ECE partitioner degenerates to a plain BFS partitioner, guaranteeing backward compatibility for experiments that only need uniform random expansion.

@ChenZiHong-Gavin ChenZiHong-Gavin changed the title Partitioner Partitioner & Generator Oct 13, 2025
@ChenZiHong-Gavin ChenZiHong-Gavin changed the title Partitioner & Generator refactor: Partitioner & Generator Oct 13, 2025
@ChenZiHong-Gavin ChenZiHong-Gavin marked this pull request as ready for review October 13, 2025 08:49
@ChenZiHong-Gavin
Copy link
Collaborator Author

/gemini-review

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

github-actions[bot]

This comment was marked as outdated.

@ChenZiHong-Gavin ChenZiHong-Gavin marked this pull request as draft October 13, 2025 09:19
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 52 out of 59 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

ChenZiHong-Gavin and others added 2 commits October 14, 2025 17:19
…r.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 55 out of 60 changed files in this pull request and generated 4 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 59 out of 64 changed files in this pull request and generated 3 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@ChenZiHong-Gavin ChenZiHong-Gavin marked this pull request as ready for review October 15, 2025 04:24
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 59 out of 64 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

ChenZiHong-Gavin and others added 5 commits October 15, 2025 13:44
…r.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 58 out of 63 changed files in this pull request and generated 3 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 58 out of 63 changed files in this pull request and generated no new comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@ChenZiHong-Gavin ChenZiHong-Gavin mentioned this pull request Oct 15, 2025
30 tasks
@ChenZiHong-Gavin ChenZiHong-Gavin merged commit 3e3b0ff into main Oct 15, 2025
3 checks passed
@ChenZiHong-Gavin ChenZiHong-Gavin deleted the partitioner branch October 15, 2025 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants