Replace FLAN-Stage 2 with SBERT-Only Flow #2
Description
Summary
We are planning to remove the FLAN model used in Stage 2 (which selects the exact field from SBERT's top fields). Instead, we will directly pass SBERT’s top fields to the FLAN Stage 3 (query generator), allowing it to generate queries by choosing the right field(s) on its own.
Objective
Improve efficiency by eliminating one stage while testing whether the final FLAN model can handle:
- Selecting the correct field(s) from a list of top fields
- Generating correct queries for both single and multi-field scenarios
- Supporting simple, moderate, and complex question types
Acceptance Criteria
- Modify pipeline to bypass FLAN Stage 2
- Feed top fields from SBERT directly into FLAN Stage 3
- Evaluate results across a wide variety of questions (1-field, 2-field, multi-field)
- Validate field selection accuracy and query structure integrity
Labels
refactor, enhancement, architecture, experiment