Alternative F: BM25 + embedding + reranker + validator

Flow

raw chunkText
  -> BM25 top 80
  -> embedding top 80
  -> candidate union
  -> reranker directly compares the raw chunk with each JSONL candidate
  -> returns top JSONL
  -> Qwen direct-evidence validator performs direct evidence verification

Role Assignment

BM25:

Find the exact string/API token candidate.

code embedding:

It supplements candidates that have different expressions but similar meanings.

reranker:

BM25 and embedding compare the candidates they bring with the raw chunk and reorder them.

Qwen direct-evidence validator:

Check whether there is a string/pattern evidence in the JSONL that directly matches the current chunk.  
Discard JSONL without evidence.

Expected Alignment in the Standard Chunk

Candidates that should be ranked high:

first_2d_game / coding_the_player
same page / clamp and AnimatedSprite2D explanation

Candidates that should be lowered:

first_3d_game / player_movement_code
unrelated api_mapping
unrelated label_prototypes

PoC Simulation

Assume that the reference chunk is inserted into the final recommendation flow.

raw chunkText
  -> BM25 top 80
  -> voyage-code-3 embedding top 80
  -> candidate union
  -> reranker
  -> Qwen direct-evidence validator

Step 1: BM25 Candidates

Candidates retrieved by BM25:

A. first_2d_game / coding_the_player
   reason: Input.is_action_pressed, move_left, move_right, AnimatedSprite2D.play/stop

B. first_2d_game / clamp section
   reason: position.clamp, Vector2.ZERO, screen_size

C. first_3d_game / player_movement_code
   reason: Input.is_action_pressed, move_left, move_right, normalized

Step 2: embedding candidates

Candidates that embedding will bring:

D. first_2d_game / movement explanation
   reason: 2D movement, keyboard input, animation

E. first_2d_game / screen bounds explanation
   reason: keep player inside screen

F. first_3d_game / movement
   reason: player movement semantic similarity

Step 3: Candidate union

After the union, combine duplicates.

A/D -> first_2d_game / coding_the_player
B/E -> first_2d_game / clamp/screen bounds
C/F -> first_3d_game / player_movement

Step 4: Reranker Reordering

The reranker looks at the raw chunk together with the candidates.

Expected rerank:

rerank	Candidate	Reason
1	`first_2d_game / coding_the_player`	Input handling, velocity, AnimatedSprite2D play/stop match directly
2	`first_2d_game / clamp/screen bounds`	`position.clamp`, `Vector2.ZERO`, `screen_size` match directly
3	`first_3d_game / player_movement`	Input/movement is similar but it is a 3D context and lacks AnimatedSprite2D/screen_size

Step 5: Qwen Direct-Evidence Validator

The Qwen validator is asked as follows.

Based on the SOURCE_CODE and the retrieved JSONL,  
Is there a string/API call/pattern in the JSONL that directly matches the SOURCE_CODE?  
First, make a judgment with “yes” or “no”.

I’m unable to translate the Markdown because the fragment to be translated wasn’t included in your message. Please provide the Markdown content you’d like me to translate.

first_2d_game / coding_the_player:
  validator = yes
  direct evidence = Input.is_action_pressed, AnimatedSprite2D.play, AnimatedSprite2D.stop

first_2d_game / clamp/screen bounds:
  validator = yes
  direct evidence = position.clamp, Vector2.ZERO, screen_size

first_3d_game / player_movement:
  validator = no or low relevance
  reason = movement input is similar but it is a 3D document and there is no direct evidence from AnimatedSprite2D/screen_size

Logs to Check in PoC

This alternative has many steps, so you need to view the following on a single screen.

1. raw chunkText  
2. BM25 candidates and matched terms  
3. embedding candidates and similarity  
4. union result  
5. reranker score and rank change  
6. Qwen validator yes/no  
7. final accept/reject JSONL

Visible conclusion:

BM25 and embedding gather candidates broadly.  
The reranker corrects the order of similar candidates.  
The Qwen validator discards JSONL without direct evidence as final.

Advantages

It is likely to have the best quality.
Can reduce BM25 false positives.
Can also reduce embedding false positives.
Maintains the raw chunk condition.
Has low hard‑coding dependency.
Can remove unsupported JSONL at the end with a Qwen validator.

Disadvantages

It incurs cost.
Latency is introduced.
It becomes slow if too many candidates are added.
Since the reranker is not a justification verifier, a final validator is required.

Judgment

Quality First Final Recommendation

The final recommendation of the original note is close to this structure.