Alternative B: BM25 only | idea_world

Flow

raw chunkText
  -> tokenizer
  -> BM25
  -> return top JSONL

What BM25 Sees

BM25 roughly creates a score from the following elements.

TF: How many times does a word appear in this document  
IDF: How rare this word is in the entire corpus  
Length normalization: Adjust so that a long document does not automatically have an advantage

So the next words of the reference chunk become important.

animatedsprite2d
is_action_pressed
move_left
move_right
normalized
clamp
screen_size
vector2
zero

Reason for success in the standard chunk

In the simulation of the original memo, the following document appears above.

outputs/godot_docs_full/pages/getting_started__first_2d_game__03.coding_the_player__161d377b.md

The directly matching token is as follows.

animatedsprite2d
is_action_pressed
move_left
move_right
normalized
play
stop
input
speed
length
velocity
vector2
zero

Especially, the following tokens are relatively rare in the corpus, so they contribute strongly to the score.

animatedsprite2d
move_left
move_right
is_action_pressed
screen_size
clamp

Failure Candidates

In BM25 only, 3D movement documents can also appear.

The reason is due to the following common token.

Input.is_action_pressed
move_right
move_left
normalized
speed

But the reference chunk is 2D, and the following clue is important.

AnimatedSprite2D
Vector2
position.clamp
screen_size

BM25 is based on word scores, so it cannot fully understand 2D/3D context.

PoC Simulation

Insert the reference chunk as a BM25 query.

Input:

func _process(delta):
  Input.is_action_pressed(...)
  velocity.normalized() * speed
  $AnimatedSprite2D.play()
  position.clamp(Vector2.ZERO, screen_size)

BM25 tokens that give a high score:

animatedsprite2d
is_action_pressed
move_left
move_right
normalized
clamp
screen_size
vector2
zero

Predicted Candidates:

Rank	Candidate	Why It Appears	Judgment
1	`first_2d_game / coding_the_player`	`Input.is_action_pressed`, `move_left`, `move_right`, `velocity.normalized`, `AnimatedSprite2D.play/stop` match directly	accept
2	`first_2d_game / clamp/screen section`	`position.clamp`, `Vector2.ZERO`, `screen_size`, `AnimatedSprite2D` match	accept
3	`first_3d_game / player_movement_code`	`Input.is_action_pressed`, `move_left`, `move_right`, `normalized`, `speed` overlap	false positive

Reason why the #1 candidate appears:

animatedsprite2d:
  Since it is a rare token throughout the document, its IDF is large

move_left / move_right:
  Appears directly in the tutorial input code

is_action_pressed:
  Directly matches the input handling code

velocity.normalized:
  Directly matches the flow of normalizing the movement vector

Reason why the third-place candidate appears together:

3D movement documentation also explains input handling and normalized movement flow.  
Therefore, from BM25's perspective, the query token overlap is quite large.  
However, it does not match the core of the reference chunk, AnimatedSprite2D, Vector2, screen_size, position.clamp.

Logs to Verify in PoC

In the BM25 PoC, it is important to show the token contribution for each candidate.

Example:

candidate: first_2d_game / coding_the_player
matched_terms:
  animatedsprite2d   high
  is_action_pressed  high
  move_left          high
  move_right         high
  normalized         medium
  play               medium
  stop               medium

candidate: first_3d_game / player_movement_code
matched_terms:
  is_action_pressed  high
  move_left          high
  move_right         high
  normalized         medium
missing_terms:
  animatedsprite2d
  screen_size
  position.clamp

Visible conclusion:

BM25 retrieves relevant documents well.  
However, it also brings up similar input/movement code.  
Therefore, a reranker or validator is needed after the BM25 stage.

Advantages

The principle is transparent.
Debugging is easy.
There is no model cost.
You can use the raw chunk as is.
Strong at exact string search in code/API.

Disadvantages

Weak with synonyms and descriptive sentences.
May miss it if the API name does not appear directly.
Similar code like 3D movement can be mixed in as false positives.

Judgment

Primary candidate search that must be reviewed  
However, using it alone is insufficient