idea_world_labDEV JOURNAL
Saturday, June 27, 2026

Line Tracking Flow from Source Input to AST and Retriever

Date: June 27, 2026

Purpose

This document reorganizes, as of the 27th, the parts where consistency was lost in docs/roadmaps/2026-06-26-source-to-ast-input-flow.md.

The core is not to redefine the new design unit. The overall architecture and DB design direction have already been documented separately. Here we trace, to the end, which line of which file goes into the AST Parser when actual input arrives, which fragment is used for Retriever search, and which evidence candidates return and lead to LLM judgment.

The principle to fix is simple.

File paths are attached to track locations.  
The Retriever contains code/text fragments, not file paths.  
.gd files are cut by the AST Parser into function and declaration units.  
Other required text files are directly cut by line range.  
Each fragment sequentially goes through the Retriever and LLM judgment.

Input Form

When you clone or upload a project, the files are first expanded with a relative path.

The format looks like this.

# <relative path>
<file content>

# <relative path>
<file content>

This notation is for human debugging. A header like # player.gd is only a marker indicating “which file the following content comes from”.

The # player.gd does not appear in Retriever searches. It also does not appear in the AST Parser. What is included is the actual file content under the header.

Example Input

Below is a small Godot project unfolded. Numbers like E001, E002 are line numbers for explanation and are not included in the actual input characters.

E001 # project.godot
E002 config_version=5
E003
E004 [application]
E005 config/name="Mini Dodge"
E006 run/main_scene="res://player.tscn"
E007
E008 # player.gd
E009 extends Area2D
E010
E011 signal hit
E012
E013 @export var speed = 400
E014 var screen_size
E015
E016 func _ready():
E017 	screen_size = get_viewport_rect().size
E018 	hide()
E019
E020 func _process(delta):
E021 	var velocity = Vector2.ZERO
E022 	if Input.is_action_pressed(&"move_right"):
E023 		velocity.x += 1
E024 	if Input.is_action_pressed(&"move_left"):
E025 		velocity.x -= 1
E026
E027 	if velocity.length() > 0:
E028 		velocity = velocity.normalized() * speed
E029 		$AnimatedSprite2D.play()
E030 	else:
E031 		$AnimatedSprite2D.stop()
E032
E033 	position += velocity * delta
E034 	position = position.clamp(Vector2.ZERO, screen_size)
E035
E036 func start(pos):
E037 	position = pos
E038 	show()
E039 	$CollisionShape2D.disabled = false
E040
E041 # player.tscn
E042 [gd_scene load_steps=3 format=3 uid="uid://mini"]
E043
E044 [ext_resource type="Script" path="res://player.gd" id="1"]
E045
E046 [node name="Player" type="Area2D"]
E047 script = ExtResource("1")

Step 1: Split File Boundaries

First, split the files based on the # <relative path> header.

project.godot starts at E001. The actual content is from E002 to E006. E001 is a file boundary marker, so it is not part of the analysis content.

player.gd starts at E008. The actual content is from E009 to E039. E008 is a file boundary marker, so it does not go into the AST Parser.

player.tscn starts at E041. The actual content is from E042 to E047. E041 is a file boundary marker, so it is not part of the Retriever search content.

At this stage, the UI can show something like “Processing player.gd E009‑E039”. However, a search term is not created yet.

Step 2: Decide Processing Direction by File Type

player.gd is a GDScript file. Therefore, the entire range E009‑E039 goes to the AST Parser.

project.godot is a configuration file. It is not sent to the AST Parser; instead, it is sliced directly into needed sections.

player.tscn is a scene file. It is not sent to the AST Parser; instead, it is sliced directly by node/resource block units.

README or Markdown documentation files are excluded by default in source‑code analysis mode. Excluded files must still be shown in the UI because a human needs to see “what is missing”.

For example, if there is a README.md, it is excluded from analysis, and the reason displayed should be “Documentation files are excluded by default in source‑code analysis mode”. This exclusion decision is only the default for analysis mode; if a mode that reads documentation later is needed, it will be handled separately.

Step 3: Pass .gd Files to the AST Parser

The input that goes from player.gd to the AST Parser is E009‑E039.

The actual content received by the AST Parser is as follows.

extends Area2D

signal hit

@export var speed = 400
var screen_size

func _ready():
	screen_size = get_viewport_rect().size
	hide()

func _process(delta):
	var velocity = Vector2.ZERO
	if Input.is_action_pressed(&"move_right"):
		velocity.x += 1
	if Input.is_action_pressed(&"move_left"):
		velocity.x -= 1

	if velocity.length() > 0:
		velocity = velocity.normalized() * speed
		$AnimatedSprite2D.play()
	else:
		$AnimatedSprite2D.stop()

	position += velocity * delta
	position = position.clamp(Vector2.ZERO, screen_size)

func start(pos):
	position = pos
	show()
	$CollisionShape2D.disabled = false

There is no # player.gd here.

The AST Parser reads this text from top to bottom, cutting it into declarations and function units.

The first piece is a single line E009.

extends Area2D

The second piece is a single line of E011.

signal hit

The third piece is E013-E014.

@export var speed = 400
var screen_size

The fourth piece is E016-E018.

func _ready():
	screen_size = get_viewport_rect().size
	hide()

The fifth piece is E020-E034.

func _process(delta):
	var velocity = Vector2.ZERO
	if Input.is_action_pressed(&"move_right"):
		velocity.x += 1
	if Input.is_action_pressed(&"move_left"):
		velocity.x -= 1

	if velocity.length() > 0:
		velocity = velocity.normalized() * speed
		$AnimatedSprite2D.play()
	else:
		$AnimatedSprite2D.stop()

	position += velocity * delta
	position = position.clamp(Vector2.ZERO, screen_size)

The sixth piece is E036-E039.

func start(pos):
	position = pos
	show()
	$CollisionShape2D.disabled = false

The important point is that the fragment does not lose its original line range even after it is created. For example, the _process fragment continues to be tracked as a “fragment from player.gd lines E020‑E034”.

Step 4: Process Fragments Manually

If you need to split text files that are not targets of the AST Parser, divide them into fragments.

In project.godot, E004‑E006 becomes the application settings fragment.

[application]
config/name="Mini Dodge"
run/main_scene="res://player.tscn"

This fragment can be used to locate the project's start scene because of run/main_scene="res://player.tscn". If you perform a Retriever search, terms such as run/main_scene, res://player.tscn, and application that appear in the text become search material. The file path project.godot itself does not become a search keyword.

In player.tscn, the line E044 becomes a script reference fragment.

[ext_resource type="Script" path="res://player.gd" id="1"]

E046-E047 becomes a node fragment.

[node name="Player" type="Area2D"]
script = ExtResource("1")

This direct fragment is not an AST chunk, but the subsequent flow is the same. A portion of the text is used to search the Retriever, and the retrieved candidates are re‑validated by the LLM.

Step 5: What Actually Goes Into the Retriever

The Retriever receives “the current fragment’s text and the search clues extracted from that text,” not a “file path.”

For example, when processing the _process fragment, the text that serves as the search basis is E020‑E034.

func _process(delta):
	var velocity = Vector2.ZERO
	if Input.is_action_pressed(&"move_right"):
		velocity.x += 1
	if Input.is_action_pressed(&"move_left"):
		velocity.x -= 1

	if velocity.length() > 0:
		velocity = velocity.normalized() * speed
		$AnimatedSprite2D.play()
	else:
		$AnimatedSprite2D.stop()

	position += velocity * delta
	position = position.clamp(Vector2.ZERO, screen_size)

In this text, clues that can be used for searching appear as follows.

Input.is_action_pressed
Vector2.ZERO
velocity.normalized
position.clamp
AnimatedSprite2D
_process
Godot 3
Godot 4
migration

The clues Godot 3, Godot 4, and migration are attached when the user prompt is something like “Check whether this project is Godot 3 or Godot 4, and provide conversion references if needed.”

Retriever search uses this snippet’s body and the clues.

Vector search includes the entire body of the _process function and the main symbols.

Keyword search includes symbol names such as Input.is_action_pressed, Vector2.ZERO, position.clamp.

The file path player.gd is not used as a search term. However, it is later used in logs and UI to indicate, for example, “This search occurred in player.gd lines E020‑E034.”

Step 6: What the Retriever Looks For

The Retriever searches docs_chunks, api_mapping, and label_prototypes with the same snippet in the same way. None of the three tables receives special treatment.

In docs_chunks, candidates such as official documentation explanations for Input.is_action_pressed, Vector2.ZERO, position, Area2D may appear.

In api_mapping, candidates for function names, class names, and property names that changed from Godot 3 to Godot 4 may appear. For example, if the code snippet contains clues like KinematicBody2D, yield, Color8, related migration candidates may be returned.

In label_prototypes, candidates such as “a pattern where the usage of a single function changes entirely” or “examples where argument composition and call form change” may appear. This is meaningful when the change is not just a rename but requires explaining the usage change itself.

For example, in the _process snippet, the following candidates could be returned.

docs_chunks candidates:
  Input.is_action_pressed official documentation description
  Vector2.ZERO official documentation description
  Node2D.position official documentation description

api_mapping candidates:
  There may be no special Godot 3 -> 4 name change candidates in this fragment

label_prototypes candidates:
  Example of Godot 4 movement code using Input.is_action_pressed and Vector2.ZERO

These candidates are not the correct answer. They are only “evidence that may be related to this code snippet.”

Step 7: Passing Retriever Results to the LLM

The LLM receives the user prompt, the current code snippet, and the Retriever candidates together.

For example, in the _process snippet, it is included as follows.

User request:  
Check whether this project is Godot 3 or Godot 4, and if necessary, find the basis for conversion.

Current code snippet:  
player.gd lines E020‑E034  

func _process(delta):
    var velocity = Vector2.ZERO
    if Input.is_action_pressed(&"move_right"):
        velocity.x += 1
    if Input.is_action_pressed(&"move_left"):
        velocity.x -= 1

    if velocity.length() > 0:
        velocity = velocity.normalized() * speed
        $AnimatedSprite2D.play()
    else:
        $AnimatedSprite2D.stop()

    position += velocity * delta
    position = position.clamp(Vector2.ZERO, screen_size)

Retriever candidates:  
- Candidate from docs_chunks: official documentation for `Input.is_action_pressed`  
- Candidate from docs_chunks: official documentation for `Vector2.ZERO`  
- Candidate from docs_chunks: official documentation for `Node2D.position`  
- Candidate from label_prototypes: example of input handling in Godot 4

LLM makes two judgments here.

First, it checks whether the retrieved candidates are actually relevant to the current code snippet.

Second, it determines whether the current code snippet is a Godot 3 signal, a Godot 4 signal, or requires migration.

In this example, Input.is_action_pressed, Vector2.ZERO, and position.clamp can be interpreted as code that is also usable in Godot 4. Therefore, this snippet is likely to be accumulated as “can be considered Godot 4 code, no migration needed.”

Step 8: Flow of a Godot 3 Example Snippet

Assume the following snippet exists in another file.

G101 # enemy.gd
G102 extends KinematicBody2D
G103
G104 var velocity = Vector2.ZERO
G105
G106 func _physics_process(delta):
G107 	velocity = move_and_slide(velocity)

enemy.gd is a .gd file, so the AST Parser includes the G102‑G107 body.

The first fragment is G102.

extends KinematicBody2D

The second piece is G106-G107.

func _physics_process(delta):
	velocity = move_and_slide(velocity)

extends KinematicBody2D When the piece is passed to the Retriever, the search clue is captured as follows.

KinematicBody2D
Godot 3
Godot 4
migration

At this point, api_mapping may yield candidates where KinematicBody2D is migrated to the CharacterBody2D family in Godot 4.

In docs_chunks, candidates for official documentation related to CharacterBody2D or physics bodies may appear.

In label_prototypes, examples where code based on KinematicBody2D is changed to use CharacterBody2D with velocity and the move_and_slide() method may be generated.

The LLM receives input as follows.

User request:  
Check whether this project is Godot 3 or Godot 4, and if necessary, find the basis for conversion.

Current code snippet:  
enemy.gd line G102

extends KinematicBody2D

Retriever candidates:  
- api_mapping: migration candidate from KinematicBody2D → CharacterBody2D  
- docs_chunks: official documentation candidate for CharacterBody2D  
- label_prototypes: example candidate for converting KinematicBody2D movement code

LLM can judge this fragment as “Godot 3 signal is strong and migration needed”.

After that, the G106‑G107 fragment will also be processed separately.

func _physics_process(delta):
	velocity = move_and_slide(velocity)

In this fragment, the way move_and_slide(velocity) is called becomes the search clue. It looks not only at the function name but also at the call form with arguments.

At this point, if it is just a simple function name change, the api_mapping candidate is important, and if the entire call style changes, the label_prototypes candidate also becomes important.

Step 9: Accumulating Fragment Results

Each fragment is evaluated one by one.

The E020‑E034 in player.gd can be accumulated as a Godot 4 signal.

The G102 in enemy.gd can be accumulated as a Godot 3 signal.

The G106‑G107 in enemy.gd can also be accumulated as a Godot 3 migration signal.

The overall project judgment is not made by file name. File paths are for tracing only. The overall project judgment is based on the accumulated results of each code fragment’s evaluation.

For example, if most signals in a project are Godot 4 but some Godot 3 migration signals appear, the judgment may be close to mixed. Conversely, if most signals are Godot 3 migration, the project can be classified as a Godot 3 project.

Pause and Resume

RunPod or a local web server may stop in the middle. Therefore, processing must be able to continue by fragment rather than by file.

For instance, if you have processed up to E009, E011, and E013‑E014 in player.gd and stopped at E016‑E018, after restarting you should start again from E016‑E018.

Do not reprocess the already completed E009, E011, and E013‑E014.

The web UI should display the following.

player.gd E009 Processing completed
player.gd E011 Processing completed
player.gd E013-E014 Processing completed
player.gd E016-E018 Processing stopped
player.gd E020-E034 Waiting
player.gd E036-E039 Waiting

This way you can verify whether “the previous processing continues or it starts over from the beginning.”

What should be displayed on the debugging screen

The debugging screen is necessary for tracing.

At a minimum, the following should be displayed.

Current file: player.gd  
Current line range: E020-E034  
Current snippet content:  
func _process(delta):
	...

Search clues entered into Retriever:  
- Input.is_action_pressed  
- Vector2.ZERO  
- position.clamp  

Found candidates:  
- docs_chunks: Input.is_action_pressed  
- docs_chunks: Vector2.ZERO  
- label_prototypes: Godot 4 input polling example  

LLM judgment:  
- Godot 4 signals  
- no migration needed

Here, the important thing is that “what went into the search” and “what was attached merely for tracking” appear separated.

The path player.gd may be shown in the current file display, but it should not be included in the Retriever search clue list.

Summary

As of the 27th, the fixed flow in the document is as follows.

# <Relative Path>
<Body>

-> Separate only file boundaries with headers.  
-> The body of .gd files goes into the AST Parser.  
-> The AST Parser cuts functions and declarations in order.  
-> The actual body of each fragment and its symbol cues are fed into Retriever search.  
-> docs_chunks, api_mapping, and label_prototypes are searched in the same way.  
-> Pass the search candidates and the current fragment together to the LLM.  
-> Store the LLM's judgments per fragment and accumulate them per project.

The purpose of this document is not to define a new structure, but to make it possible to see where the actual input is cut off and which content moves on to the next step. When implementing, you should also display, according to this standard, the current file, current line range, actual transmitted content, Retriever search cues, search candidates, and LLM judgments together in the web UI.