2026-06-25 Source Analysis and Markdown Classification Review

Today I reorganized the criteria for splitting the official documentation Markdown into three JSONL tables.

At first I considered dividing the Godot official documentation Markdown into the three tables docs_chunks, api_mapping, and label_prototypes. While the roles of docs_chunks and api_mapping were clear, the boundary for label_prototypes was vague.

Initially the boundaries among the three tables were unclear. By reviewing the flowchart again, I clarified that all three tables serve the same purpose: storing the official documentation Markdown converted to JSONL.

The conclusions are as follows.

docs_chunks is the source for the Godot 4 official documentation.
api_mapping is the JSONL target that stores how function names, class names, and symbol names changed from Godot 3 to Godot 4.
label_prototypes is the JSONL target that stores how to write entries when the usage pattern, argument composition, or call pattern of a function changes completely.

From now on I will analyze Godot projects at the file‑system level. Files such as .gd, .tscn, .tres, project.godot, etc., will be split into AST/code‑snippet units, and the necessary official‑documentation JSONL references will be used. Then I will call Qwen 3.6 on demand to determine whether each snippet belongs to Godot 3, Godot 4, needs migration, or can be used as code‑explanation data.

The result of this judgment will be stored in a score DB, and the file system will be classified per project as Godot 3 / Godot 4 / mixed / unknown. Based on the classified file system, I will later create sources for designing SFT and DPO.

Related roadmap:

docs/roadmaps/2026-06-25-source-analysis-scoring-architecture.md
docs/roadmaps/2026-06-25-markdown-jsonl-llm-classification.md

Markdown Classification Reset

Today I discarded the Markdown → JSONL classification results.

The reason is that, during the classification stage, only the first 3,000 characters of the Markdown were being sent to the LLM instead of the entire document. The user's intention and requirement were to classify based on the whole official documentation from the start. However, the generated code limited the transmission range to the front part, and I failed to notice this in time. Some official documents do not reveal their nature from just the beginning, and especially the boundaries of api_mapping and label_prototypes require checking whether the change is a function/symbol rename or a change in usage pattern/arguments/call pattern.

The clarified criteria:

For Markdown classification, send both the file name and the entire Markdown body.
Do not select a table based only on an excerpt of the front part.
Treat the existing JSONL outputs and the data inserted into PostgreSQL as results of a flawed criterion and discard them all.
Empty the DB and JSONL files, then restart classification.

Reset results:

godot_rag.docs_chunks: 0 entries
godot_rag.api_mapping: 0 entries
godot_rag.label_prototypes: 0 entries
godot_rag.ingest_reports: 0 entries
Local JSONL outputs deleted
Streamlit app restarted

A similar mistake occurred again in this work.

It was not that the user designed it incorrectly from the beginning; rather, the code and documents generated by the LLM diverged from the user's intent, and I failed to detect the discrepancy in time. It was not a one‑off oversight; similar omissions recurred while organizing table boundaries and data flow. As the saying goes, repeated mistakes are part of one’s skill level, so this area needs extra attention.

Specifically, there were issues with asserting the documentation before confirming the implementation, and writing undecided schemas and flows as if they were finalized. Even when docs_chunks, api_mapping, and label_prototypes should be treated at the same level, I described only certain tables as having a special flow, breaking document consistency.

From now on, writing documentation will be treated as an almost mandatory step for every task. The purpose of documentation is not just a tidy summary but a way to verify what criteria are being used for judgment. Whenever code or data changes even slightly, record the basis of that change to avoid repeating the same mistake.

If I had continued without documenting the current code structure, I might have discovered this issue much later. While writing the LLM classification method document, I realized that the actual code was sending only the first 3,000 characters of the Markdown instead of the whole document, contrary to the original requirement. This allowed me to modify the code to send the entire Markdown as intended. Thus, this incident serves as a reminder that documentation is not merely a simple summary but a verification process to ensure the code’s actual behavior matches the user’s intent.

Observation of RunPod Interruption and Conversion Resumption

During the Markdown → JSONL conversion, the RunPod server stopped unexpectedly, causing the conversion in the local web app to halt midway. After relaunching the Streamlit app, I checked whether the previous state was preserved or if the process started from scratch.

The observation criteria were the per‑file states stored in state.json and the Latest Processing Log of the web app. The key evidence before and after the restart is as follows.

Before restart the state was done 47, deferred 7, converting 1, pending 1515.
The file being processed at that time was pages/classes__class_astar3d__23ef6ac2.md.
After the app restart, the log contained Paused auto‑resume after app restart.
Then a log Returned processing file to pending appeared, and the target file was the same pages/classes__class_astar3d__23ef6ac2.md.
When restarted again, a log Reusing existing classification appeared for that file, after which it was completed and the process moved to the next file.
Subsequently, files pages/classes__class_astargrid2d__51463855.md, pages/classes__class_atlastexture__336f837e.md, pages/classes__class_audiobuslayout__a2e0df1f.md were processed sequentially, and the done count increased.

From these observations I confirmed the following.

The overall conversion result was not reset.
The file that was being processed was safely returned to pending.
Files already completed were not reprocessed.
Existing classification results were reused when available.
Restarting continued the process from the point where it was last interrupted.

However, when the RunPod server stops unexpectedly, the conversion flow can be delayed or halted for a long time. For lengthy conversion tasks, it is necessary to set up alerts that quickly indicate the RunPod server status, API response failures, or local app restarts. Simply looking at the web‑app screen does not allow you to distinguish whether the server has died, is waiting for an API response, or the local app has terminated.

If this flow continues to be used, at a minimum the following states should be made clearly visible.

RunPod server unreachable
API timeout or repeated 5xx responses
Streamlit app process termination
A processing file remains in the same table stage for an extended period
Auto‑resume paused due to app restart