Re‑examined questions about the Godot 4 dataset/RAG pipeline while on a bus to Seoul
Previously imagined crawling the official Godot documentation, converting GitHub projects to md and jsonl formats, and using a RAG chatbot to decide whether the project is Godot 3 or 4
Became uncertain whether the entire project context and the retrieved official‑documentation snippets could fit into the model’s input context at once
Considered the input‑context limits of a RAG chatbot
A single GitHub project includes a README, directory structure, multiple .gd files, scene files, and resource paths
Adding RAG‑retrieved official documentation would further enlarge the input, so designing a selection strategy for which files and document fragments to include is more important than simply gathering many documents
Questioned the shape of a question/answer dataset
Requests like “make a map” may not be solvable with a single answer code snippet
In practice, several inference steps are needed: understanding project structure, identifying assets, analyzing existing code style, reviewing Godot 4 syntax, deciding which files to modify, etc.
This raised the issue of whether an instruction dataset should contain only the final code or also preserve the exploration and decision‑making process
Worried that Python weighting might become dominant again during chunk‑level processing
Even if the prompt asks “make a map with Godot 4”, the model might revert to Python‑centric pre‑training weights while reading project and documentation chunks
Felt the need to inject Godot 4 context strongly at the beginning of the prompt and to filter more aggressively during data preprocessing so that Godot 3 code or Python‑style answers do not slip in
Today’s summary
RAG or crawling alone is not the solution; the key is to design what context the model reads and how it makes decisions for real requests
Plan to split project context by file/role/dependency, and to include not only the final answer but also, when needed, the exploration and judgment flow in the instruction data
To prevent dilution of Godot 4 context during inference, we need stronger designs for prompts, tags, filtering, and preferred‑data criteria