idea_world_labDEV JOURNAL
Friday, June 12, 2026

June 12, 2026

  • Decided to write a development retrospective again after a long time

    • Because I wanted to leave only perfect records, I ended up postponing documentation
    • To meaningfully record the attempts and concerns of the past ~10 days, I organized the research and architecture design process for creating a Godot‑specific coding model

  • Conducted an experiment to deploy a Qwen‑series model on a local PC to reduce RunPod costs

    • Tried to run a 9B model on WSL in an RTX 3060 environment
    • However, network speed and response latency were severe, and even the reasoning stage before generating an actual answer took more than 5 minutes, so the local deployment experiment was halted

  • Investigated dataset collection methods for training a Godot‑specific model

    • Checked the dataset referenced by Hugging Face’s wallstoneai/godot-gdscript-dataset
    • Analyzed how the dataset was created using Gemini
    • The core idea was to merge a GitHub repository’s README.md, .gd files, and project structure into a single text, then use the project.godot configuration file and GDScript syntax differences to classify Godot 3/4 versions
    • Notably, by leveraging version‑specific clues such as config_version, config/features, onready var, @onready, KinematicBody, CharacterBody3D, we discovered that even non‑mainstream languages without JSON‑based dependency files can be version‑filtered

  • Watched a fine‑tuning video about the classic programming language OPL to understand the fine‑tuning workflow

  • Asked an SSAFY coach how to efficiently collect data for a specific version of a non‑mainstream language

    • Received the answer that the current Godot dataset is closer to a raw code dataset than an assistant‑training Q&A dataset
    • Concluded that to build a chatbot‑style product, it is better to generate question/answer pairs with an LLM and process them into an instruction dataset rather than feeding raw data directly
    • Without this step, a request like “design a map” would likely yield a Python‑centric answer that the base model has heavily learned, which is undesirable

  • Evaluated the instruction‑dataset candidate ise-uiuc/Magicoder-Evol-Instruct-110K

    • Determined it is mostly Python‑centric and not suitable for direct use as Godot 4‑specific training data
    • Considered the question “Can we get Godot answers without explicitly mentioning Godot?” but concluded that because the base model’s Python weights dominate, explicitly providing the context Godot in the prompt increases the chance of a correct answer

  • Consulted a senior at school about RAG and prompting strategies

    • Received advice that, rather than injecting all data into the model, building a markdown‑document‑based vector search structure and guiding the system to retrieve needed information may be more realistic
    • Since re‑indexing large documents is costly and time‑consuming, a retrieval/prompting‑based approach may be more appropriate at the current stage than full‑scale training

  • Designed an initial architecture for creating a Godot‑specific coding model

  • Initially thought of a simple structure like dataset collection -> question/answer dataset creation -> model training

    • However, there was an issue that accurate filtering and answer data generation required a thorough understanding of the changes from Godot 3 to 4
    • Decided that if not careful, Godot 3 code, Python code, and legacy APIs could get mixed into the answer data, so rethought the architecture

  • Designed a structure that places an official‑document‑based RAG chatbot at the front for classifying and converting Godot 3/4 versions

    • Crawled the Godot official migration documents and Godot 4 documentation to build a RAG chatbot, and envisioned using this chatbot to classify whether the collected data belongs to Godot 3 or 4
    • Then planned a flow that processes only the data identified as Godot 4 into an instruction dataset

  • Gained additional insights on SFT/DPO training directions through ChatGPT

    • In SFT, tasks such as Godot 3/4 classification, Godot 3 → 4 conversion, Godot 4 code generation, Godot 4 error fixing, and Godot 3 API rejection/correction can be created
    • In DPO/Preference, the preference data can be organized as bad answer = answer containing Godot 3 code, good answer = pure Godot 4 code answer

  • Used unclecode/crawl4ai to crawl the official Godot documentation

  • Consulted a senior at school about disk I/O bottlenecks in the data storage and training pipeline

    • Received advice that it is better to handle data acquisition and preprocessing/post‑processing in near‑real‑time, while performing training in batch mode, rather than running fine‑tuning in real time
    • Decided to consider metric‑based batch processing that triggers reinforcement learning or fine‑tuning when the dataset exceeds a certain size
    • Concluded that because re‑indexing costs cannot be completely eliminated structurally, the stability of the data acquisition and processing pipeline is more important than real‑time training

  • Current overall plan

    • Crawl official documents to build a RAG knowledge base based on Godot 4
    • Collect GitHub Godot projects and merge README, project structure, and GDScript files on a per‑repository basis
    • Perform first‑stage filtering using project.godot settings and differences between Godot 3/4 syntax
    • Use the RAG chatbot to further determine Godot 3/4 status, legacy API usage, and suitability for Godot 4
    • Generate instruction/response data from the data refined for Godot 4
    • Train a Godot 4 coding model with SFT and DPO/Preference data

  • Reflection

    • Because I wanted to leave only the finished results over the past 10 days, I ended up not recording the process
    • However, I realized that failed experiments, stuck points, and mid‑process judgment changes are the crucial records for guiding the next steps
    • Going forward, I will focus less on leaving only perfect outcomes and more on consistently documenting the flow of attempts and decisions to keep improving

  • Development Retrospective: docs/retrospectives/2026-06-12.md