2026-06-02 Retrospective
What I Did Today
Today I re‑experimented with a Qwen‑family model to improve the gongju map of a Godot project. Assuming that the model has difficulty reliably interpreting asset images directly, Codex first examined the images and organized the asset path, pixel size, purpose, duplication status, and cautions in English in assets/ASSET_LABELS.md.
This document is an auxiliary index that lets Qwen know which assets to use when writing Godot code without having to interpret PNGs directly. In practice, the map code had issues such as using tiles of the wrong style as natural objects, or placing a house at an arbitrary coordinate on a grass sheet instead of using the actual building asset, and these problems could have been caught more easily if asset labels were present.
Qwen2.5 Coder Experiment
I tried the helper model for Godot work bekoozkan/godot-qwen2.5-coder-7b-instruct-bnb-4bit.
At first I ran it with llama.cpp, but Qwen Code’s tool calling was not properly connected. The model output text such as <function=glob> instead of actually invoking a tool, so it never progressed to file‑listing or code‑modification steps. I later confirmed that it needed to be run with vLLM.
Even though the execution method changed, Qwen2.5 Coder still did not reach the level where it could grasp the whole project context and reliably modify code. The response patterns differed between vLLM and llama.cpp, but in either case the model was not trustworthy enough to hand over real code changes.
However, this does not mean we should limit Qwen’s use. Rather, it is a failure case that shows what we need to teach Qwen to become a genuine coding tool. What is needed now is not to abandon the model but to align tool calling, reading the Godot project context, using asset labels, and a revision‑based workflow with the training data and execution environment.
Additional Tools Found
I discovered the reference project club-3090 for making local model execution and operation easier.
I also noted LLaMA-Factory as a candidate for future model adaptation or fine‑tuning experiments.
To use Qwen as a coding tool, we ultimately need to collect the failed and successful patterns from actual Godot work and fine‑tune the model. Without enough failure cases, merely swapping models or adding training utilities will likely repeat the same trial‑and‑error, so these failures should be kept as training‑data candidates.
MCP and Research
Through Qwen’s web-search-mcp I found a Reddit MCP candidate and added reddit-mcp-buddy.
Tools like Godot, local models, and Qwen Code often encounter issues that cannot be solved by official documentation alone, so Reddit or community examples can be necessary. However, attaching an MCP and actually improving development judgment are separate matters. The ability to turn search results into code modifications must continue to be validated.
Difficulties
Honestly, my mental stamina was pretty low today.
Even after changing the model, switching the server execution method, attaching an MCP, and creating asset descriptions, Qwen still did not reach the stage where it could read and fix the entire project. The fact that tool calls were output as text instead of being executed was especially disheartening. It looked like a coding agent on the surface, but in reality it could neither read nor modify files.
In the end I had to take a short break. Pushing forward would not give me better insight, only more repeated disappointment.
Today’s Judgment
Qwen2.5 Coder is not, in its current state, an agent that can be entrusted with the whole Godot project. However, the goal is not to use Qwen only in a limited way, but to make it a reliable coding agent.
Therefore, today’s failure serves as a basis for fine‑tuning and aligning the execution environment, not as a reason to shrink the scope of use.
- Codex handles image interpretation, file navigation, actual modifications, and verification.
- Asset labels, actual modification diffs, execution errors, and revision processes created by Codex are kept as Qwen training‑data candidates.
- Qwen is taught tool calling and workflow so that it can read the whole project context and directly edit Godot code.
- An execution environment where tool calling breaks is judged a failure immediately, and a vLLM‑based setup that enables real tool calls is prioritized.
- Use LLaMA‑Factory to attempt fine‑tuning based on Godot work patterns and failure cases.
Next Criteria
- When testing Qwen again, first verify that vLLM‑based tool calling actually works.
- If tool calls are output as text, consider that session a failure right away.
- Godot work must always verify execution results; it should not be marked complete based solely on the model’s description.
- Asset usage follows
assets/ASSET_LABELS.md, and tilesets with different styles should not be mixed. - Prepare tuning based on Godot failure cases with LLaMA‑Factory.
- The ultimate goal is to make Qwen a coding tool that performs actual code modifications.