Because it was World Cup season and I couldn’t sleep properly while watching the World Cup, I couldn’t do much work
The Markdown → JSONL conversion at localhost:8501 is currently confirmed to be at about 655 items
Today I focused on simply creating and polishing a validation test debugging tool rather than large-scale collection/training
Established a strategy for the validation test debugging tool and implemented a separate web tool called Qwen Validation Debugger
Execution path is tools/qwen-validation-debugger
Local run command is npm run validation-debug:debug
Base address is http://127.0.0.1:8520/
On June 28, I manually created Godot code chunks and JSONL with the Qwen chatbot and performed “yes”/“no” validation
This method proved feasible, but as the number of items grew, copy/paste and result comparison became overwhelming
In particular, all three tables docs_chunks, api_mapping, label_prototypes need to be tested, requiring 6 slots for common syntax and 12 slots for version‑separated syntax, making manual management rapidly complex
The tool loads 50 Godot test items as a sample, and for each item Qwen first labels whether it is common syntax or separated into Godot 3/4
If it is common syntax, one common code is generated
If there is a version difference, Godot 3 code and Godot 4 code are generated separately
Then JSONL slots for each table are automatically assembled
Re‑confirmed that docs_chunks, api_mapping, and label_prototypes cannot be lumped together under a single “yes/no”
docs_chunks are code‑explanation evidence, so if the code matches directly, the answer is “yes”
api_mapping and label_prototypes are migration source evidence, so often the answer is “no” when the code is already applied to Godot 4 or is common syntax
Therefore the slot names are divided into Explanation Evidence, Irrelevant Explanation, Conversion Needed, Already Applied, Common/Unnecessary, Irrelevant Conversion
The biggest change while building the tool was separating the JSONL generation criteria code from the validation target code
Initially, JSONL created from Godot 3 code was validated only against Godot 3 code, and JSONL from Godot 4 code was validated only against Godot 4 code
However, we also need to verify whether a Godot 3‑based JSONL attached to Godot 4 code should yield “no”
Thus each slot now has separate Godot 3 code validation and Godot 4 code validation buttons, and results are saved individually
After creating the tool, development productivity changed considerably
Previously I had to request code and JSONL from Qwen each time, paste them back into a prompt for validation, and compare the expected response mentally
Now item selection, labeling, code generation, JSONL generation, Godot 3/4 cross‑validation, and raw prompt/response inspection happen on a single screen
Rather than just producing a lot quickly, the biggest benefit is that I no longer get confused about which criteria were used for generation and which code was used for validation
When repeating 50 test items, the amount of state the person has to remember is reduced, allowing failure patterns to be identified more quickly
With this tool I can see directly on the screen which prompts are sent, the order in which batch JSONL creation/validation is called, and how the validation prompt and request prompt themselves are constructed
Expected responses are fixed according to labeling and slot nature, so the “yes/no” basis does not become confusing
Because response results appear per slot immediately, it is much faster to see which code‑JSONL combination failed
Even if a case fails because of the prompt, the used prompt remains on the screen, making later prompt adjustments easy
Having built the tool to a reasonable extent, I plan to slow the tempo a bit moving forward, focusing on gradually refining validation criteria rather than aggressively increasing volume