It was World Cup season, so I couldn’t sleep well while watching the World Cup and couldn’t do much work
The Markdown → JSONL conversion at localhost:8501 is currently confirmed to be at about 655 items
Today I focused on simply creating and polishing a validation test debugging tool rather than large collection/training
Established a strategy for the validation test debugging tool and implemented a separate web tool called Qwen Validation Debugger
Execution path is tools/qwen-validation-debugger
Local run command is npm run validation-debug:debug
Base address is http://127.0.0.1:8520/
On June 28, I manually created Godot code chunks and JSONL with the Qwen chatbot and performed “yes”/“no” validation
This method proved feasible, but as the number of items grows, copying/pasting and result comparison become overwhelming
In particular, all three tables docs_chunks, api_mapping, label_prototypes need to be tested, requiring 6 slots for common syntax and 12 slots for version‑separated cases, making manual management rapidly complex
The tool loads a sample of 50 Godot test items, and for each item Qwen first labels whether it is common syntax or Godot 3/4 separated
If it is common syntax, one common code is generated
If there is a version difference, both a Godot 3 code and a Godot 4 code are generated
Afterwards, JSONL slots for each table are automatically assembled
Re‑confirmed that docs_chunks, api_mapping, and label_prototypes cannot be lumped together under a single “yes/no”
docs_chunks serve as code‑explanation evidence, so they are “yes” when they match the code directly
api_mapping and label_prototypes are migration‑source evidence, so they are often “no” when the code is already applied to Godot 4 or is common syntax
Therefore the slot names are divided into Explanation Evidence, Irrelevant Explanation, Conversion Needed, Already Applied, Common/Unnecessary, Irrelevant Conversion
The biggest change while building the tool was separating the JSONL generation criteria code from the validation target code
Initially, JSONL created from Godot 3 code was validated only against Godot 3 code, and JSONL from Godot 4 code was validated only against Godot 4 code
In practice, we also need to verify whether a Godot 3‑based JSONL attached to Godot 4 code should yield “no”
Thus each slot now has separate Godot 3 code validation and Godot 4 code validation buttons, and results are saved individually
After creating the tool, development productivity changed noticeably
Previously I had to request code and JSONL from Qwen each time, paste them back into a prompt for validation, and compare the expected response mentally
Now item selection, labeling, code generation, JSONL generation, Godot 3/4 cross‑validation, and raw prompt/response inspection all happen on a single screen
Rather than just producing a lot quickly, the biggest benefit is that I no longer get confused about which criteria were used for generation and which code was used for validation
When repeating the 50 test items, the amount of state the person has to remember is reduced, allowing faster identification of failure patterns
With this tool I can directly see on the screen which prompts are sent, the order in which bulk JSONL creation/validation is invoked, and how the validation prompt and request prompt themselves are composed
Expected responses are fixed according to labeling and slot nature, so the “yes”/“no” basis does not become ambiguous
Because response results appear per slot instantly, it is much quicker to see which code‑JSONL combination failed
Even if a case fails due to the prompt, the used prompt remains on screen, making later prompt adjustments easy
Having built the tool to a reasonable extent, I plan to slow the tempo a bit moving forward and, rather than pushing quantity, gradually refine the validation criteria