2026 June 22 Retrospective

Today I created a workflow that first converts the Godot official documentation Markdown into structured JSONL for verification, rather than feeding the Markdown directly into RAG. At first I wasn’t sure how to split the official documentation Markdown, but after building a conversion UI and seeing the results per file, it became much easier.

Markdown → JSONL Converter

When a Godot official documentation Markdown file is uploaded, the Qwen API classifies the target tables and splits the results into JSONL for docs_chunks, api_mapping, and label_prototypes.

Initially I thought of simply reading the Markdown and inserting it straight into the DB, but that made it hard to see which document went to which table. So I first generate JSONL as an intermediate artifact and then compare the conversion results with the original text on screen.

The advantages of this approach are:

• You can compare the original Markdown with the converted JSONL.
• You can see where a single document ends up—docs_chunks, api_mapping, or label_prototypes.
• Mis‑classified documents or empty results can be filtered out before being loaded into the DB.
• If the conversion rules change later, you can regenerate the JSONL and compare again.

Checking Conversion Results

The converter displays the number of saved JSONL files and the number of errors on the screen. In the view I checked today, the docs_chunks side accumulated results first, while api_mapping, label_prototypes, and error files were still empty.

This state is not so much “wrong” as it is natural, because the early documents are mostly explanatory and therefore belong in docs_chunks. The important point is that the results do not go directly into the DB; they are first saved as JSONL and can be reviewed by a human.

JSONL Preview and Table View

The JSONL results can be viewed on screen both as raw JSON and as a table. For example, a docs_chunks record has fields such as chunk_id, doc_version, source_url, source_file, source_sha256, doc_type, section_path, heading, content, code_blocks, api_symbols, token_count, metadata.

Seeing it this way is far better than just holding the Markdown files. Especially with chunk_id and source_sha256 displayed together, it becomes easy to trace which chunk came from which source later. In RAG, the most critical thing is not losing the provenance of the evidence, and this intermediate JSONL artifact can serve that role.

Conversion Log

I also checked the per‑file conversion logs. You can see which file started, how Qwen classified it, and how many valid records were produced.

In the example I looked at today, the about__complying_with_licenses document was classified into docs_chunks and split into multiple chunks. Conversely, a document like 404 had no target table and was skipped. Having these logs is essential for tracing where problems occur when processing all 1,570 documents later.

Local PostgreSQL Setup

After creating the JSONL, I also prepared the DB so it can be loaded into a local PostgreSQL instance. I spun up a container based on pgvector/pgvector:pg16 and created the docs_chunks, api_mapping, label_prototypes, and ingest_reports tables.

What I paid particular attention to today is that the DB schema must not diverge from the JSONL field names. If the DB arbitrarily adds columns, the JSONL contract becomes fuzzy, and later the converter and injector could operate on different assumptions. Therefore, the DB payload column matches the JSONL schema, and operational columns are minimized to id, embedding, search_tsv, created_at.

I ran the local DB, performed sample JSONL‑style inserts and searches, and verified rollback behavior. While I haven’t yet bulk‑loaded all documents into the DB, the path from JSONL → PostgreSQL is now much clearer.

Today’s Assessment

The workflow I built today is not the final RAG classifier itself, but it is a significant step at this stage.

Previously, it was ambiguous how to handle the official documentation Markdown, and jumping straight to chunking or DB insertion risked losing structure. Today, by inserting a JSONL conversion/validation step in between, I created a human‑reviewable intermediate artifact.

In conclusion, the future workflow appears to be on the right track.

Godot Official Documentation Markdown
-> JSONL Conversion
-> JSONL Preview/Validation
-> PostgreSQL Injection
-> Retriever Search Validation
-> Validator/Qwen Response Summary

After tomorrow, when all 1,570 documents have been fully converted, we need to check the proportions of docs_chunks, api_mapping, and label_prototypes. In particular, api_mapping and label_prototypes must not be created arbitrarily by Qwen, so we should not trust the automatically generated results outright and should have a separate approval/verification step.

Processing Speed Measurement

Additionally, calculating the conversion speed suggests that converting all 1,570 Markdown files to JSONL will take about 42 hours.

The actual processing time so far has been about 1 hour 9 minutes. During this time, a total of 43 files were processed, comprising 39 done and 4 deferred. The average speed is about 1.6 minutes per file.

Actual processing time: about 1 hour 9 minutes  
Files processed so far: done 39 + deferred 4 = 43  
Average speed: about 1.6 minutes per file  
Estimated total time for 1,570 conversions: about 42 hours

At first I thought it would be about one item per minute based on perception, but when calculated from the actual logs it takes a little longer. It may be difficult to finish converting all 1,500 items today, so we need to keep recording processing speed and the number of failed/held files as we go.

Next Verification Plan

It is likely that we won’t have time tomorrow and the day after, so we probably won’t be able to continue the work immediately. Still, the tasks that will be resumed later have been roughly organized.

First, the JSONL files already generated during collection and conversion need to be pushed into a local PostgreSQL database. The converters we have built so far have focused on splitting Markdown into JSONL for docs_chunks, api_mapping, and label_prototypes; the next step is to load this JSONL into the actual DB and verify that it is searchable.

After that, we plan to lightly validate the workflow described in docs/roadmaps/2026-06-21-initial-rag-classifier-architecture.md using a small Python script.

source code
-> AST Parser
-> Retriever
-> evidence JSONL / evidence bundle
-> Qwen 3.6 API call
-> Check response

So, we will insert arbitrary Godot source code into a Python script and check whether the AST Parser extracts symbols and version signals. Then we feed the results into a Retriever to fetch relevant official documentation chunks or API mappings from PostgreSQL, and when we pass the returned JSONL/evidence bundle to the Qwen 3.6 API, we plan to intermittently check what response is produced.

This process is not a final automation but rather a small end‑to‑end verification to see if each step actually connects. In particular, we need to verify whether Qwen answers from memory or answers based on the evidence provided by the Retriever.

Thoughts on Public Repository and Switching to Private

Recently I made this repository public once and then switched it back to private. The reason seemed simple: I didn’t want to show my skills.

I still feel my current abilities are inadequate. At the same time, I thought about creating a Godot‑specific model, uploading it to Hugging Face, and using that as a starting point to expand into lecture videos, academia, a portfolio, reputation, and other areas. It may sound like a far‑fetched dream. Still, I thought that if someone could use my retrospectives and work logs as a stepping stone to grow, I might also be able to grow faster in the process.

In fact, I still have a reluctance to make it public. I’m afraid my creations might seem shallow, and conversely I was wary that the parts where I loosely stitched together various bits of knowledge to make something look easy might be ripped off as‑is. It wasn’t that the output was an incredible technology; rather, the entire trace of my thinking and connections being exposed felt burdensome.

I set up PR and CI/CD pipelines, but the original idea was that when a PR is opened in GitHub workflow, a local LLM endpoint hosted on Oracle Cloud would automatically review the PR. The Oracle Cloud side has a 24 GB VRAM environment, so most local models could run, and I imagined attaching a hosted LLM to automate code‑review. However, I lost my Oracle Cloud account, so this flow can’t be reused for now. RunPod also has the inconvenience of needing to be re‑configured each time for PR validation. Therefore, for the time being I’ll proceed with manual work and documentation‑focused efforts, and revisit LLM‑based PR review automation later.

The conclusion seems to be this: although it’s likely that no one will look at it, I was extremely wary of being ripped off. Yet even if someone references or takes my work, I can use that as a stepping stone to become even better. The decision to make it public should still be made cautiously, but I must not stop recording out of fear.