Recorded a retrospective of the Markdown → JSONL conversion and local DB ingestion preparation process
Described the workflow of converting official‑document Markdown to intermediate JSONL for preview/verification before loading into the DB.
Explained why the conversion results are split into docs_chunks, api_mapping, and label_prototypes and structured for easy insertion into local PostgreSQL.
Documented the reason for making the repository public and then private again, and reflected on the decision to publish
Although I felt insecure about my current skill level and found publishing burdensome, I concluded that sharing the record can help others and also accelerate my own growth.
Also outlined a plan to run a local LLM endpoint on an Oracle Cloud 24 GB VRAM instance and attach it to GitHub workflow/PR review automation.
Because I lost access to the Oracle Cloud account and found RunPod setup cumbersome, I postponed LLM‑based PR review automation for later.
Decided to focus today on converting roughly 1,500 official‑document Markdown files to JSONL and loading them into the local DB
Converting a single file takes longer than expected, so it is still uncertain whether all 1,500 can be finished today.
Based on current logs, 39 files are done and 4 are deferred (total 43 files) after about 1 hour 9 minutes of processing.
The average speed is about 1.6 minutes per file, so converting all 1,570 files is estimated to require roughly 42 hours of compute time.
Since tomorrow and the day after are likely unavailable, I outlined the verification plan for the subsequent work
Plan to inject the already generated JSONL into local PostgreSQL and verify that it is searchable.