Recorded a retrospective of the Markdown → JSONL conversion and local DB ingestion preparation process
Described the workflow of converting official‑document Markdown to intermediate JSONL for preview/validation before loading into the DB.
Explained why the conversion output is split into docs_chunks, api_mapping, and label_prototypes, and structured for easy insertion into a local PostgreSQL.
Documented the reasons for making the repository public and then private again, and reflected on the decision to publish
Although I felt insecure about my current skill level and found public release daunting, I concluded that sharing the record can help others and accelerate my own growth.
Also outlined a plan to run a local LLM endpoint on an Oracle Cloud 24 GB VRAM environment and attach it to GitHub workflow/PR review automation.
Due to losing the Oracle Cloud account and the hassle of RunPod setup, I postponed LLM‑based PR review automation for later.
Decided to focus today on converting roughly 1,500 official‑document Markdown files to JSONL and loading them into the local DB
Converting a single file takes longer than expected, so it is still uncertain whether all 1,500 can be finished today.
Based on current logs, 39 files are done and 4 are deferred, totaling 43 files processed over about 1 hour 9 minutes.
The average speed is about 1.6 minutes per file, so converting all 1,570 files is estimated to require roughly 42 hours.
Planned verification steps for tomorrow and the day after, assuming time will be limited
Will inject the already generated JSONL into the local PostgreSQL and verify that it is searchable.