Godot RAG Classifier Local PostgreSQL Setup

Date: June 22, 2026

Purpose

Set up a local PostgreSQL database for the Godot official documentation RAG classifier. The standard workflow is as follows.

JSONL -> PostgreSQL -> Retriever -> Validator -> Qwen 3.6

The goal of the current stage is not to generate training data, but to make it possible to reproduce the reference DB based on official documentation locally. Keep the payload column of the table with the same name as the JSONL schema, and only retain the minimal columns needed for search/storage for DB operation, such as id, embedding, search_tsv, created_at.

Reference design document:

docs/roadmaps/2026-06-21-initial-rag-classifier-architecture.md

Local DB Configuration

Docker Compose file:

infra/postgres/docker-compose.yml

Used image:

pgvector/pgvector:pg16

Connection Information:

Item	Value
database	`godot_rag`
user	`godot_rag`
password	`godot_rag_local`
host	`localhost`
port	`5432`

Default URL for local development:

postgresql://godot_rag:godot_rag_local@localhost:5432/godot_rag

Generated Extensions

extension	purpose
`vector`	Store document chunk and label prototype embeddings
`pg_trgm`	Assist fuzzy/partial search of API names, headings, and symbols

Note:

The embedding column is a vector type without a fixed dimension.  
Since pgvector’s HNSW/IVFFlat indexes require a fixed dimension,  
the actual vector index is created separately via migration after the embedding model’s dimension is determined.

Template:

infra/postgres/init/004_vector_index_templates.sql

Generated Schema

DB Schema Name:

godot_rag

Table:

Table	Role
`godot_rag.docs_chunks`	Official documentation description, tutorial, class reference chunks
`godot_rag.api_mapping`	Godot 3 → 4 API changes, rename, replacement, deprecation rules
`godot_rag.label_prototypes`	Classification/translation/rejection/modification example prototypes
`godot_rag.ingest_reports`	JSONL conversion/ingestion warning, skip, verification logs

Table Design

`docs_chunks`

Stores official documentation chunks. Every row must preserve the original URL, original file path, and original Markdown hash.

Key columns:

Column	Description
`chunk_id`	Deterministic ID that does not change on re‑execution
`doc_version`	Document version such as `stable`, `4.6`, etc.
`source_url`	Original URL of the Godot official documentation
`source_file`	Markdown path inside the repository
`source_sha256`	Original Markdown hash
`doc_type`	Document type such as `class_reference`, `tutorial`, `migration`, etc.
`symbol`	Representative symbol of a class/API document
`section_path`	Title hierarchy JSON
`content`	Body text for search and embedding
`code_blocks`	Array of code blocks extracted from the body
`api_symbols`	Godot API symbols detected in the body
`embedding`	pgvector embedding
`search_tsv`	tsvector for keyword search

`api_mapping`

Stores Godot 3/4 change rules.

Important principles:

If there is official documentation evidence, set confidence = 'verified_from_docs'.
Automatically extracted rules that have not been reviewed are set to confidence = 'candidate'.
Rules used directly for training/labeling are managed after human review with confidence = 'approved' or via a separate approval JSONL.

`label_prototypes`

Stores label criteria and representative patterns for classifier output.

Initial label candidates:

Label	Meaning
`godot4_valid_api`	Use of an API valid for Godot 4
`godot3_api_in_godot4`	Godot 3 API mixed into a Godot 4 project
`deprecated_or_removed_api`	Use of removed/deprecated API
`migration_required`	Conversion from Godot 3 → 4 required
`ambiguous_version_signal`	Insufficient or conflicting evidence for version determination
`non_godot_noise`	Data unrelated to Godot such as Python/web/Unity
`unsafe_or_obfuscated_code`	Obfuscated code, control characters, potentially malicious code

How to Run

Start the DB:

docker-compose -f infra/postgres/docker-compose.yml up -d

In the current local environment, the docker compose plugin is not used; instead, the docker‑compose command is used.

Health check:

docker inspect --format='{{json .State.Health.Status}}' godot-rag-postgres

Connect from inside the container:

docker exec -it godot-rag-postgres psql -U godot_rag -d godot_rag

Check the table:

docker exec godot-rag-postgres \
  psql -U godot_rag -d godot_rag \
  -c "\\dt godot_rag.*"

Extension check:

docker exec godot-rag-postgres \
  psql -U godot_rag -d godot_rag \
  -c "select extname from pg_extension where extname in ('vector', 'pg_trgm') order by extname;"

Initialization

Use only when creating the DB completely anew. Be careful, as the local volume will be deleted.

docker-compose -f infra/postgres/docker-compose.yml down -v
docker-compose -f infra/postgres/docker-compose.yml up -d

JSONL Injection Target Path

The injection script has not been created yet. After generating and validating the following JSONL outputs, attach the upsert script.

File	Target Table
`work/godot_rag/jsonl/docs_chunks.jsonl`	`godot_rag.docs_chunks`
`work/godot_rag/jsonl/api_mapping.jsonl`	`godot_rag.api_mapping`
`work/godot_rag/jsonl/label_prototypes.jsonl`	`godot_rag.label_prototypes`
`work/godot_rag/jsonl/ingest_report.jsonl`	`godot_rag.ingest_reports`

Next Tasks

Create an analysis report of the outputs/godot_docs_full/pages structure.
Finalize the chunking criteria for each document type.
Write a schema validation script for docs_chunks.jsonl.
Write the JSONL upsert script.
Verify Retriever search quality with sample questions.