kgva/HANDOVER.md

openbureau-core — handover

For a fresh instance — START HERE

• Repo (local): cloned at /home/karim/openbureau-core. Remote git.openbureau.ch/karim/openbureau-core is private; auth via an HTTPS token in ~/.git-credentials (the SSH key id_ed25519_openbureau is currently rejected by gitea/proxmox — server seems rebuilt; fix the key or keep using the token).
• Where the work is: branch stufe-2-5-scaffold (pushed, NOT merged). main is still the pre-stage-2 foundation. Open a PR or merge when happy.
• Done: stages 1–4 complete, stage 5 scaffolded. See "Migration stages" below (markers [done]/[scaffold]). New code: api/src/collections.js, api/src/plugin-manager.js, api/src/plugins/dialog/index.js + their tests.
• Run tests: cd api && npm install && node --test → 44 green. node_modules is git-ignored; the fresh clone needs npm install once.
• Status: stages 1–8 (+5.5) DONE; stage 6 partial (schema/system/System-panel done, editor-from-schema deferred). openbureau runs on core on dev CT 134, verified. Next = stage 9 (onboard kgva: DB-less, auth:local, no plugins) — and finish stage 6's editor-from-schema there. Original stage-5 note kept below for history.
• Immediate next action (historic): stage 5 + 5.5 DONE on the branch (commits c4230a4 cutover, 5e06337 DDL, a93d11a migration runner). Plugin-manager singleton (api/src/plugins.js) wired into index.js/stats.js/publish.js, hard-coded dialog removed, source moved under api/src/plugins/dialog/; DDL captured into plugins/dialog/migrations/001_dialog.sql (idempotent); api/src/migrate.js applies migrations once via schema_migrations at boot. 53 tests green + a real boot. Live-verified on the dev stack (CT 134): a 2nd cms instance from cms-cms:latest (my src mounted over /app/src, real Supabase via kong:8000, empty SITE_DIR → no writes) returned byte-identical /api/forums & /api/recent vs the running container, /api/content 401 on both; the runner's tracking SQL was proven on a throwaway DB. NOT deployed (the running stack still runs old code). NEXT (stage 8): deploy core to openbureau — needs DATABASE_URL set for the cms service (e.g. postgres://postgres:$POSTGRES_PASSWORD@db:5432/postgres) so the runner can apply migrations, then full login/edit/publish verify before cutting the live container over. Then stages 6 (admin /api/schema), 7 (local auth), 9 (kgva). Dev stack: Proxmox CT 134 openbureau-dev, repo /opt/openbureau, compose cms/docker-compose.yml, containers openbureau-{cms,auth,kong,rest,db}, dev URL dev.openbureau.ch. Do NOT push a blind refactor to prod.
• Env gotcha: editing a file that's open in Karim's VSCode makes the Edit/Write tool hang/"interrupt" — write such files via Bash (cat > f <<'EOF' … EOF) instead.

Goal

Extract a generic, schema-driven Hugo CMS engine ("openbureau-core") from the openbureau site's bundled CMS, so that both openbureau and karimgabrielevarano.xyz (kgva) consume it as a dependency. Decision: own repo (this one), and migrate openbureau to depend on it.

State (done this session)

• Repo karim/openbureau-core created on Gitea, foundation pushed.
  • api/ = engine copied verbatim from karim/OPENBUREAU → cms/api (Hono/Node).
  • admin/ = React/Vite SPA copied verbatim from OPENBUREAU/cms/admin.
  • api/src/config.js = NEW loader: reads CMS_CONFIG → a site config module.
  • examples/openbureau.config.js, examples/kgva.config.js = target schemas.
  • README.md = config API + plugin model + the migration checklist.
• Nothing in the live openbureau CMS was changed yet.

Engine is ~80% generic. openbureau-specific seams to make config-driven:

1. [DONE] api/src/files.js classify(rel) + the order map — now derived from config.collections via api/src/collections.js (classify/buildPath/compareEntries); buildPath exposed there too.
2. [DONE] api/src/routes/stats.js — now per-collection (statKey/draftStatKey), dialog counts gated by hasPlugin('dialog').
3. api/src/index.js — always mounts dialog routes + runs syncLibrary on boot. → only when config.plugins includes dialog.
4. api/src/routes/publish.js — calls syncLibrary. → gate by plugin.
5. admin/src/App.jsx (714 lines) — hard-codes SECTIONS, KIND_LABEL, the field set, type dropdown, buildPath. → render sidebar groups + editor fields from the schema (add GET /api/schema returning config.collections).
6. dialog subsystem (dialog-store.js, routes/dialog.js, routes/comments.js
   • Supabase tables forums/threads/comments + library↔thread sync) = the dialog plugin (openbureau on, kgva off). First gate by config, later move to api/src/plugins/dialog/.

CRITICAL

api and admin share a data contract (stats keys, field shapes). Generalising the backend alone breaks the live openbureau admin — they must change together and be tested on the live Supabase/Hugo stack. Do NOT push a blind half-refactor to production openbureau.

Plugin system (decided architecture)

Everything beyond the generic engine is a plugin; the engine gets a small plugin manager. A plugin = api/src/plugins/<name>/index.js exporting a manifest { name, routes:[{path,app,public?,admin?}], onBoot, onPublish, onPreview, migrations:[…], stats, admin:{…} }. The manager reads config.plugins, imports each, mounts routes in the right auth tier (public before requireAuth, rest after, admin:true behind requireAdmin), registers boot/publish/preview/stats hooks, runs migrations; the admin SPA loads UI from admin/src/plugins/<name>. openbureau's extras become plugins (first: dialog); kgva enables none. (Full spec in README → "Plugins".)

Migration stages (also README checklist)

1. [done] repo + engine + config loader + example configs.
2. [done] files.js classify/order/buildPath ← config.collections (new api/src/collections.js = pure classify/buildPath/compareEntries/contentStats; files.js uses it, loads config lazily in listEntries; api/test/collections.test.js proves 1:1 vs openbureau.config.js). All api tests green.
3. [done] stats.js ← collections: content via statKey + draftStatKey (openbureau beitrag got draftStatKey: 'entwuerfe'), dialog counts gated by hasPlugin('dialog') (pluginless site does zero DB reads here).
4. [built, not wired] plugin manager — api/src/plugin-manager.js (loadPlugins(names) imports plugins/<name>/index.js; createManager(manifests) = pure wiring: mountPublic/mountPrivate(+requireAdmin), runBoot/runPublish/ runPreview, collectStats (merged under plugin name), migrations() (resolves file URLs; execution deferred to the live DB)). Unit-tested in api/test/plugin-manager.test.js (tiers/hooks/stats/migrations). TODO: wire into index.js (stage 5), and admin loads plugin UI from admin/src/plugins/<name>.
5. [DONE — commits c4230a4 (cutover) + 5e06337 (DDL), live e2e verified] extract openbureau extras into the dialog plugin. DONE: api/src/plugins/dialog/index.js = manifest wrapping the existing modules unchanged (public reads + widget login(rate-limited)
   • authed writes + self-guarding mod/adminForums sub-apps; onBoot/onPublish = syncLibrary; stats = forum/thread/comment counts). CUTOVER DONE: shared manager singleton api/src/plugins.js wired into index.js (mountPublic before requireAuth, mountPrivate(+requireAdmin) after, manager.runBoot at serve()), publish.js (manager.runPublish) and stats.js (manager.collectStats, {content,users,dialog} contract preserved); hard-coded dialog REMOVED from all three; dialog source physically moved to plugins/dialog/{dialog-store,dialog,comments}.js (imports fixed). Tested: api/test/{dialog-plugin,cutover}.test.js, 48 green + real boot. DDL CAPTURED (commit 5e06337): migrations/001_dialog.sql from the live dev DB, idempotent, validated on a throwaway DB; manifest migrations: ['001_dialog.sql']. LIVE E2E VERIFIED (dev CT 134): 2nd cms instance from cms-cms:latest with my src mounted, real Supabase via kong:8000 — /api/forums & /api/recent byte-identical to the running container, /api/content 401 both, no writes. Migration runner = stage 5.5 (commit a93d11a, see new item below). Deploy is stage 8. NB: dialog-store.js still filters e.kind === 'beitrag' — openbureau-specific, already moved with the plugin. 5.5 [done, commit a93d11a] migration runner — api/src/migrate.js runMigrations tracks applied migrations in public.schema_migrations and applies each declared file once, in its own transaction, at boot (wired into index.js before runBoot). Lean: no declared migrations → no DB connection, no pg import (kgva stays DB-less). Executor is injectable (unit-tested without a DB); default uses a lazily-imported pg against DATABASE_URL, warning-and-skipping if unset. Needs DATABASE_URL wired into the cms service env for openbureau (stage 8). Tested: migrate.test.js + tracking SQL proven on the live dev Postgres (throwaway DB).
6. [PARTLY DONE, commit f709b5d] GET /api/schema (site/auth/plugins/collections)
   • GET /api/system (admin: core version, plugins, content model, hugo, auth) + an admin System panel — all additive, the editor is untouched (stays 1:1). DEFERRED by design: making the editor itself render its fields from the schema — that needs a 2nd content model (kgva) + visual verification, not a blind rewrite of the live editor. Do it when onboarding kgva (stage 9).
7. [DONE, commit 86f9f57] auth provider (config.auth = supabase | local). api/src/auth-local.js = file-based users (bcryptjs) + self-signed HS256 JWTs of the SAME claim shape; routes/auth.js POST /api/auth/login (mounted only when auth:local); supabase.js made null-safe (fail-fast moved to index.js); auth/stats/ users.js branch local↔GoTrue; admin supabase.js gains a local-auth SHIM (same interface) when VITE_SUPABASE_URL is absent → App/api untouched, supabase path byte-identical. examples/kgva.config.js → auth:'local'. Verified: 59 tests + DB-less boot smoke (no Supabase: local login → /api/me admin → stats-from-file)
   • openbureau redeployed and confirmed unregressed (auth=supabase).
8. [DONE, OB main 3d902a0] openbureau consumes core. Mechanism: git subtree at OPENBUREAU/cms/core (fits the pull-based deploy; update via git subtree pull --prefix=cms/core <core> main --squash). docker-compose builds the cms image from ./core; cms/openbureau.config.js (CMS_CONFIG) carries the content model; old vendored cms/api+cms/admin removed. DATABASE_URL left unset (the stack's migrate service owns db/schema.sql incl. dialog). Deployed to dev CT 134 and FULL write-verified: login/list/edit/preview/publish/dialog all identical, no git divergence (GIT_PUBLISH=false). Rollback image cms-cms:rollback kept.
9. onboard kgva (CMS_CONFIG=…/kgva.config.js, auth: 'local', no plugins) as the 2nd consumer — DB-less.

How to run / test

openbureau CMS stack: OPENBUREAU/cms/docker-compose.yml (Node api + Hugo + Supabase: postgres/kong/gotrue). Env in cms/.env (see .env.example): SUPABASE_URL/SERVICE_KEY, JWT_SECRET, ANON_KEY, SERVICE_ROLE_KEY (derive via scripts/generate-keys.mjs), ADMIN_EMAILS, SITE_URL, GIT_*. For core add CMS_CONFIG (path to the site config) and SITE_DIR (site repo mount, default /site). Admin at /admin, preview /_preview, publish builds public/.

Infra access (NO secrets in this file)

• Proxmox node 192.168.1.2, user root. Password: ask the user (provided ad hoc, not stored). No SSH key installed — use SSH_ASKPASS, or install a key first.
• Gitea = container 120 (pct exec 120 …), ROOT_URL git.openbureau.ch, sqlite at /var/lib/gitea/data/gitea.db. Make a token: pct exec 120 -- su gitea -s /bin/bash -c "/usr/local/bin/gitea admin user generate-access-token --username karim --scopes all --token-name X --raw --config /etc/gitea/app.ini". Keep token kgva-deploy. Delete tokens via sqlite (DELETE FROM access_token WHERE name LIKE '…') since the API needs the account password.
• Push without local git auth: tar → scp to node → pct push 120 into the gitea container → git push http://karim:<token>@127.0.0.1:3000/karim/<repo>.git.
• kgva site deploy: container 130 (kgva-website). nginx serves /var/www/karimgabrielevarano.xyz (= ZFS tank/kgva-website, owned uid 100000). systemd timer runs /opt/kgva-deploy.sh every 60s: pull karim/kgva (token kgva-deploy) → hugo build → copy into webroot. So a push to karim/kgva is live in ~1 min. Self-hosted video at /var/www/media (nginx /media/ alias, outside the build). Public TLS for the domain terminates upstream → 192.168.1.130:80.

Watch out

• Gitea does NOT send CORS on its OAuth token endpoint → a browser git-CMS (Decap-style) can't auth without a same-origin proxy. openbureau-core uses its own Supabase auth, so this doesn't affect it; it's why kgva edits go via git push.
• Hugo versions: openbureau CMS bundles 0.161.1; the kgva site needs 0.163.3 features. Mind the version a core instance builds with.