203 lines
15 KiB
Markdown
203 lines
15 KiB
Markdown
# openbureau-core — handover
|
||
|
||
## For a fresh instance — START HERE
|
||
- **Repo (local):** cloned at `/home/karim/openbureau-core`. Remote
|
||
`git.openbureau.ch/karim/openbureau-core` is **private**; auth via an HTTPS token
|
||
in `~/.git-credentials` (the SSH key `id_ed25519_openbureau` is currently rejected
|
||
by gitea/proxmox — server seems rebuilt; fix the key or keep using the token).
|
||
- **Where the work is:** branch **`stufe-2-5-scaffold`** (pushed, NOT merged). `main`
|
||
is still the pre-stage-2 foundation. Open a PR or merge when happy.
|
||
- **Done:** stages 1–4 complete, stage 5 *scaffolded*. See "Migration stages" below
|
||
(markers `[done]`/`[scaffold]`). New code: `api/src/collections.js`,
|
||
`api/src/plugin-manager.js`, `api/src/plugins/dialog/index.js` + their tests.
|
||
- **Run tests:** `cd api && npm install && node --test` → **44 green**. `node_modules`
|
||
is git-ignored; the fresh clone needs `npm install` once.
|
||
- **Status:** stages 1–9 DONE; openbureau + kgva BOTH LIVE on core. openbureau on dev CT134 (schema editor + supabase); kgva LIVE at karimgabrielevarano.xyz via core (Variant A: CMS serves the site; CT130 rebuilt in-place — Docker + kgva-cms on :8081, nginx :80 proxies to it and keeps /media, static timer disabled, Caddy untouched). CAVEAT: kgva GIT_PUBLISH=false → edits on CT130 disk, not git-backed (enable GIT_PUBLISH + creds to fix). Rollback for kgva: restore nginx .static.bak + re-enable kgva-deploy.timer + stop kgva-cms. Original stage-5 note kept below for history.
|
||
- **Immediate next action (historic):** stage 5 + 5.5 **DONE** on the branch (commits `c4230a4`
|
||
cutover, `5e06337` DDL, `a93d11a` migration runner). Plugin-manager singleton
|
||
(`api/src/plugins.js`) wired into `index.js`/`stats.js`/`publish.js`, hard-coded
|
||
dialog removed, source moved under `api/src/plugins/dialog/`; DDL captured into
|
||
`plugins/dialog/migrations/001_dialog.sql` (idempotent); `api/src/migrate.js`
|
||
applies migrations once via `schema_migrations` at boot. **53 tests green** + a real
|
||
boot. **Live-verified on the dev stack** (CT 134): a 2nd cms instance from
|
||
`cms-cms:latest` (my `src` mounted over `/app/src`, real Supabase via `kong:8000`,
|
||
empty SITE_DIR → no writes) returned **byte-identical** `/api/forums` & `/api/recent`
|
||
vs the running container, `/api/content` 401 on both; the runner's tracking SQL was
|
||
proven on a throwaway DB. **NOT deployed** (the running stack still runs old code).
|
||
NEXT (stage 8): deploy core to openbureau — needs `DATABASE_URL` set for the cms
|
||
service (e.g. `postgres://postgres:$POSTGRES_PASSWORD@db:5432/postgres`) so the
|
||
runner can apply migrations, then full login/edit/publish verify before cutting the
|
||
live container over. Then stages 6 (admin `/api/schema`), 7 (local auth), 9 (kgva).
|
||
Dev stack: Proxmox CT 134 `openbureau-dev`, repo `/opt/openbureau`, compose
|
||
`cms/docker-compose.yml`, containers openbureau-{cms,auth,kong,rest,db}, dev URL
|
||
dev.openbureau.ch. **Do NOT push a blind refactor to prod.**
|
||
- **Env gotcha:** editing a file that's open in Karim's VSCode makes the Edit/Write
|
||
tool hang/"interrupt" — write such files via Bash (`cat > f <<'EOF' … EOF`) instead.
|
||
|
||
## Goal
|
||
Extract a generic, **schema-driven** Hugo CMS engine ("openbureau-core") from the
|
||
openbureau site's bundled CMS, so that **both** openbureau and
|
||
karimgabrielevarano.xyz (kgva) consume it as a dependency. Decision: own repo
|
||
(this one), and migrate openbureau to depend on it.
|
||
|
||
## State (done this session)
|
||
- Repo `karim/openbureau-core` created on Gitea, foundation pushed.
|
||
- `api/` = engine copied verbatim from `karim/OPENBUREAU` → `cms/api` (Hono/Node).
|
||
- `admin/` = React/Vite SPA copied verbatim from `OPENBUREAU/cms/admin`.
|
||
- `api/src/config.js` = NEW loader: reads `CMS_CONFIG` → a site config module.
|
||
- `examples/openbureau.config.js`, `examples/kgva.config.js` = target schemas.
|
||
- `README.md` = config API + plugin model + the migration checklist.
|
||
- Nothing in the live openbureau CMS was changed yet.
|
||
|
||
## Engine is ~80% generic. openbureau-specific seams to make config-driven:
|
||
1. [DONE] `api/src/files.js` `classify(rel)` + the `order` map — now derived from
|
||
`config.collections` via `api/src/collections.js` (classify/buildPath/compareEntries);
|
||
`buildPath` exposed there too.
|
||
2. [DONE] `api/src/routes/stats.js` — now per-collection (`statKey`/`draftStatKey`),
|
||
dialog counts gated by `hasPlugin('dialog')`.
|
||
3. `api/src/index.js` — always mounts dialog routes + runs `syncLibrary` on boot.
|
||
→ only when `config.plugins` includes `dialog`.
|
||
4. `api/src/routes/publish.js` — calls `syncLibrary`. → gate by plugin.
|
||
5. `admin/src/App.jsx` (714 lines) — hard-codes `SECTIONS`, `KIND_LABEL`, the field
|
||
set, type dropdown, `buildPath`. → render sidebar groups + editor fields from the
|
||
schema (add `GET /api/schema` returning `config.collections`).
|
||
6. dialog subsystem (`dialog-store.js`, `routes/dialog.js`, `routes/comments.js`
|
||
+ Supabase tables forums/threads/comments + library↔thread sync) = the `dialog`
|
||
**plugin** (openbureau on, kgva off). First gate by config, later move to
|
||
`api/src/plugins/dialog/`.
|
||
|
||
## CRITICAL
|
||
api and admin share a data contract (stats keys, field shapes). Generalising the
|
||
**backend alone breaks the live openbureau admin** — they must change together and
|
||
be tested on the live Supabase/Hugo stack. Do NOT push a blind half-refactor to
|
||
production openbureau.
|
||
|
||
## Plugin system (decided architecture)
|
||
Everything beyond the generic engine is a **plugin**; the engine gets a small
|
||
**plugin manager**. A plugin = `api/src/plugins/<name>/index.js` exporting a
|
||
manifest `{ name, routes:[{path,app,public?,admin?}], onBoot, onPublish, onPreview,
|
||
migrations:[…], stats, admin:{…} }`. The manager reads `config.plugins`, imports
|
||
each, mounts routes in the right auth tier (public before `requireAuth`, rest
|
||
after, `admin:true` behind `requireAdmin`), registers boot/publish/preview/stats
|
||
hooks, runs migrations; the admin SPA loads UI from `admin/src/plugins/<name>`.
|
||
openbureau's extras become plugins (first: `dialog`); kgva enables none.
|
||
(Full spec in README → "Plugins".)
|
||
|
||
## Migration stages (also README checklist)
|
||
1. [done] repo + engine + config loader + example configs.
|
||
2. [done] files.js classify/order/buildPath ← config.collections
|
||
(new `api/src/collections.js` = pure classify/buildPath/compareEntries/contentStats;
|
||
files.js uses it, loads config lazily in listEntries; `api/test/collections.test.js`
|
||
proves 1:1 vs openbureau.config.js). All api tests green.
|
||
3. [done] stats.js ← collections: `content` via `statKey` + `draftStatKey`
|
||
(openbureau beitrag got `draftStatKey: 'entwuerfe'`), dialog counts gated by
|
||
`hasPlugin('dialog')` (pluginless site does zero DB reads here).
|
||
4. [built, not wired] **plugin manager** — `api/src/plugin-manager.js`
|
||
(`loadPlugins(names)` imports `plugins/<name>/index.js`; `createManager(manifests)`
|
||
= pure wiring: `mountPublic`/`mountPrivate(+requireAdmin)`, `runBoot/runPublish/
|
||
runPreview`, `collectStats` (merged under plugin name), `migrations()` (resolves
|
||
file URLs; execution deferred to the live DB)). Unit-tested in
|
||
`api/test/plugin-manager.test.js` (tiers/hooks/stats/migrations). TODO: wire into
|
||
index.js (stage 5), and admin loads plugin UI from `admin/src/plugins/<name>`.
|
||
5. [DONE — commits c4230a4 (cutover) + 5e06337 (DDL), live e2e verified] extract openbureau extras
|
||
into the **dialog** plugin. DONE: `api/src/plugins/dialog/index.js` = manifest
|
||
wrapping the existing modules unchanged (public reads + widget login(rate-limited)
|
||
+ authed writes + self-guarding mod/adminForums sub-apps; onBoot/onPublish =
|
||
syncLibrary; stats = forum/thread/comment counts). CUTOVER DONE: shared manager
|
||
singleton `api/src/plugins.js` wired into index.js (mountPublic before requireAuth,
|
||
mountPrivate(+requireAdmin) after, manager.runBoot at serve()), publish.js
|
||
(manager.runPublish) and stats.js (manager.collectStats, `{content,users,dialog}`
|
||
contract preserved); hard-coded dialog REMOVED from all three; dialog source
|
||
physically moved to `plugins/dialog/{dialog-store,dialog,comments}.js` (imports
|
||
fixed). Tested: `api/test/{dialog-plugin,cutover}.test.js`, 48 green + real boot.
|
||
DDL CAPTURED (commit 5e06337): `migrations/001_dialog.sql` from the live dev DB,
|
||
idempotent, validated on a throwaway DB; manifest `migrations: ['001_dialog.sql']`.
|
||
LIVE E2E VERIFIED (dev CT 134): 2nd cms instance from cms-cms:latest with my src
|
||
mounted, real Supabase via kong:8000 — /api/forums & /api/recent byte-identical to
|
||
the running container, /api/content 401 both, no writes. Migration runner = stage
|
||
5.5 (commit a93d11a, see new item below). Deploy is stage 8.
|
||
NB: dialog-store.js still filters `e.kind === 'beitrag'` — openbureau-specific,
|
||
already moved with the plugin.
|
||
5.5 [done, commit a93d11a] migration runner — `api/src/migrate.js` `runMigrations`
|
||
tracks applied migrations in `public.schema_migrations` and applies each declared
|
||
file once, in its own transaction, at boot (wired into index.js before runBoot).
|
||
Lean: no declared migrations → no DB connection, no pg import (kgva stays DB-less).
|
||
Executor is injectable (unit-tested without a DB); default uses a lazily-imported
|
||
`pg` against `DATABASE_URL`, warning-and-skipping if unset. Needs `DATABASE_URL`
|
||
wired into the cms service env for openbureau (stage 8). Tested: migrate.test.js +
|
||
tracking SQL proven on the live dev Postgres (throwaway DB).
|
||
6. [DONE, commits f709b5d + f8f4131 + 59b9e20] schema-driven admin. `GET /api/schema`
|
||
+ `GET /api/system` + a System panel; AND the editor now renders sidebar groups +
|
||
fields from the site's collections (admin/src/fields.jsx: per-type controls
|
||
string/text/slug/number/date/bool/select/list/image/color/markdown; form↔frontmatter
|
||
+ path building). One admin serves any site. Unknown frontmatter is PRESERVED on
|
||
save (__raw) — no data loss. Verified working via kgva (stage 9). NOTE: openbureau
|
||
is NOT yet redeployed with the schema editor (its running image keeps the bespoke
|
||
one); when it is, its config field types (color→swatch, layout opts) are already
|
||
tuned. Visual review of the openbureau editor by Karim still recommended before
|
||
redeploying it.
|
||
7. [DONE, commit 86f9f57] **auth provider** (`config.auth` = `supabase` | `local`).
|
||
`api/src/auth-local.js` = file-based users (bcryptjs) + self-signed HS256 JWTs of
|
||
the SAME claim shape; `routes/auth.js` POST /api/auth/login (mounted only when
|
||
auth:local); supabase.js made null-safe (fail-fast moved to index.js); auth/stats/
|
||
users.js branch local↔GoTrue; admin supabase.js gains a local-auth SHIM (same
|
||
interface) when VITE_SUPABASE_URL is absent → App/api untouched, supabase path
|
||
byte-identical. examples/kgva.config.js → auth:'local'. Verified: 59 tests +
|
||
DB-less boot smoke (no Supabase: local login → /api/me admin → stats-from-file)
|
||
+ openbureau redeployed and confirmed unregressed (auth=supabase).
|
||
8. [DONE, OB main 3d902a0] openbureau consumes core. Mechanism: **git subtree** at
|
||
`OPENBUREAU/cms/core` (fits the pull-based deploy; update via
|
||
`git subtree pull --prefix=cms/core <core> main --squash`). docker-compose builds
|
||
the cms image from `./core`; `cms/openbureau.config.js` (CMS_CONFIG) carries the
|
||
content model; old vendored cms/api+cms/admin removed. DATABASE_URL left unset (the
|
||
stack's migrate service owns db/schema.sql incl. dialog). Deployed to dev CT 134
|
||
and FULL write-verified: login/list/edit/preview/publish/dialog all identical, no
|
||
git divergence (GIT_PUBLISH=false). Rollback image `cms-cms:rollback` kept.
|
||
9. [DONE — LIVE on core (Variant A)] kgva consumes core. In
|
||
`karim/kgva` (git.kgva.ch, main 51a94b8): `cms/core` = core subtree, `cms/kgva.config.js`
|
||
(project/page/section, auth:'local', no plugins), `docker-compose.yml` builds the core
|
||
image (Hugo 0.163.3, no VITE_SUPABASE_URL → local-auth admin), `cms/seed-admin.mjs`,
|
||
updated deploy/provision-lxc.sh + ADMIN.md. VERIFIED DB-less end-to-end on real content
|
||
(staged on CT134): local login → schema editor → 42 entries (de/en) → stats →
|
||
create/preview/cleanup; image galleries preserved. The LIVE site is UNCHANGED — CT130
|
||
still deploys statically (timer → `hugo` → nginx; hugo ignores cms/); the CMS is dormant.
|
||
PENDING (deliberate, his call — repoints a live domain): provision a kgva-core container
|
||
(deploy/provision-lxc.sh, free CTID + temp IP) and decide topology — CMS serves the site
|
||
(Caddy → it) OR CMS edits + GIT_PUBLISH=true commits to git and the CT130 static timer
|
||
keeps serving. Then Caddy cutover per deploy/README. openbureau also not yet redeployed
|
||
with the schema editor (stage 6 note).
|
||
|
||
## How to run / test
|
||
openbureau CMS stack: `OPENBUREAU/cms/docker-compose.yml` (Node api + Hugo +
|
||
Supabase: postgres/kong/gotrue). Env in `cms/.env` (see `.env.example`):
|
||
SUPABASE_URL/SERVICE_KEY, JWT_SECRET, ANON_KEY, SERVICE_ROLE_KEY (derive via
|
||
`scripts/generate-keys.mjs`), ADMIN_EMAILS, SITE_URL, GIT_*. For core add
|
||
`CMS_CONFIG` (path to the site config) and `SITE_DIR` (site repo mount, default
|
||
`/site`). Admin at `/admin`, preview `/_preview`, publish builds `public/`.
|
||
|
||
## Infra access (NO secrets in this file)
|
||
- Proxmox node `192.168.1.2`, user `root`. Password: **ask the user** (provided ad
|
||
hoc, not stored). No SSH key installed — use SSH_ASKPASS, or install a key first.
|
||
- Gitea = container 120 (`pct exec 120 …`), ROOT_URL `git.openbureau.ch`, sqlite at
|
||
`/var/lib/gitea/data/gitea.db`. Make a token:
|
||
`pct exec 120 -- su gitea -s /bin/bash -c "/usr/local/bin/gitea admin user
|
||
generate-access-token --username karim --scopes all --token-name X --raw
|
||
--config /etc/gitea/app.ini"`. Keep token `kgva-deploy`. Delete tokens via sqlite
|
||
(`DELETE FROM access_token WHERE name LIKE '…'`) since the API needs the account
|
||
password.
|
||
- Push without local git auth: tar → scp to node → `pct push 120` into the gitea
|
||
container → `git push http://karim:<token>@127.0.0.1:3000/karim/<repo>.git`.
|
||
- kgva site deploy: container 130 (kgva-website). nginx serves
|
||
`/var/www/karimgabrielevarano.xyz` (= ZFS `tank/kgva-website`, owned uid 100000).
|
||
systemd timer runs `/opt/kgva-deploy.sh` every 60s: pull `karim/kgva` (token
|
||
`kgva-deploy`) → `hugo` build → copy into webroot. So a push to `karim/kgva` is
|
||
live in ~1 min. Self-hosted video at `/var/www/media` (nginx `/media/` alias,
|
||
outside the build). Public TLS for the domain terminates upstream → 192.168.1.130:80.
|
||
|
||
## Watch out
|
||
- Gitea does NOT send CORS on its OAuth token endpoint → a browser git-CMS
|
||
(Decap-style) can't auth without a same-origin proxy. openbureau-core uses its own
|
||
Supabase auth, so this doesn't affect it; it's why kgva edits go via git push.
|
||
- Hugo versions: openbureau CMS bundles 0.161.1; the kgva site needs 0.163.3
|
||
features. Mind the version a core instance builds with.
|