Update all documentation to current MySQL + React SPA + vm_api.py architecture
README, docs/{ARCHITECTURE,CONFIGURATION,INSTALL,OPERATIONS,SECURITY,TESTING}
and CHANGELOG were stale: they described the original SQLite + CardDAV +
zero-JS vm_web portal. Brought them in line with the actual code:
- MySQL (PyMySQL) as the default store; SQLite fallback only
- vm_contacts resolves from the MySQL contacts table (file/google/carddav removed)
- React SPA frontend + vm_api.py JSON API (:8098); vm_web.py now serves /audio/
- Apache vhost proxies /api/ -> :8098, /audio/ -> :8099, serves the SPA
- CSP relaxed to script-src 'self' 'unsafe-inline' (UI is now JavaScript)
- Added CHANGELOG 1.2.0 entry for the React/vm_api frontend work
This commit is contained in:
@ -1,7 +1,8 @@
|
||||
# Architecture
|
||||
|
||||
How the pieces fit and *why* they are shaped the way they are. Read this
|
||||
before changing anything in `vm_mailcmd.py` or `vm_store.py`.
|
||||
before changing anything in `vm_mailcmd.py`, `vm_store.py`, `vm_api.py` or
|
||||
`vm_contacts.py`.
|
||||
|
||||
---
|
||||
|
||||
@ -13,12 +14,12 @@ Asterisk ──(RFC822 on stdin)──▶ vm_mailcmd.py
|
||||
┌──────────┬───────────────┼───────────────┬───────────────┐
|
||||
▼ ▼ ▼ ▼ ▼
|
||||
transcribe summarise resolve store telegram
|
||||
(whisper) +intents contact (sqlite) (DM)
|
||||
(whisper) +intents contact (MySQL) (DM)
|
||||
│ │ │ │ │
|
||||
└────▶ build multipart email ─▶ Postfix :25
|
||||
(plain + HTML + audio attached)
|
||||
│
|
||||
└──▶ vm_store.add_message() (best-effort)
|
||||
└──▶ vm_store.add_message() (best-effort, MySQL)
|
||||
└──▶ vm_telegram.send() (best-effort)
|
||||
```
|
||||
|
||||
@ -34,6 +35,58 @@ silently swallow a voicemail.
|
||||
|
||||
---
|
||||
|
||||
## Storage: MySQL, with a SQLite escape hatch
|
||||
|
||||
`vm_store.py` is backend-agnostic. It defaults to **MySQL (PyMySQL)** when a
|
||||
credentials source is present (`/opt/vm-transcribe/db_secret`, mode `640
|
||||
root:root`, or `VM_MYSQL_*` env vars). If neither exists it falls back to the
|
||||
original SQLite file (`VM_DB`).
|
||||
|
||||
The same `asterisk` MySQL database is shared with Asterisk's own CDR
|
||||
(`cdr_adaptive_odbc` + `res_odbc` + the MariaDB ODBC driver), so voicemail
|
||||
metadata and call records live side by side. This is why the migration kept the
|
||||
`asterisk` DB and a least-privilege `asterisk` user rather than spinning up a
|
||||
separate instance.
|
||||
|
||||
The connection object returned by `connect()` mimics the `sqlite3` cursor surface
|
||||
callers use (`.execute()`/`.executemany()`, `.commit()`, `.close()`,
|
||||
`.fetchone()`, `.fetchall()`, `lastrowid`, and `row['col']` access), so the rest
|
||||
of the code is written once and works against either backend.
|
||||
|
||||
---
|
||||
|
||||
## Two backends in front of one store
|
||||
|
||||
- **`vm_api.py` (FastAPI, :8098)** — the JSON API consumed by the React SPA.
|
||||
Login/logout, message list, mark-read, delete, settings, and full contacts
|
||||
CRUD (`GET/POST /api/contacts`, `GET/PUT/DELETE /api/contacts/{id}`,
|
||||
`/api/contacts/import_vcf`, `/api/contacts/history/{number}`,
|
||||
`/api/add_contact`, `/api/contacts/delete_all`). Apache proxies `/api/` here.
|
||||
- **`vm_web.py` (FastAPI, :8099)** — the original server-rendered HTML portal.
|
||||
It still runs and now primarily serves `/audio/` playback and `/healthz`;
|
||||
Apache proxies `/audio/` here. The interactive UI has moved to the React SPA.
|
||||
|
||||
Both read and write the same MySQL `messages` / `contacts` tables, so the React
|
||||
UI and the legacy endpoints never disagree.
|
||||
|
||||
---
|
||||
|
||||
## Frontend: React SPA + Vite
|
||||
|
||||
The portal UI is a React app (`/home/jp/Work/voicemail-ui`) using
|
||||
`react-router-dom`, TanStack Query, and a Vite build. It is built to `dist/` and
|
||||
rsync'd to `/home/txt3/domains/vm.txt3.net/public_html/`. Apache serves the
|
||||
static build and uses an SPA fallback (any non-`/api/`, non-`/audio/` path
|
||||
returns `index.html`). All data goes through `/api/` → `vm_api.py`.
|
||||
|
||||
Because the UI is JavaScript, the Apache CSP is **relaxed** to
|
||||
`script-src 'self' 'unsafe-inline'` (see SECURITY.md). The dark "cyber" theme
|
||||
uses neon accents; callers get a dropdown (⋮) on each message/contact row with
|
||||
**Call back** (`tel:` link from sanitised digits) and **Lookup number** (opens a
|
||||
who-called.co.uk search in a new tab, using the 0-prefixed UK number).
|
||||
|
||||
---
|
||||
|
||||
## Why faster-whisper, CPU, int8, base.en
|
||||
|
||||
- **faster-whisper** (CTranslate2) is dramatically faster than OpenAI's
|
||||
@ -74,23 +127,28 @@ A stored path like `…/INBOX/msg0042.wav` therefore points at a *different*
|
||||
recording a week later. So:
|
||||
|
||||
- On store, the audio is copied to
|
||||
`audio/<sha256[:2]>/<sha256>.<ext>` (blake2/sha256 in `vm_store`).
|
||||
`audio/<sha256[:2]>/<sha256>.<ext>`.
|
||||
- The spool path is kept only as a *delete hint*.
|
||||
- On delete, the blob is unlinked only if no other row references that hash.
|
||||
|
||||
This decouples the portal's playback from Asterisk's ever-shifting filenames.
|
||||
This decouples the portal's playback (served by `vm_web.py` :8099 via
|
||||
`/audio/`) from Asterisk's ever-shifting filenames.
|
||||
|
||||
---
|
||||
|
||||
## Why the portal is its own service behind Apache
|
||||
## Contacts: MySQL only
|
||||
|
||||
- A long-running `uvicorn` is the natural fit; Asterisk's `mailcmd` model
|
||||
(one short-lived process per call) is wrong for a web app.
|
||||
- It binds **127.0.0.1:8099 only** (`IPAddressAllow=localhost` in the unit), so
|
||||
the only way in is the Apache reverse proxy, which terminates TLS and enforces
|
||||
headers.
|
||||
- Running as the `asterisk` user means it can read `voicemail.conf` (for PIN
|
||||
auth) and the spool (for future direct-file features) without widening root.
|
||||
`vm_contacts.py` resolves caller IDs **exclusively** from the MySQL `contacts`
|
||||
table. The older `file` / `google` / `carddav` backends were removed.
|
||||
|
||||
- Numbers are stored canonical E.164; `normalize_uk()` collapses `+44…` and
|
||||
`0…` (and `0044…`) forms so both match.
|
||||
- Matching uses the full digit string (up to 15 digits), not just last-9, and
|
||||
guards against emoji/placeholder contacts shadowing real names.
|
||||
- Creating/updating a contact backfills `messages.contact_name` /
|
||||
`contact_email` for every callerid that matches (spaces/quotes stripped before
|
||||
the LIKE).
|
||||
- `vm_import_contacts.py` loads a vCard/CSV export into the table.
|
||||
|
||||
---
|
||||
|
||||
@ -98,7 +156,7 @@ This decouples the portal's playback from Asterisk's ever-shifting filenames.
|
||||
|
||||
Users log in with their **mailbox number + existing voicemail PIN** parsed from
|
||||
`/etc/asterisk/voicemail.conf` (`vm_auth.py`). No password database to keep in
|
||||
sync, no new credentials to provision. Sessions are stored in the DB (random
|
||||
sync, no new credentials to provision. Sessions are stored in MySQL (random
|
||||
token, `Secure` cookie, `HttpOnly`, `SameSite=lax`, 12 h) so logout and expiry
|
||||
are enforced server-side, not just by deleting a cookie.
|
||||
|
||||
@ -132,6 +190,7 @@ All audio and transcripts stay on the PBX. The only egress is:
|
||||
|
||||
- **Postfix → your MX** for email (the existing path),
|
||||
- **Telegram API** (only if you enable Telegram),
|
||||
- **Google People API** (only if you enable the `google` contacts backend).
|
||||
- **who-called.co.uk** (only when a user clicks *Lookup number* in the UI — a
|
||||
manual, user-initiated action that opens a new browser tab).
|
||||
|
||||
Nothing is sent to OpenAI, Anthropic, or any transcription vendor.
|
||||
|
||||
Reference in New Issue
Block a user