asterisk-voicemail/docs/ARCHITECTURE.md

# Architecture

How the pieces fit and *why* they are shaped the way they are. Read this
before changing anything in `vm_mailcmd.py` or `vm_store.py`.

---

## Flow

```
Asterisk  ──(RFC822 on stdin)──▶  vm_mailcmd.py
                                       │
            ┌──────────┬───────────────┼───────────────┬───────────────┐
            ▼          ▼               ▼               ▼               ▼
        transcribe  summarise      resolve        store           telegram
        (whisper)   +intents       contact        (sqlite)         (DM)
            │          │               │              │               │
            └────▶ build multipart email ─▶ Postfix :25
                       (plain + HTML + audio attached)
                                       │
                                       └──▶ vm_store.add_message()  (best-effort)
                                       └──▶ vm_telegram.send()      (best-effort)
```

**Email is sent before anything else.** The portal and Telegram are conveniences;
the email is the contract with the user. If the database is down or Telegram's
API throws, the mail has already gone. Each later stage is in its own `try/except`
that only logs. The reverse — a transcription error blocking the email — never
happens, because the email builder is given whatever text we have, even `""`.

At the very top, the whole `main()` body is wrapped so that *any* unexpected
exception relays the **original** Asterisk message unchanged. Nothing we add can
silently swallow a voicemail.

---

## Why faster-whisper, CPU, int8, base.en

- **faster-whisper** (CTranslate2) is dramatically faster than OpenAI's
  `whisper` on CPU — ~8 s for 23 s of 8 kHz audio here, versus minutes.
- **CPU/int8** because this host has no GPU and the voicemail rate is low. A
  GPU would be idle 99.9 % of the time.
- **base.en** is plenty for clear phone audio; `small.en` only if you see
  accuracy problems with names/numbers. Set `VM_WHISPER_MODEL`.

The model is loaded once per process. Asterisk fires a fresh `mailcmd` per
voicemail, so there is a ~1 s model-load cost *per message* — acceptable at PBX
volumes, and the model is pre-cached at install so the first call doesn't also
pay a download.

---

## Why the summary is extractive, not an LLM

The user explicitly declined both OpenAI and Ollama. `summarise()` is:

1. split into sentences,
2. score each by word frequency (stop-words removed) plus a boost for
   position (first/last sentence) and for sentences containing digits,
3. pick the top sentences, ordered as spoken, capped at ~3 and a length limit,
4. wrap with regex intent tags (`Call back requested`, `Urgent`,
   `Payment / invoice`, …) and number extraction.

It reads well on phone voicemail and needs no network. The single function
`summarise()` is a deliberate **swap-in point** — replace it with a local model
or an API call without touching the rest of the pipeline.

---

## Why audio is content-addressed

Asterisk **renumbers** `msgNNNN` files when a message is deleted from a folder.
A stored path like `…/INBOX/msg0042.wav` therefore points at a *different*
recording a week later. So:

- On store, the audio is copied to
  `audio/<sha256[:2]>/<sha256>.<ext>` (blake2/sha256 in `vm_store`).
- The spool path is kept only as a *delete hint*.
- On delete, the blob is unlinked only if no other row references that hash.

This decouples the portal's playback from Asterisk's ever-shifting filenames.

---

## Why the portal is its own service behind Apache

- A long-running `uvicorn` is the natural fit; Asterisk's `mailcmd` model
  (one short-lived process per call) is wrong for a web app.
- It binds **127.0.0.1:8099 only** (`IPAddressAllow=localhost` in the unit), so
  the only way in is the Apache reverse proxy, which terminates TLS and enforces
  headers.
- Running as the `asterisk` user means it can read `voicemail.conf` (for PIN
  auth) and the spool (for future direct-file features) without widening root.

---

## Auth model

Users log in with their **mailbox number + existing voicemail PIN** parsed from
`/etc/asterisk/voicemail.conf` (`vm_auth.py`). No password database to keep in
sync, no new credentials to provision. Sessions are stored in the DB (random
token, `Secure` cookie, `HttpOnly`, `SameSite=lax`, 12 h) so logout and expiry
are enforced server-side, not just by deleting a cookie.

Every data query is filtered by the session's mailbox. Cross-mailbox access must
404 — there is a test for this in [TESTING.md](TESTING.md).

---

## Lockout

Voicemail PINs are short (often 4 digits) and the portal is internet-facing, so
failed logins are throttled in-process: 5 per `(mailbox, IP)` then 15 minutes.
State is in memory by design (single worker). A restart clears it — fine, because
a restart is an admin action, not an attacker one.

---

## Telegram routing

`telegram.conf` is re-read on every voicemail (no reload). Resolution order:
`[mailbox:N]` `chat_id` → `default_chat_id` → nothing. A chat id may fan out to
several recipients (comma list) or be a group (negative). Per-mailbox sections
can override any global (`send_audio`, `send_transcript`). Delivery degrades
`sendVoice → sendDocument → sendMessage` so a summary always gets through.

---

## Privacy posture

All audio and transcripts stay on the PBX. The only egress is:

- **Postfix → your MX** for email (the existing path),
- **Telegram API** (only if you enable Telegram),
- **Google People API** (only if you enable the `google` contacts backend).

Nothing is sent to OpenAI, Anthropic, or any transcription vendor.