Initial build: Asterisk voicemail transcription + portal
- mailcmd replacement (vm_mailcmd.py): faster-whisper transcription (CPU int8), extractive summary + intent tags + spoken-digit number extraction, multipart/alternative HTML email, fail-safe relay of original message - Telegram DM delivery (vm_telegram.py) with per-mailbox routing - Caller-ID -> name (vm_contacts.py): file / google / carddav backends - SQLite store (vm_store.py) with content-addressed audio - FastAPI portal (vm_web.py): PIN login, list/play/delete, per-user settings, zero JS, loopback-only behind Apache TLS - Backfill importer (vm_import.py) for existing spool recordings - systemd unit, Apache vhost + certbot TLS, install.sh - Docs: INSTALL, CONFIGURATION, ARCHITECTURE, OPERATIONS, SECURITY, TESTING Verified end-to-end on mail.txt3.net: 157 historical messages backfilled, live voicemail -> transcribed -> stored -> visible at https://vm.txt3.net.
This commit is contained in:
162
README.md
Normal file
162
README.md
Normal file
@ -0,0 +1,162 @@
|
||||
# Asterisk Voicemail Transcription & Portal
|
||||
|
||||
Turns Asterisk voicemail into something you can actually read, search and
|
||||
manage:
|
||||
|
||||
- **Transcribes** each recording locally with [faster-whisper](https://github.com/SYSTRAN/faster-whisper) (CPU, no cloud, no API key)
|
||||
- **Summarises** it, tags intent (callback requested, urgent, invoice…) and
|
||||
extracts callback numbers — including spoken-out digits
|
||||
- **Emails** a graphically designed `multipart/alternative` notification
|
||||
(plain + HTML) with the recording attached
|
||||
- **DMs Telegram** optionally, as a playable voice note with the summary
|
||||
- **Resolves caller ID to a name** from Google Contacts or any CardDAV server
|
||||
- **Serves a web portal** where mailbox users log in with their existing phone
|
||||
PIN to read transcripts, play/download recordings, delete messages and
|
||||
manage their own notification settings
|
||||
|
||||
Everything runs on the PBX host. No third-party service sees your voicemail.
|
||||
|
||||
---
|
||||
|
||||
## What it looks like
|
||||
|
||||
**Email notification** — styled HTML card with metadata, summary panel, intent
|
||||
chips, a `tel:` callback link, full transcript and the audio attached. A plain
|
||||
text alternative is always included.
|
||||
|
||||
**Telegram** — voice note (ogg/opus) with the summary as its caption, intent
|
||||
hashtags, and the transcript as a follow-up message.
|
||||
|
||||
**Portal** — one card per voicemail: caller, timestamp, duration, summary,
|
||||
tags, callback link, inline player, collapsible transcript, and Mark read /
|
||||
Download / Delete.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
incoming call
|
||||
│
|
||||
▼
|
||||
┌─────────────┐ voicemail.conf: mailcmd=… vm_mailcmd.py
|
||||
│ Asterisk │ ─────────────────────────────┐
|
||||
└─────────────┘ pipes an RFC822 message │
|
||||
(notification + audio) ▼
|
||||
┌───────────────┐
|
||||
│ vm_mailcmd.py │
|
||||
└───────┬───────┘
|
||||
┌──────────────┬──────────────┬────┴─────────┬──────────────┐
|
||||
▼ ▼ ▼ ▼ ▼
|
||||
faster-whisper summarise() vm_contacts vm_store vm_telegram
|
||||
(transcribe) + intents (name lookup) (SQLite + (voice note
|
||||
+ numbers audio CAS) DM)
|
||||
│ │
|
||||
▼ ▼
|
||||
multipart email ──▶ Postfix :25 ┌──────────────┐
|
||||
│ vm_web.py │
|
||||
│ (FastAPI) │
|
||||
└──────┬───────┘
|
||||
│ :8099 loopback
|
||||
▼
|
||||
Apache (TLS)
|
||||
│
|
||||
▼
|
||||
https://vm.txt3.net
|
||||
```
|
||||
|
||||
**Delivery order is deliberate**: email first, then the database, then Telegram.
|
||||
Each later stage is wrapped so a failure only logs. If anything throws at the
|
||||
top level, the *original* Asterisk notification is relayed unchanged. A
|
||||
voicemail notification is never lost because a summariser or an API failed.
|
||||
|
||||
---
|
||||
|
||||
## Components
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `src/vm_mailcmd.py` | The `mailcmd` — entry point for every voicemail. Orchestrates everything. |
|
||||
| `src/vm_store.py` | SQLite store: schema, settings, sessions, content-addressed audio. |
|
||||
| `src/vm_telegram.py` | Telegram delivery, per-mailbox routing. |
|
||||
| `src/vm_contacts.py` | Caller-ID → name via local export, Google People API, or CardDAV. |
|
||||
| `src/vm_web.py` | FastAPI portal: login, list, play, delete, settings. |
|
||||
| `src/vm_auth.py` | Parses `voicemail.conf` so users log in with their phone PIN. |
|
||||
| `src/vm_import.py` | Backfills existing spool recordings into the database. |
|
||||
| `src/vm_tg_setup.py` | Helper: discover Telegram chat IDs, test a route. |
|
||||
|
||||
---
|
||||
|
||||
## Install
|
||||
|
||||
See **[docs/INSTALL.md](docs/INSTALL.md)** for the full walkthrough. Short version:
|
||||
|
||||
```bash
|
||||
sudo scripts/install.sh # venv, deps, model cache, mailcmd wiring
|
||||
sudo cp systemd/vm-portal.service /etc/systemd/system/
|
||||
sudo systemctl enable --now vm-portal
|
||||
# then follow docs/INSTALL.md §5 for the Apache vhost + TLS
|
||||
```
|
||||
|
||||
Backfill your existing voicemails:
|
||||
|
||||
```bash
|
||||
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
|
||||
/opt/vm-transcribe/vm_import.py --dry-run # preview
|
||||
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
|
||||
/opt/vm-transcribe/vm_import.py # do it
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Documentation
|
||||
|
||||
| Document | Contents |
|
||||
|---|---|
|
||||
| [docs/INSTALL.md](docs/INSTALL.md) | Step-by-step install, including TLS ordering |
|
||||
| [docs/CONFIGURATION.md](docs/CONFIGURATION.md) | Every config file and option |
|
||||
| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Design decisions and why |
|
||||
| [docs/OPERATIONS.md](docs/OPERATIONS.md) | Day-to-day running, backup, troubleshooting |
|
||||
| [docs/SECURITY.md](docs/SECURITY.md) | Threat model, hardening, privacy |
|
||||
| [docs/TESTING.md](docs/TESTING.md) | How to verify each part |
|
||||
| [CHANGELOG.md](CHANGELOG.md) | Version history |
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
- Asterisk with `app_voicemail` (file-based spool, not ODBC/IMAP storage)
|
||||
- Python 3.9+
|
||||
- `ffmpeg` (Telegram voice-note transcoding; `sox` optionally for gsm)
|
||||
- An MTA listening on `localhost:25` (Postfix here)
|
||||
- ~200 MB disk for the whisper `base.en` model
|
||||
- Apache with `proxy`, `proxy_http`, `headers`, `rewrite`, `ssl` for the portal
|
||||
|
||||
Runs comfortably on 4 CPU cores with no GPU: ~8 s to transcribe 23 s of audio.
|
||||
|
||||
---
|
||||
|
||||
## Design notes worth knowing
|
||||
|
||||
**Summarisation is local and extractive.** Frequency-scored sentence selection
|
||||
with position/digit weighting, plus regex intent tags. No LLM, by choice — it
|
||||
keeps voicemail content on your own hardware. `summarise()` in
|
||||
`vm_mailcmd.py` is a single swap-in point if you want an abstractive model.
|
||||
|
||||
**Audio is content-addressed, not referenced by spool path.** Asterisk renumbers
|
||||
`msgNNNN` files when a message is deleted, so a stored path silently starts
|
||||
pointing at the wrong recording. Recordings are copied to
|
||||
`audio/<sha[:2]>/<sha>.wav`.
|
||||
|
||||
**App passwords cannot read Google Contacts.** Google disabled basic auth for
|
||||
CardDAV/CalDAV/IMAP/SMTP/POP on 2024-09-30. Use the `file` backend (a vCard
|
||||
export) or the `google` backend (OAuth). The `carddav` backend with an app
|
||||
password works for Nextcloud, Fastmail and iCloud.
|
||||
|
||||
**The portal ships zero JavaScript**, which lets its CSP be `script-src 'none'`.
|
||||
|
||||
---
|
||||
|
||||
## Licence
|
||||
|
||||
MIT — see [LICENSE](LICENSE).
|
||||
Reference in New Issue
Block a user