Initial build: Asterisk voicemail transcription + portal
- mailcmd replacement (vm_mailcmd.py): faster-whisper transcription (CPU int8), extractive summary + intent tags + spoken-digit number extraction, multipart/alternative HTML email, fail-safe relay of original message - Telegram DM delivery (vm_telegram.py) with per-mailbox routing - Caller-ID -> name (vm_contacts.py): file / google / carddav backends - SQLite store (vm_store.py) with content-addressed audio - FastAPI portal (vm_web.py): PIN login, list/play/delete, per-user settings, zero JS, loopback-only behind Apache TLS - Backfill importer (vm_import.py) for existing spool recordings - systemd unit, Apache vhost + certbot TLS, install.sh - Docs: INSTALL, CONFIGURATION, ARCHITECTURE, OPERATIONS, SECURITY, TESTING Verified end-to-end on mail.txt3.net: 157 historical messages backfilled, live voicemail -> transcribed -> stored -> visible at https://vm.txt3.net.
This commit is contained in:
159
docs/TESTING.md
Normal file
159
docs/TESTING.md
Normal file
@ -0,0 +1,159 @@
|
||||
# Testing
|
||||
|
||||
How to verify each part of the system. Do the mailcmd tests **as the `asterisk`
|
||||
user**, not as yourself — that's where permission bugs hide.
|
||||
|
||||
---
|
||||
|
||||
## 0. Pre-checks
|
||||
|
||||
```bash
|
||||
df -h /var # must not be full (Postfix 452)
|
||||
grep '^mailcmd' /etc/asterisk/voicemail.conf
|
||||
sudo systemctl is-active vm-portal
|
||||
curl -s http://127.0.0.1:8099/healthz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 1. Unit tests (no sudo, no asterisk)
|
||||
|
||||
```bash
|
||||
cd /home/jp/Work/asterisk-voicemail
|
||||
./venv/bin/python tests/test_telegram.py # routing, caption clipping, opus transcode
|
||||
./venv/bin/python tests/test_contacts.py # vCard/CSV parse, digit-normalised match
|
||||
```
|
||||
|
||||
`test_telegram.py` asserts: per-mailbox → one chat; comma list → N chats;
|
||||
group id negative; unmapped → default; `enabled=no` → no route; caption clipped
|
||||
to 1017 chars on a word boundary when over 1024; a 374 KB wav → 64 KB `OggS`
|
||||
opus.
|
||||
|
||||
`test_contacts.py` asserts: `+447700900123`, `07700900123`, `447700900123` all
|
||||
match one contact (last-9-digit key); multi-TEL cards; Google CSV `:::` split.
|
||||
|
||||
---
|
||||
|
||||
## 2. Build a fake voicemail and run the real pipeline
|
||||
|
||||
```bash
|
||||
# needs ffmpeg; uses test_vm.wav shipped in the repo (or any wav)
|
||||
python3 tests/make_test_mail.py test_vm.wav you@example.com > /tmp/t.eml
|
||||
sudo chmod 644 /tmp/t.eml
|
||||
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
|
||||
/opt/vm-transcribe/vm_mailcmd.py < /tmp/t.eml
|
||||
sudo tail -5 /var/log/asterisk/vm_mailcmd.log
|
||||
```
|
||||
|
||||
Expect `transcribed … chars` then `sent enriched notification to you@example.com`.
|
||||
Then confirm real delivery:
|
||||
|
||||
```bash
|
||||
sudo grep "to=<you@example.com>" /var/log/mail.log | tail -1 # status=sent
|
||||
```
|
||||
|
||||
### Fail-safe: a message with no audio
|
||||
|
||||
```bash
|
||||
printf 'From: a@b\nTo: you@example.com\nSubject: no audio\n\nbody\n' > /tmp/n.eml
|
||||
sudo chmod 644 /tmp/n.eml
|
||||
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
|
||||
/opt/vm-transcribe/vm_mailcmd.py < /tmp/n.eml
|
||||
# log must show: "no audio attachment; relaying original"
|
||||
```
|
||||
|
||||
### Fail-safe: a thrown error relays the original
|
||||
|
||||
Temporarily point `VM_DB` at an unreadable path; the script should fall back to
|
||||
relaying the original Asterisk message and log `relaying original, mailcmd error`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Telegram (needs a real token)
|
||||
|
||||
```bash
|
||||
sudo /opt/vm-transcribe/venv/bin/python3 /opt/vm-transcribe/vm_tg_setup.py ids
|
||||
# each recipient must have messaged the bot once (/start) to appear
|
||||
sudo /opt/vm-transcribe/venv/bin/python3 /opt/vm-transcribe/vm_tg_setup.py test 1001
|
||||
# sends a real voice note + summary; exits non-zero on any failure
|
||||
```
|
||||
|
||||
Offline, the routing logic is covered by `tests/test_telegram.py` so you don't
|
||||
need a token to verify the config parser.
|
||||
|
||||
---
|
||||
|
||||
## 4. Contacts
|
||||
|
||||
```bash
|
||||
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
|
||||
/opt/vm-transcribe/vm_contacts.py '<07700900123>'
|
||||
# resolves against the configured backend; prints name or the raw caller id
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Portal — auth, authz, playback, delete
|
||||
|
||||
Run the app locally (or against the live service) and exercise it with curl.
|
||||
Use `VM_INSECURE_COOKIE=1` only for plain-HTTP local tests so the `Secure`
|
||||
cookie can be set.
|
||||
|
||||
```bash
|
||||
# login ok
|
||||
curl -s -c /tmp/j -o /dev/null -w '%{redirect_url}\n' \
|
||||
-d 'mailbox=7940&pin=5159' http://127.0.0.1:8099/login
|
||||
|
||||
# wrong pin rejected
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -d 'mailbox=7940&pin=1111' http://127.0.0.1:8099/login
|
||||
|
||||
# authenticated list
|
||||
curl -s -b /tmp/j http://127.0.0.1:8099/ | grep -c 'class="card'
|
||||
|
||||
# audio streams
|
||||
curl -s -b /tmp/j -o /tmp/x.wav http://127.0.0.1:8099/audio/1
|
||||
file /tmp/x.wav # RIFF WAVE, 8000 Hz mono
|
||||
|
||||
# delete
|
||||
curl -s -b /tmp/j -X POST -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8099/delete/1
|
||||
```
|
||||
|
||||
### Authorization (IDOR) — must 404
|
||||
|
||||
A session for mailbox A must never reach mailbox B's data. With a cookie for
|
||||
mailbox 7940, hitting `/audio/<id-owned-by-1001>` and `/delete/<id-owned-by-1001>`
|
||||
must both return **404/303**, never serve or delete the other mailbox's message.
|
||||
Add this assertion whenever you change `vm_store` or `vm_web`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Portal over the real URL (TLS + proxy)
|
||||
|
||||
```bash
|
||||
R="--resolve vm.txt3.net:443:ORIGIN.IP"
|
||||
curl -s $R https://vm.txt3.net/healthz
|
||||
curl -s $R -o /dev/null -w '%{http_code}\n' https://vm.txt3.net/login
|
||||
curl -s $R -D- -o /dev/null https://vm.txt3.net/login | grep -iE 'strict-transport|content-security|x-frame'
|
||||
# http -> https redirect
|
||||
curl -s $R -o /dev/null -w '%{redirect_url}\n' http://vm.txt3.net/
|
||||
```
|
||||
|
||||
Behind a CDN, `--resolve` to the **origin** IP; otherwise you are testing the CDN,
|
||||
not your server.
|
||||
|
||||
---
|
||||
|
||||
## 7. Backfill
|
||||
|
||||
```bash
|
||||
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
|
||||
/opt/vm-transcribe/vm_import.py --dry-run --limit 5
|
||||
# preview: lists 5 messages with caller + date, no writes
|
||||
|
||||
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
|
||||
/opt/vm-transcribe/vm_import.py --limit 3
|
||||
# real: 3 imported, others skipped; re-running is a no-op (idempotent)
|
||||
```
|
||||
|
||||
Verify they appear in the portal and are playable (§5). Expect a `no_speech`
|
||||
count — 44-byte WAVs are hung-up calls, not failures.
|
||||
Reference in New Issue
Block a user