asterisk-voicemail/docs/INSTALL.md

Installation

Written against Debian 12 + Asterisk 20 + Apache 2.4 + Postfix + MySQL/MariaDB, running under Virtualmin. Adjust paths if your layout differs.

Everything installs to /opt/vm-transcribe with audio in /var/lib/vm-transcribe. Nothing is installed into system Python.

Frontend lives in a separate repo. This backend repo ships a synced copy under frontend/. Build the real app in /home/jp/Work/voicemail-ui and rsync the result to the web root (see §6). Keep the two in sync with rsync -a /home/jp/Work/voicemail-ui/ frontend/ then commit.

---

0. Prerequisites

# Asterisk voicemail must use file-based storage (the default), not ODBC/IMAP
grep -E '^(odbcstorage|imapserver)' /etc/asterisk/voicemail.conf   # expect nothing

# tools
sudo apt install ffmpeg sox python3-venv          # sox optional (gsm)
sudo apt install default-mysql-server             # or mariadb-server
which certbot                                     # for the portal's TLS
node --version && npm --version                   # Node 18+ for the frontend

# an MTA on localhost:25
sudo ss -ltnp | grep ':25 '

Check disk space before you start. A full /var makes Postfix reject all mail with 452 4.3.1 Insufficient system storage, which looks like a bug in this pipeline but is not:

df -h /var

---

1. MySQL (storage + contacts + CDR)

The backend shares the Asterisk asterisk database. Create a least-privilege user and a credentials file:

sudo mysql -e "CREATE USER IF NOT EXISTS 'asterisk'@'localhost' IDENTIFIED BY 'PICK_A_STRONG_PASSWORD';"
sudo mysql -e "GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX,ALTER ON asterisk.* TO 'asterisk'@'localhost';"
sudo mysql -e "FLUSH PRIVILEGES;"

Write /opt/vm-transcribe/db_secret (mode 640 root:root):

sudo install -o root -g root -m 640 /dev/null /opt/vm-transcribe/db_secret
sudo tee /opt/vm-transcribe/db_secret >/dev/null <<'EOF'
MYSQL_HOST=localhost
MYSQL_USER=asterisk
MYSQL_PASSWORD=PICK_A_STRONG_PASSWORD
MYSQL_DB=asterisk
EOF

vm_store uses MySQL whenever this file (or VM_MYSQL_* env vars) exists; otherwise it falls back to SQLite. The schema is created on first connect.

(Optional) To load call records into the same DB, wire Asterisk CDR via cdr_adaptive_odbc + res_odbc + the MariaDB ODBC driver, then run src/vm_backfill_cdr.py to import Master.csv history.

---

2. Core pipeline (mailcmd + venv)

The install.sh script builds the venv, installs faster-whisper, deploys the mailcmd scripts, seeds telegram.conf, pre-caches the whisper model as the asterisk user, and rewrites voicemail.conf's mailcmd=.

install.sh reads sources from /home/jp/asterisk-vm (the deploy path in AGENTS.md). Either symlink/clobber that path to this repo, or edit the SRC= line at the top of scripts/install.sh before running.

sudo scripts/install.sh

Verify the wiring:

grep '^mailcmd' /etc/asterisk/voicemail.conf
# mailcmd=/opt/vm-transcribe/venv/bin/python3 /opt/vm-transcribe/vm_mailcmd.py

Test it before trusting it

Build a fake notification and pipe it through as the asterisk user — testing as yourself hides permission problems:

cd /home/jp/Work/asterisk-voicemail
python3 tests/make_test_mail.py /path/to/some.wav you@example.com > /tmp/t.eml
sudo chmod 644 /tmp/t.eml
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
     /opt/vm-transcribe/vm_mailcmd.py < /tmp/t.eml
sudo tail -5 /var/log/asterisk/vm_mailcmd.log

You should see transcribed … chars then sent enriched notification to …. Confirm the mail actually left:

sudo grep "to=<you@example.com>" /var/log/mail.log | tail -1   # status=sent

Also verify the fail-safe — a message with no audio must relay unchanged rather than erroring:

printf 'From: a@b\nTo: you@example.com\nSubject: no audio\n\nbody\n' > /tmp/n.eml
sudo chmod 644 /tmp/n.eml
sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
     /opt/vm-transcribe/vm_mailcmd.py < /tmp/n.eml
# log: "no audio attachment; relaying original"

---

3. Telegram (optional)

1. Create a bot: message @BotFather → /newbot → copy the token.
2. Put it in /opt/vm-transcribe/telegram.conf (sudo, mode 640 root:asterisk).
3. Each recipient must message the bot once (/start) — Telegram forbids bots from initiating conversations.
4. Discover chat IDs and test:

sudo /opt/vm-transcribe/venv/bin/python3 /opt/vm-transcribe/vm_tg_setup.py ids
sudo /opt/vm-transcribe/venv/bin/python3 /opt/vm-transcribe/vm_tg_setup.py test 1001

The test subcommand sends a real DM with a playable voice note and exits non-zero on failure. See CONFIGURATION.md for per-mailbox routing.

---

4. Contacts (MySQL)

Contacts live in the MySQL contacts table — there is no file/Google/CardDAV backend anymore. Load your address book from a vCard/CSV export:

sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
     /opt/vm-transcribe/vm_import_contacts.py /path/to/contacts.vcf

Confirm lookup works:

sudo -u asterisk /opt/vm-transcribe/venv/bin/python3 \
     /opt/vm-transcribe/vm_contacts.py '<07700900123>'

---

5. Backend services

Install both units (API + legacy portal/audio):

sudo install -d -o asterisk -g asterisk -m 750 \
     /var/lib/vm-transcribe /var/lib/vm-transcribe/audio
sudo /opt/vm-transcribe/venv/bin/pip install fastapi 'uvicorn[standard]' python-multipart pymysql

# API unit reads MySQL creds from api.env
sudo install -o root -g root -m 640 /opt/vm-transcribe/db_secret /opt/vm-transcribe/api.env
sudo install -o root -g root -m 644 systemd/vm-api.service /etc/systemd/system/
sudo install -o root -g root -m 644 systemd/vm-portal.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now vm-api vm-portal
sudo systemctl is-active vm-api vm-portal

curl -s http://127.0.0.1:8098/api/healthz      # {"ok":true,...}
curl -s http://127.0.0.1:8099/healthz          # {"ok":true,"messages":N}

The services run as asterisk (so they can read voicemail.conf and the spool) and bind loopback only — IPAddressAllow=localhost means Apache is the only possible client.

---

6. React frontend (build + deploy)

The interactive UI is a React SPA built in /home/jp/Work/voicemail-ui:

cd /home/jp/Work/voicemail-ui
npm install
npm run build                       # outputs dist/

Deploy the static build to the web root (Apache serves it and proxies /api/ to :8098 and /audio/ to :8099):

sudo -A rsync -a --exclude node_modules --exclude dist \
     /home/jp/Work/voicemail-ui/ /home/txt3/domains/vm.txt3.net/public_html/

Keep this repo's copy in sync for documentation/commit purposes:

rsync -a /home/jp/Work/voicemail-ui/ frontend/      # then git commit under frontend/

---

7. Apache vhost + TLS

Order matters. Installing a vhost that references a cert which does not yet exist breaks apache2ctl configtest, which blocks reloads for every site on the box. So: DNS → cert → full vhost.

7a. DNS

dig +short vm.txt3.net A       # must return your ORIGIN ip

Behind Cloudflare, set the record to DNS only (grey cloud) first, or the HTTP-01 challenge is intercepted by the edge.

7b. Minimal :80 vhost, so certbot can answer the challenge

sudo install -d -o www-data -g www-data -m 755 /var/www/vm.txt3.net/public_html
sudo cp apache/vm.txt3.net-step1.conf /etc/apache2/sites-available/vm.txt3.net.conf
# edit ServerName / paths / IPs to match your host
sudo a2ensite vm.txt3.net
sudo apache2ctl configtest && sudo systemctl reload apache2

7c. Issue the certificate

sudo certbot certonly --webroot -w /var/www/vm.txt3.net/public_html \
     -d vm.txt3.net --cert-name vm.txt3.net

7d. Full vhost — HTTPS + reverse proxy

sudo a2enmod proxy proxy_http headers rewrite ssl
sudo cp apache/vm.txt3.net.conf /etc/apache2/sites-available/vm.txt3.net.conf
# edit ServerName, cert paths, IPs, DocumentRoot
sudo install -d /var/www/vm.txt3.net/public_html/.well-known/acme-challenge
sudo apache2ctl configtest && sudo systemctl reload apache2

The shipped apache/vm.txt3.net.conf already proxies /api/ → :8098, /audio/ → :8099, serves the React dist as DocumentRoot, and adds an SPA fallback (RewriteRule ^ /index.html).

7e. Verify through the real hostname

Not localhost — the whole point is to exercise Apache, TLS and the proxy:

R="--resolve vm.txt3.net:443:ORIGIN.IP"
curl -s $R https://vm.txt3.net/api/healthz
curl -s $R -o /dev/null -w '%{http_code}\n' https://vm.txt3.net/login
curl -s $R -D- -o /dev/null https://vm.txt3.net/login | grep -iE 'strict-transport|content-security|x-frame'
# SPA fallback
curl -s $R -o /dev/null -w '%{http_code}\n' https://vm.txt3.net/some/route

Behind a CDN, curl https://host/ tests the CDN, not your origin. Always --resolve to the origin IP when verifying a change.

---

8. Backfill existing voicemails

V=/opt/vm-transcribe/venv/bin/python3
sudo -u asterisk $V /opt/vm-transcribe/vm_import.py --dry-run          # preview
sudo -u asterisk $V /opt/vm-transcribe/vm_import.py --limit 5          # trial
sudo -u asterisk $V /opt/vm-transcribe/vm_import.py                    # all
sudo -u asterisk $V /opt/vm-transcribe/vm_backfill_cdr.py              # CDR history

Idempotent — safe to re-run; it resumes rather than duplicating. Expect a no_speech count: 44-byte WAVs are hung-up calls with no audio, not failures.

Reckon on ~1.5 s per message plus transcription time; 157 messages took 3m40s on 4 cores.

---

9. Log in

Browse to https://vm.txt3.net/ and log in with a mailbox number and its existing voicemail PIN from voicemail.conf. No new passwords are created.

---

Uninstall

sudo systemctl disable --now vm-api vm-portal
sudo rm /etc/systemd/system/vm-api.service /etc/systemd/system/vm-portal.service
sudo systemctl daemon-reload
sudo a2dissite vm.txt3.net && sudo systemctl reload apache2
# restore the original mailcmd
sudo cp /etc/asterisk/voicemail.conf.bak-<timestamp> /etc/asterisk/voicemail.conf
sudo asterisk -rx 'voicemail reload'
sudo rm -rf /opt/vm-transcribe          # keep /var/lib/vm-transcribe for the data
# MySQL: DROP USER 'asterisk'@'localhost';  (optionally DROP DATABASE asterisk;)