Telecom / communications · AI Engineering
Shipping an AI call-summary platform to paying customers
Built and shipped the backend for AI Assist, Telavox’s production platform for call transcription and summarisation, and owned the LLM integration architecture behind it.
- Live
- in production for paying customers
- Lead
- tech lead for the AI Assist backend
- GCP
- on Kubernetes with ArgoCD
The challenge
Telavox wanted to give customers AI-generated transcriptions and summaries of their calls. Building a demo is easy. Running it reliably for paying customers is not: audio has to be processed at scale, LLM calls are slow and expensive, and when something goes wrong you need to know why, fast.
The approach
- Owned the LLM integration architecture. Designed how the platform talks to language models, so that models and prompts can change without rewriting the system around them.
- Built the audio processing pipelines. Took calls from raw audio through transcription to summary, designed for production load rather than a handful of test calls.
- Drove down cost and latency. Tuned inference cost and response times under real traffic, not synthetic benchmarks.
- Made AI workloads observable. Set up observability and quality gates on GCP, Kubernetes and ArgoCD, so the team can find and fix production issues quickly.
The outcome
AI Assist is live with paying customers. The team has a platform it can operate and keep improving with confidence, and production issues take far less time to diagnose.