Akoe
2024 — 2025
LLM-powered quality assurance for customer service calls
overview
Overview
Akoe was an LLM-powered quality assurance platform for grading and reviewing customer service calls. I co-founded it, built the core grading engine and the reporting layer on top of it, and owned the AWS infrastructure the whole thing ran on.
Call centers already knew their call quality mattered. What they didn't have was a way to look at more than a sliver of it. Akoe's premise was that if grading a call costs cents instead of a supervisor's afternoon, you stop sampling and start grading everything.
The company was sold in 2025.
problem
The problem
Quality assurance in a call center is a sampling problem. A supervisor listens to a handful of calls per agent per month, fills in a rubric by hand, and extrapolates. The sample is small, the rubric is applied inconsistently between reviewers, and by the time feedback reaches the agent the call is weeks old.
The obvious fix — grade every call — had never been economical. Transcription plus a human reviewer is far too expensive to run across an entire call volume, and pre-LLM automated scoring could only check for keyword presence, which is a poor proxy for whether an agent actually handled a customer well.
The harder constraint was that no two organizations grade calls the same way. A rubric that ships hard-coded is a rubric that fits nobody. Whatever we built had to let an admin define what 'good' meant for their team, and then apply that definition consistently across every call.
architecture
Architecture
- 01
Ingest
Call audio and metadata land in AWS and are queued for processing. Work is decoupled from the request path from the start — grading is a batch concern, not something a user waits on.
- 02
Benchmarks
Admins define benchmarks — the criteria their agents are scored against — through the application. These are data, not code, which is what makes the same engine work across organizations with different standards.
- 03
Grading engine
The core of the product. A Python service scores each call against the customer's configured benchmarks using LLM inference, producing a per-criterion result rather than a single opaque number.
- 04
Insights layer
Automated summary and insights reporting rolls individual gradings up into call-quality trends, and turns those trends into concrete recommendations for agents and for the managers reviewing them.
- 05
Application
A Next.js and TypeScript front end where admins configure benchmarks, agents see their own results, and managers work through the reporting.
- 06
Infrastructure
AWS throughout, owned end to end — the compute behind the grading pipeline, the storage, and the deployment path for the full-stack application.
challenges
What was hard
Making the rubric a product surface
Because benchmarks were admin-defined, the grading prompt could not be a constant. The engine had to compose a scoring pass from configuration written by someone who had never seen a prompt, and still produce results consistent enough that two similar calls scored similarly.
Cost per call is the business model
The entire pitch rested on grading being cheap enough to run on every call rather than a sample. That makes inference cost a product constraint, not an infrastructure detail — it shapes how much of a transcript you send, how many passes you make, and where a cheaper check can stand in for a model call.
Scores nobody trusts are worthless
A number an agent disagrees with and can't interrogate does more harm than no number at all. Grading per-criterion rather than in aggregate was as much about making results arguable as it was about making them precise.
lessons
What I took from it
Configuration is a feature, and it is much harder than the hard-coded version. Letting customers define their own benchmarks was the thing that made Akoe sellable and the thing that made the engine difficult.
When inference cost sits directly under the value proposition, cost work stops being optimization and becomes product work.
Owning the infrastructure as well as the application meant nothing was ever someone else's problem — which is exhausting at the time and enormously clarifying in retrospect.
A small team ships faster than it can decide what to ship. The bottleneck was rarely engineering.
screenshots
Screenshots
screenshot slot
screenshot slot
screenshot slot
stack
Tech stack
next
Future work
The product was sold, so this list is what I would have built next rather than what is planned.
Calibration tooling: let an admin grade a handful of calls by hand and measure how closely the engine agrees, so trust in the rubric is measured rather than assumed.
Real-time assistance during a call rather than review after it.
Pushing more of the scoring work onto cheaper deterministic checks, reserving inference for the criteria that genuinely need judgment.