Why our refund flow never asks the model for permission
Language models are good at conversation and bad at policy. Here is how we split the two, and what it did to our error rate.
By Tomasz Wrona
Blog
Engineering notes, research and playbooks from the team building Synth.
Grading every conversation: what changed when we stopped sampling
Reviewing fifty tickets a week felt rigorous. Scoring all of them showed us how much the sample was hiding.
Sub-second voice: where the 800 milliseconds go
People notice a pause on the phone at around a second. This is the latency budget we work to, stage by stage.
A five-week launch plan for your first support agent
The plan we run with every new customer, week by week, including what we check before real traffic arrives.
Writing rubrics a reviewer model can actually apply
Most quality rubrics are written for people who already know what good looks like. Here is how to write one for a model.
Routing each step to a different model
One model per agent is simple and expensive. One model per step is cheaper, faster and easier to change. Here is how routing works.
Hand-offs are a feature, not a failure
An agent that never hands over is either doing very easy work or doing hard work badly. How we design the moment it passes to a person.
How to build a regression suite from real transcripts
Synthetic test cases miss the strange things customers actually say. A step-by-step guide to turning your history into tests.
Nine thousand rebookings before breakfast
When a winter storm grounded two hundred flights, an airline's agent handled the morning. What we saw from the inside.