Case study — On-prem classification
Classifying what the cloud couldn't touch.
A Tier-1 US mobile operator needed to understand what was actually hitting their SMS chatbot — at a volume no human team could review, in data no cloud AI service was allowed to see.
The problem nobody could measure.
A wireless carrier's SMS chatbot was drowning in noise — random text, spam, off-topic messages — and nobody could say how bad it was. An earlier review had estimated that roughly six in ten incoming messages weren't real customer questions at all, against an internal target of no more than two in ten. But an estimate isn't a measurement, and measuring meant reading the traffic.
Reading the traffic was the wall. A few hundred messages took about an hour by hand; the real monthly volume — tens of thousands of messages — would have cost a team two full work-weeks, every month. And the obvious shortcut, sending it all to a cloud AI service to classify, was not slow or expensive. It was illegal.
The compliance wall.
The messages carried regulated customer data — the personal and network information (PII, CPNI) that telecom operators are bound to protect. That data could not be sent to any cloud-based AI service. Not the popular ones everyone reaches for by default. Not encrypted in transit to a compliant region. Not at all.
This is a wall more companies are standing in front of than realise. "Can we use AI on this data" and "can we send this data to a cloud AI vendor" are two different questions — and a lot of organisations sitting on the second "no" have never seriously asked whether the first answer could still be "yes."
It can. A well-tuned open-weight model on dedicated hardware, inside the compliance boundary, is not a downgrade from "real AI." It's the same capability in a different deployment shape — one where the data never has to leave.
What we built.
A two-stage classification pipeline, running entirely on a single local machine with two consumer-grade GPUs — a few thousand dollars of hardware, not a cloud cluster.
The first stage sorts every message: noise, off-topic, or a real customer question. The second stage runs only on the real questions, to determine what they're actually about — so the heavier step never touches data that doesn't need it.
The pipeline is fully automated end to end. It fetches new conversations on a schedule, classifies them, and delivers each day's results — labels and counts, never the messages themselves — the moment they're done. It worked through roughly two months of historical traffic, more than 15,000 messages, day by day, unattended, with zero processing errors. The results feed a live dashboard the client's team opens every day. It isn't a pipeline that produces files nobody reads; it closes the loop.
Privacy was engineered in, not bolted on. Wherever messages are retained for evaluation, they're stored as one-way cryptographic hashes — never raw text. Credentials are hardware-sealed, so even the service account running the pipeline never sees a plaintext key. Nothing about a customer's message — not the text, not the model's own working notes on it — leaves the boundary, in output or in logs.
The whole pipeline — from first line of code to unattended production — took about two weeks.
What the tuning taught us.
Getting reliable classification out of a small local model took real experimentation, and two findings are worth passing on because neither is intuitive.
First: asking the model to explain its reasoning alongside its answer made it measurably less accurate — by several percentage points, reproduced across two runs. Dropping the explanation requirement and keeping the model's job simple was the single biggest accuracy win of the whole tuning process. That's the kind of thing you only find by testing, not by reading the model card.
Second: giving the model more "thinking time" — the extended reasoning mode some models support — didn't help either. It made classification dramatically slower and, on the hardest category, pure noise, noticeably less reliable, with a real jump in outright failures. We turned it off.
Against a human-labeled test set, the tuned pipeline measured around 85% accuracy — with zero processing errors across the full production run.
The result.
The data settled the question the estimate had only raised: the noise problem was real, and worse than assumed. Nearly two-thirds of everything hitting the chatbot wasn't a usable customer question. For the first time, the team had that number on a dashboard, refreshed automatically, instead of as a guess.
And the arithmetic of the build speaks for itself: what would have taken a team two full work-weeks a month by hand now takes about five hours, inside the compliance boundary, with zero errors — on a machine that costs less than a month of the manual alternative.
The interesting constraint here was that the standard playbook was illegal before the project started. What made it work wasn't a bigger model — it was a small model made honest: tested variable by variable, stripped of what didn't help, and wrapped in the automation and privacy discipline that lets it run without anyone having to choose between using AI and staying compliant.
If your data has the second "no."
Telecom, healthcare, finance, insurance, legal — if you're sitting on data that can't go to a cloud AI vendor, that doesn't mean AI is off the table. It means the deployment shape matters.
Lean builders. We deliver.