AI Translation Accuracy: How to Verify Quality, Not Trust a Number

Almost every translation vendor leads with a headline: "98% accurate," "near-human quality," "best-in-class." None of those phrases mean anything on their own. Accuracy is not a single number you can print on a banner — it depends on the language pair, the subject matter, the metric you measured with, and who did the measuring.

This is the part of the industry we think is broken. At Mind.com — the company behind InterMIND — our rule is verifiable over trust-us: quality you can check against a public standard, not a number we ask you to believe. This article is the buyer's-side version of that principle: how to read accuracy claims, and how to verify them yourself.

Why "X% accurate" tells you almost nothing

When a vendor says "94% accuracy," ask four questions before the number means anything:

• Measured how? Automatic metrics like BLEU, chrF, and COMET score machine output against reference translations — and they disagree with each other constantly. A human-evaluation framework like MQM (Multidimensional Quality Metrics) produces a different picture again, because it counts kinds of errors, not surface overlap. A percentage with no metric attached is marketing, not measurement.
• On which language pair? Quality for English↔Spanish is not quality for English↔Japanese, and neither predicts a low-resource pair. An average across "major languages" hides exactly the pair you care about.
• On what content? A model that handles technical documentation cleanly can fall apart on marketing copy, legal text, or idiom. The domain of the test set decides the score.
• Against whose reference, on which test set? A number you cannot trace back to a named, public test set is a number you cannot reproduce — and an unreproducible result is not evidence.

If a claim survives all four questions, it might be useful. Most don't.

What a verifiable measurement actually looks like

The good news: public standards for this already exist. You don't have to take a vendor's word, and you shouldn't.

• Public benchmarks. The annual WMT (Conference on Machine Translation) evaluation campaigns and open test sets such as FLORES-200 let anyone compare systems on the same data, across many languages. Results are published and reproducible.
• Named metrics, stated together. A serious quality claim names its metric and its test set — e.g. "COMET on FLORES-200, EN→DE" — so a third party can re-run it. One metric in isolation is easy to game; reporting several together is harder to fake.
• Human evaluation with a rubric. For anything high-stakes, automatic scores are a screen, not a verdict. MQM-style human review, with a documented error taxonomy, is what tells you whether the output is usable, not just statistically close.

The shift here is simple: stop asking "how accurate are you?" and start asking "against which public standard, and can I reproduce it?"

Where AI translation is strong — and where it isn't

You don't need invented percentages to state what's well established across the field:

• Strong: high-resource language pairs, technical and structured content, consistent terminology at volume, and speed that human workflows can't match.
• Weaker, and still needs a human: creative and marketing copy, humor and idiom, culturally loaded nuance, and legal or medical text where an error carries liability.

Honest positioning means saying both halves out loud. A vendor that only tells you the strong half is selling you the gap.

What this means for real-time meetings

Live multilingual conversation raises the bar past document translation. There's no time for post-editing, the input is messy spoken language, and the cost of a misunderstanding lands in the moment — in a negotiation, a support call, a clinical conversation. An "average accuracy" figure is the wrong lens entirely; what matters is whether tone, intent, and your specific terminology survive.

That's why InterMIND is built so each participant speaks their own language and is understood in real time, with the things that actually determine quality made checkable rather than asserted:

• Customer-controlled glossaries — you decide how your product names, legal terms, and domain vocabulary are rendered, instead of hoping a general model guesses right.
• An audit trail — what was said and how it was translated is reviewable after the fact, so quality is something you can inspect, not just trust.
• EU / neutral infrastructure and on-premise deployment — where the words are processed is your decision, which matters as much as the translation itself for regulated work.

A short checklist for evaluating any translation vendor

1. Which metric is the claim based on, and on which public test set?
2. Can the result be reproduced by a third party?
3. How does it perform on your language pairs and your content — not the vendor's averages?
4. Can you control terminology (glossaries), or are you stuck with the default?
5. Is there an audit trail to review quality after the fact?
6. Where is your data processed and stored, and is it used to train models?

If a vendor can answer these, the conversation is real. If the only answer is a percentage on a slide, you've learned something too.