Independent, native-speaker evaluation of speech recognition, text-to-speech and voice agents on Saraiki, Pakistani Punjabi and Urdu speech your model has never heard.
Each engagement is scoped to the model you actually deploy, through your API or your own output files.
Measure word and character errors on regional accents, dialect vocabulary and noisy real-world audio.
Test whether your agent understands what callers mean, not only what they say.
Urdu–English switching inside one sentence is normal speech. We test it directly.
Native listeners rate pronunciation, naturalness and intelligibility of generated speech.
Native raters judge LLM answers for correctness, fluency and cultural fit in Urdu and Saraiki.
Run the same private test set on every new model version to catch silent regressions.
Each failed utterance is labelled with the reason it failed, so your team can prioritise fixes.
| Error class | What it means | Example (illustrative) |
|---|---|---|
| Dialect substitution | A regional word is replaced by the nearest Urdu or Hindi word | Saraiki wanjna → jana |
| Code-switch failure | English words inside Urdu speech are misspelt or dropped | reschedule → re shedule |
| Noise-induced error | Background sound causes missing or invented words | market noise → [inaudible] |
| Segmentation | Word boundaries split or merged incorrectly | karanga → kar anga |
| Meaning-critical | Transcript looks close but the intent changes | kal (tomorrow) ↔ aj (today) |
| Omission / insertion | Words are missing or added | dropped negation nahin |
We agree languages, speaker mix, recording conditions and metrics with you.
Consented speech is recorded and transcribed by native speakers, then double-checked.
We run your API, or you send outputs. Your model stays yours.
Errors are classified, scored and explained in a written report.
Every report includes methodology, limitations and examples, so results can be reproduced.
Illustrative example of our format. Live consented samples publish at launch.
A fixed-scope pilot across Saraiki, Pakistani Punjabi and Urdu with a full error report. You keep the findings.