# DubEval

> DubEval compares a dubbed video with its original. It checks timing, speech, voice and translation, puts every problem on a timeline, and tells you which of several dubs to deliver, and why.

DubEval is free and open source (AGPL-3.0-or-later). It is a Windows 10/11 desktop app (latest version 0.1.1, released 2026-10-27); the command line, batch mode, REST API and Docker image run on Windows, macOS and Linux. Every audio and video check runs on the user's computer; only the optional translation check sends text (the two transcripts) to the AI the user chooses.

## Get it
- [Download for Windows (DubEval-Setup.exe, 505 MB)](https://github.com/shox17/DubEval/releases/latest/download/DubEval-Setup.exe)
- [Source code on GitHub](https://github.com/shox17/DubEval)
- [Install guide](https://github.com/shox17/DubEval/blob/main/INSTALL.md)
- [Release notes](https://github.com/shox17/DubEval/releases)

## What it checks
- **Same video**: Confirms the dub is the same video as the original, corrects a time shift, and finds parts cut out of the dub.
- **Timing**: Does the dub speak when the original speaks? Each line’s length is compared, and lines that run into the next one or leave long silences are flagged.
- **Speech**: Right language (and not the original voice left in), clear speech, and speaking pace for Korean, English, Spanish, Japanese and Chinese.
- **Voice**: Does the dub sound clean, not robotic or distorted, next to the original at the same moment? For cloned voices: does it still sound like the speaker?
- **Meaning**: Every line scored 1 to 10 for meaning, natural wording, context, terms and localization, with a suggested fix. Uses Claude, Gemini or ChatGPT with your key, or a local model.
- **Sound and file**: Sudden loudness jumps, clipping and distortion, and a picture or audio track that changed.
- Lip sync is shown for information only in this beta and never changes the verdict.

## How results are shown
- There is no single 0–100 score on purpose: weights between checks would be arbitrary. DubEval gives each dub one of three verdicts, and shows the reason behind it.
- **Good**: Every automatic check passed.
- **Needs review**: Something should be looked at by a person. The report says where.
- **Poor**: At least one check failed clearly, such as a long silence or the wrong language.
- Every problem has a time: click it to play both videos at that moment.
- Words the speech recognizer probably misheard are marked and never lower a grade.
- Mark each problem as a real problem or a false alarm, so you can see which checks to trust.
- Export a report, a problem list, a summary or the raw data (HTML, CSV, text, JSON).

## Compare dubs
Dubbed the same video with different services, voices or vendors? Add 2 to 8 dubs and get a ranking and a recommendation, with the reason in plain words.
- The ranking rule is fixed and printed, so the same dubs always give the same answer.
- Every check side by side, and a full review of each dub.
- A warning when the winner still couldn’t be measured on key checks.

## How accurate it is
DubEval measures; it doesn’t replace watching the dub. Use the verdict to decide where to look first. Here is how it has done so far, honestly.
- **5 of 6** English → Korean dubs where the verdict matched a person’s. Korean is the language pair with enough checks to trust.
- **6 of 12** across every dub graded by a person so far (Korean, Uzbek, Spanish). Other language pairs work, but treat them as a first look.
- **Never** does a likely misheard word lower a grade. Possible mishearings are marked and kept out of the verdict.

## Who it is for
- **Localization and content teams**: Check every language before release without watching every minute.
- **Dubbing studios and vendors**: Catch problems before delivery and send clients a report with times and reasons.
- **AI dubbing teams**: Run batches and catch regressions when you change a model, a voice or a pipeline.
- **Creators going multilingual**: Try several AI dubbing services on one video and see which dub to publish.

## FAQ
### What is DubEval?
DubEval is a free, open-source tool that checks the quality of a dubbed video by comparing it with the original. It measures timing, speech, voice and translation, shows every problem as a time range on a timeline, and gives a verdict: Good, Needs review or Poor. It evaluates dubs; it doesn’t make them.

### Is DubEval free?
Yes. DubEval is free and open source under the GNU AGPL-3.0 license. There is no account and no subscription. The optional translation check uses your own Claude, Gemini or ChatGPT key, billed by that provider, or a free local model.

### Are my videos uploaded anywhere?
No. Every audio and video check runs on your computer. Only the optional translation check sends text, the two transcripts, to the AI you choose; with the local model, nothing leaves your computer.

### Which dubbing services does it work with?
Any. DubEval only needs the original video and the dub, so it works with dubs from any AI dubbing service and with voice-actor dubs. You can choose files, paste YouTube, TikTok or Instagram Reels links, or paste a Perso share link, which already includes the original.

### Which languages are supported?
Any language pair the Whisper speech recognizer knows. Speaking pace is graded for Korean, English, Spanish, Japanese and Chinese. A check that can’t work for a language says “not measured” with the reason instead of guessing. English → Korean is the most tested pair so far.

### Does it check lip sync?
Only roughly, for now. In this beta, lip movement is shown for information and never changes the verdict. A proper lip-sync check is planned after the first version.

### Can I compare dubs from different services or vendors?
Yes. Add 2 to 8 dubs of the same video. You get a ranking, the dub to deliver and the reason. The rule is fixed and printed, so the result is repeatable.

### What do I need to run it?
A 64-bit Windows 10 or 11 PC with about 2 GB of free disk. An NVIDIA graphics card is optional: it makes speech recognition faster and enables a more accurate speech model. On macOS and Linux, run it from source or in Docker.

### Windows says “Windows protected your PC”. Is it safe?
The beta isn’t code-signed yet, so Windows shows this warning for any new unsigned app. Click More info, check that the file is DubEval-Setup.exe, then Run anyway. The source code is public on GitHub.

### Does it replace human reviewers?
No. It tells reviewers where to look. Instead of watching every minute in every language, they spend their time on the moments that were flagged.

## Documentation
- [How to use it](https://github.com/shox17/DubEval/blob/main/docs/usage.md)
- [Every check and how it is graded](https://github.com/shox17/DubEval/blob/main/docs/metrics.md)
- [Accuracy, limits and troubleshooting](https://github.com/shox17/DubEval/blob/main/docs/faq.md)
- [Privacy: what is sent, and when](https://github.com/shox17/DubEval/blob/main/docs/privacy.md)

## Pages
- [English](https://dubeval.com/)
- [한국어](https://dubeval.com/ko/)
- [Español](https://dubeval.com/es/)
- [Português (BR)](https://dubeval.com/pt/)
- [Русский](https://dubeval.com/ru/)
- [Bahasa Indonesia](https://dubeval.com/id/)
- [Deutsch](https://dubeval.com/de/)
- [ไทย](https://dubeval.com/th/)
- [日本語](https://dubeval.com/ja/)
- [繁體中文](https://dubeval.com/zh/)

## Community
- [Discord](https://discord.gg/AA7P3zPqY)
- [GitHub issues](https://github.com/shox17/DubEval/issues)