How translating text inside an image actually works

Four steps, not one

Translating an image is not "OCR plus a translation API". Four separate problems have to be solved, and each one can ruin the result.

1. Recognition

The engine returns lines, not paragraphs. Lines are grouped back into blocks by measuring vertical gaps against line height and horizontal overlap, and columns are found by looking for vertical gutters that no text crosses.

2. Understanding the layout

A heading is not just bigger text — it is bigger *relative to the body text*, and short. We compute the body size as a length-weighted median so that a two-word title cannot drag the average up and hide itself.

3. Removing the original text

Flat backgrounds are filled with the sampled paper colour, which is perfect. Textured backgrounds go through inpainting. Photographic backgrounds get a translucent plate, because a smeared repair looks worse than an honest one.

4. Typesetting the translation

German is often 35% longer than English; Chinese is roughly half the width. Font size is binary-searched against real glyph metrics — never an average character width — and we warn you when text genuinely does not fit.