Photo translator: from the camera or a file

"Translate with camera" opens a viewfinder: the frame is recognized in the browser, and the translation is drawn as plates over the lettering. One pass costs roughly 0.13 s on a device with WebGPU and about 0.29 s on plain wasm, plus the translation of any new strings — captions already seen come back with no wait. If a pass takes longer than 2.6 s, continuous scanning switches itself off and is replaced by a "Read the frame" button: on such a device the frame changes before the translation lands on it. With automatic detection the source language is decided once, from the first frame that was read. "Capture frame" keeps the current image and hands it to the normal path, where the translation is set into the picture itself.

Before the first read the browser downloads the model: 4.5 MB for finding text, plus 7.5–15.8 MB for the chosen script. The size is shown before the download starts. After that the model comes from cache and later photos start immediately. Switching to a different script adds only the new recogniser — text detection is shared by all of them.

Usually it is the size of the lettering: if a stroke is thinner than a couple of pixels there is nothing to read. A closer shot or a higher resolution fixes it. The second common cause is "detect automatically" on a two-word caption — there is nothing for detection to work from, so set the language by hand. The third is a steep angle: text tilted more than about forty degrees reads noticeably worse.

The type is resized and wrapped to fit the space the original occupied — German and Russian typically run about a third longer than English. If it came out too small, shorten the translation by hand: tap the caption on the photo itself, or its row in the fragment list — the edit redraws the picture straight away.

Yes, and screenshots are where recognition is at its most accurate — even type, no glare, no perspective. Messages are read as separate fragments, so the conversation stays line by line instead of collapsing into one paragraph.

Japanese is read by the Chinese model: they share kanji, which come out accurate, while hiragana and katakana do worse because that model was never shown them specifically. A dedicated Japanese model would add about 13 MB to the download; if Japanese turns out to be in demand, it will be added.

It is covered with a color sampled from the ring of pixels around it, which is why the join is invisible on the flat background of a sign or a document. On a busy background the fill is feathered towards its edges and the translation is given a thin outline in the background color so it stays readable. The original is kept either way — the "Show original" switch brings the untouched photo back.

A menu, a sign or a manual is made of short text blocks, and one by one they are translated literally: “Grilled” and “salmon” become two separate words instead of a dish. In precise mode all the text blocks of the photo go to a language model as one task, in reading order, and it translates them as connected text. For this the recognized text is sent to our server and to the model (the recipients are listed on the AI providers page); the photo itself is recognized in the browser.