Designing a Better Read-Aloud Screen for Any Language

DEV Community

Building a read-aloud screen: phrases, tappable words and a finger that drifts

Background A lot of what I read is still on paper: letters from the bank, school handouts, a recipe, a library book. I often want to listen to them instead, and I want to follow the words while I listen, so my eyes have something to hold on to. So I built Outloud Reader. You take a photo of a printed page (or open a PDF, or paste text), the app reads it out loud, and the phrase being spoken is highlighted on screen. It runs on iOS and Android, and it can translate a page before reading it. This post is about the reading screen, the part people look at for minutes at a time. It turned out to be the hardest screen in the app to get right. What I wanted from it: the current phrase highlighted, the current sentence easy to find, the rest of the page calm; tap any word to jump the audio there; a "just listen" mode with the text hidden, which you can still peek at; Japanese, Chinese and Thai behaving like any other language. What I tried Word-by-word highlighting, like karaoke. It is still in the code as an option, but it was exhausting to watch. The highlight jumped several times a second, and on a phone the eye kept chasing it instead of reading. A UIKit text view. That was the original plan, since it gives you text selection and word hit-testing for free. But the highlight needed to look a specific way, and I kept fighting the framework over small visual details. A long press on the text itself to peek at hidden text. On a real device it often did nothing at all. More on that below. How I did it 1. Highlight phrases, not words The unit of highlighting is a phrase: a few words up to the next punctuation mark, at most four of them. The phrase sits on a soft yellow plate, the current sentence is in full black, and the rest of the page is greyed out. Grouping is a simple pass over the words of a paragraph: private static let phraseBreaks: Set = [",", ".", ";", ":", "!", "?", "—", "…"]

var phraseID = 0
var wordsInPhrase = 0
for word in paragraph.words {
word.phraseID = phraseID
wordsInPhrase += 1
// A phrase ends on a punctuation mark or after four words.
if let last = word.text.last, phraseBreaks.contains(last) || wordsInPhrase >= 4 {
phraseID += 1
wordsInPhrase = 0
}
}
Punctuation always stays attached to its word, so checking the last character is enough. One extra rule I added after the first tests: a quoted line of dialogue counts as its own sentence. Otherwise the whole "Come here," she said, "and look." turned black at once, and the "current sentence" covered half the page. 2. Every word is its own view To make every word tappable, I dropped text views entirely. The paragraph is a custom SwiftUI Layout that wraps separate word views into lines, like a paragraph, except each word can be hit with a finger: struct WordFlowLayout: Layout {
var lineSpacing: CGFloat

func sizeThatFits(proposal: ProposedViewSize, subviews: Subviews, cache: inout ()) -> CGSize {
    let width = proposal.replacingUnspecifiedDimensions().width
    return geometry.size(for: subviews.map { $0.sizeThatFits(.unspecified) }, availableWidth: width)
}

func placeSubviews(in bounds: CGRect, proposal: ProposedViewSize, subviews: Subviews, cache: inout ()) {
    let frames = geometry.frames(for: subviews.map { $0.sizeThatFits(.unspecified) }, availableWidth: bounds.width)
    for (index, frame) in zip(subviews.indices, frames) {
        subviews[index].place(at: CGPoint(x: bounds.minX + frame.minX, y: bounds.minY + frame.minY),
                              proposal: ProposedViewSize(frame.size))
    }
}

}
The line-breaking arithmetic lives in a separate plain struct (geometry), so I can unit-test it without rendering anything. The highlight is just the background of each word in the phrase. That gave me a bug I didn't expect: thin white hairlines between the words of a highlighted phrase. The plates sat edge to edge, and the seams showed. The fix is to let neighb...

Powered by Readybot.io